How AI is transforming economic research and analysis
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Sarah Grillo/Axios
AI tools have become shockingly good at carrying out much of the work produced by the economic profession — from the most elite halls of academia to think tanks, government agencies, consultancies, research shops and humble newsroom economics teams.
The big picture: This fact raises important — and, we confess, unnerving — questions about the role of humans in making sense of the economy.
- It's hard to escape the sense that we are in an uncomfortable moment in which the capabilities of AI models to do economic research and analysis have far outpaced our understanding of how best to deploy and oversee it.
- "Every conversation I have with a colleague at Harvard or a think tanker in D.C. is some combination of excitement, angst and confusion," Harved economist Jason Furman tells Axios.
State of play: Furman, who was the top White House economist under President Obama and now co-teaches Harvard's signature introductory economics course, has created a site called Econvitals, which went live Friday, with a wide array of data and analysis.
- No single component of the site is earth-shatteringly new or cutting-edge. But neither is most economic analysis.
- What is stunning is that this is a site that a busy professor produced in his spare time, with a little help from research assistants and a lot of help from Claude (with some cross-checking by OpenAI's Codex).
- It offers, for free, the kind of expansive number-crunching and analysis that used to require thousands of person-hours and a seven-figure budget to produce.
Zoom in: For example, the site includes AI analysis of speeches by Federal Reserve officials that shows how hawkish or dovish each official is, and how their views have evolved over time.
- Not rocket science, certainly — but it is the kind of work Fed watchers at banks and consulting shops have done in more artisanal fashion for ages, in service to paying clients.
- It produces email notes summarizing interesting economic research and analyzing the latest data that may not have quite the human touch of the better email tip sheets (ahem), but are accurate, efficient and free.
- Furman shows samples of economic papers crafted entirely by AI, using short prompts. It's not the kind of output that would get a young economist tenure, but they are remarkably competent given how little human effort was involved.
What they're saying: "In terms of factual reliability, they are at this stage, if done properly, better than any human," Furman says. "If you have fact-checking agents, what comes out is completely reliable. The hallucination idea is a combination of outdated or incompetent usage."
Yes, but: "It is still not as creative and exciting as the best humans," he adds, and is better at replacing routine analysis than generating groundbreaking ideas. That said, the definition of what counts as routine keeps changing with each new version of the frontier models.
- "It's really exciting how much AI can do for us and empower all of us," he said. "Partly it's scary. I find it is able to do a lot of what I do better than I do it, and I thought I was decently good at what I did, and that can be dispiriting at times."
- "My hope for economic research is we don't see increase in quantity but in quality. Any idiot could write 300 papers a year now, and they'd be fine and factually correct. But I'm hoping that people don't write more papers but write better papers," he said.
The intrigue: Furman says that if he were running the White House Council of Economic Advisers today, he would heavily rely on AI for analysis.
- "If I were running the CEA today and had a choice between 20 staff members who had a combination of current Ph.D.s and future Ph.D.s versus the AI systems that we have today, I would somewhat regretfully and sadly choose the AI systems," he said.
- "You would need people at CEA to interact in meetings, to talk to the press, talk to Congress. There's a lot of things only humans can do, but that was a fraction of what we did at CEA. For the bigger fraction of it, at this point I think machines are faster and more reliable than humans."
The bottom line: So what does Furman tell the undergraduates he teaches, particularly those who aspire to research or shape economic policy?
- "I think people need to figure out how they can have value add and can do something distinctive and unique. Repackaging and reinterpreting data and numbers is not that unique and exciting compared to other things people can do."
