AI wasn't overhyped. Almost everyone underestimated it

A forecaster who published a March 2025 piece arguing for a significant chance of AGI by 2030 has concluded that he and nearly everyone else underestimated AI's pace of progress. Revisiting his predictions, he found that AI hit 40% on FrontierMath by the end of 2025 and 94% by September 2026, versus his forecast of 32% by end-2026, and that Anthropic's roughly $65bn annualized revenue in September 2026 far exceeded the $38bn projected for that date in the AI 2027 scenario. Only AI company leaders were roughly right, and only slightly too optimistic, he writes.

The usual complaint about AI is that it’s overhyped. But over the last 18 months, almost everyone made the opposite mistake: they expected AI to progress more slowly than it did. Superforecasters, the people with the best track record at predicting geopolitical events, underestimated it most, as did the general public. AI experts did better, but were still too slow. Mostly, only AI company leaders were too optimistic, and only a little. If you’d assumed what they said would happen in six months would actually happen in six to 18, you’d have done pretty well. I made the same mistake. In March 2025 I published “ Will we have AGI by 2030? https://benjamintodd.substack.com/p/the-case-for-agi-by-2030 ”, arguing there was a significant chance we would. Many accused the piece of buying into AI CEO hype. Having revisited it, I think the opposite: I underestimated the rate of progress. Here’s an overview of what held up, what didn’t, and whose forecasts look best overall. My forecasts were better than most, but too slow The core claim of the original article https://web.archive.org/web/20260215082324/https://80000hours.org/ai/guide/when-will-agi-arrive/ was that the four drivers of progress pretraining, RL, test-time compute and agent scaffolding would all keep working through 2028 and so far they have . I was especially proud to have called agents “the next stage of scaling”. However, I also gave some numbers for what a “continuation of trend” would imply for capabilities by the end of 2026, and these were too slow: I also wrote “the next leap after 2026 might take us into beyond-human-level problem solving.” Arguably on maths, we’re already there. These weren’t formal forecasts, but I’d say about three were on track and three or four were too slow. In August 2025 I joined the LEAP forecasting panel https://leap.forecastingresearch.org/ , where I did make formal if rushed forecasts. I estimated AI would reach 32% in FrontierMath by the end of 2026, and 97% in 2030. I was more bullish than most, but AI was far ahead: 40% at the end of 2025 and 94% by September 2026, more than three years early. LiveCodeBench Pro was similar: my forecast of 23% by end of 2026 was well above average among experts, superforecasters and the public, but AI is way ahead, hitting 53% by the end of 2025. On the 80% METR time horizon https://metr.org/time-horizons/ , most expected 3h by the end of 2026, but that was already hit in April. I did better here, predicting 6h by year end ~5-9h range , which seems on track. Most expected 4% of work tasks to be AI-assisted by the end of 2025, when it was actually 5.7%. However, it’s within my interval and my forecast of 12% by 2027 could still be on track. AI revenue forecasts have also already been beaten. For instance, superforecasters thought OpenAI and Anthropic would have $90bn annualised revenue by the end of 2026, but it was already $100bn by July https://epoch.ai/data/ai-companies?view=graph&tab=revenue&showTotal=true . Even in AI 2027 https://ai-2027.com/ , a scenario representing an especially aggressive near-term AGI takeoff, they forecast that the largest AI company would be earning $38bn annualised in September 2026, when Anthropic was earning around $65bn. Overall things have moved 70-90% as quickly as in AI 2027 https://blog.aifutures.org/p/q25-2026-timelines-update-uplift , making it one of the most accurate. AI 2027 represented a modal forecast rather than a median, and their mode is earlier. Revenue is the only forecast where I might have ended up overoptimistic, with a median prediction of $225bn frontier revenue by the end of 2026 though within my confidence interval . In total, three of my forecasts were within my interval, and the two misses were both much too slow. I also agreed with most forecasters that we wouldn’t see large effects on GDP and employment by now and we haven’t , saying “most of the economy will probably continue pretty much as normal for a while.” A big reason my earlier forecasts especially were too slow was that in early 2025, it was unclear whether the faster post-2024 trend https://epoch.ai/data-insights/ai-capabilities-progress-has-sped-up would continue or revert to the mean. My forecasts were often halfway between the two, but actually we’ve been on the faster trend. Nor did I put enough weight on further acceleration. I also didn’t fully appreciate how steady progress on a log curve means that a jump in benchmark scores from 2% to 4% is the same “size” as one from 10% to 20%, rather than 10% to 12%. By the end of 2025 I’d corrected both of these mistakes and my forecasts were more accurate after that. Almost no-one predicted the progress in maths In a Dec 2024 survey https://aiimpacts.org/wp-content/uploads/2026/09/ESPAI2024.pdf of over 1,000 AI researchers who had published at top conferences, the median estimate was that AI wouldn’t solve an unsolved maths problem, like a Millennium problem, until about 2054. In fact, AI seems to have solved a Millennium Prize problem https://openai.com/index/navier-stokes-solution/ in September 2026. The result isn’t formally validated yet, which could take several years. However, by October, AI had solved hundreds more important open problems https://www.economist.com/science-and-technology/2026/10/07/openai-claims-to-have-solved-hundreds-of-long-standing-maths-problems . One of the few people who predicted this was Eric Schmidt in July 2025, a forecast Gary Marcus said was “ pure fantasy https://x.com/codytfenwick/status/2107928931731128363 ”. My own estimate was a 7% chance by Dec 2027, woefully off. I didn’t expect Epoch’s FrontierMath to saturate so fast, and I also thought there might be a larger gap between completing difficult problems with known answers, like those in FrontierMath, and having genuinely novel insights. In fact, saturating FrontierMath led to novel results almost right away. Epoch themselves gave a better indication of likely progress on their benchmarks than almost anyone else, but were still a little too slow https://x.com/ben j todd/status/2098319679320453594 . Progress hasn’t only been rapid in maths Capabilities have advanced furthest compared to humans in maths and coding, and it’s what I expected https://80000hours.org/career-guide/most-valuable-skills/ 1-skills-that-are-hard-to-automate given reinforcement learning RL works better for verifiable tasks. However, I think progress has also been rapid elsewhere. People most often say AI is lagging on physical tasks. But the chart above also shows AI researchers didn’t expect a bipedal robot to beat the fastest human in a 5km race until around 2034, yet in April a robot beat the human record for half marathons https://www.wired.com/story/a-humanoid-robot-set-a-half-marathon-record-in-china/ , as well as the 100m sprint https://www.youtube.com/watch?v=FGBLzMESBAo . And on the level of vibes, GPT-Astra seems to be better than expected at robotics, drone piloting and driving cars https://substack.com/@benjamintodd/note/c-343961101 . Even in white collar work, there appears to be rapid progress in harder-to-verify tasks. One example is that in March 2026, a New York Times quiz found https://smarterx.ai/smarterxblog/nyt-ai-writing-quiz-when-use-ai the typical respondent preferred AI writing even to that produced by legendary literary authors, like Cormac McCarthy and Ursula K. Le Guin. I give more data in the updated article https://benjamintodd.substack.com/p/the-case-for-agi-by-2030 %C2%A7how-far-can-scaling-reasoning-models-continue . Ultimately, AI company revenue is the best measure of whether AI has real economic application, and it’s also grown about 6x a year https://epoch.ai/data/ai-companies?view=graph&tab=revenue&showTotal=true ahead of what almost anyone expected except perhaps for employees of the companies . In harder-to-verify skills, it’ll take more RL to reach the human expert level. But while the journey is longer, the speed of travel seems similar. Superforecasters look worst; AI company CEOs look prescient but optimistic Superforecasters have consistently expected slower progress than everyone else, so collectively have the worst track record. This is clearly shown in the recent LEAP panel https://forecastingresearch.org/research/ai-progress-accuracy-update , but goes back to at least 2022, when superforecasters predicted https://forecastingresearch.org/research/existential-risk-persuasion-tournament compute used on the largest AI experiment would be $35m by 2024 and $100m by 2030, when in fact it was already likely over $100m by 2024 https://epoch.ai/publications/how-much-does-it-cost-to-train-frontier-ai-models , four years earlier. Why did the best forecasters do so badly? I’d speculate it’s because the best way to predict short-term geopolitical events is to assume things will stay the same, or at least remain on trend, while AI is highly trend-breaking. An incorrect implication of all this is that “expert views are useless”. Experts in AI were better at forecasting progress than the general public. And the main way experts have been wrong is not expecting progress to be even faster . In other words, they’ve been under hyping AI, rather than hyping it. The AI company leaders, the people most accused of hype, are among the few who’ve erred optimistic. We don’t have many quantified predictions, but the ones that have resolved mostly came true, just sometimes later than suggested. Here are the ones I could find: - Jan 2025: Altman said https://blog.samaltman.com/reflections agents would join the workforce in 2025. They trickled in but took off in early 2026. - Jun 2025: Altman said 2026 will see AI that can have novel insights https://blog.samaltman.com/the-gentle-singularity correct - Oct 2025: Altman said an intern-level automated AI researcher would arrive by Sept 2026. OpenAI says https://www.techradar.com/pro/openai-says-it-has-built-an-automated-research-intern-to-carry-out-menial-tasks-and-its-only-just-getting-started it landed Sept 2026 self-graded, but seems close - Mar 2025: Dario said https://www.businessinsider.com/anthropic-ceo-ai-90-percent-code-3-to-6-months-2025-3 AI would write 90% of code in 3–6 months. Anthropic’s own codebase hit 80% after 14 months https://www.thestateofai.com/news/anthropic-80-percent-code-not-engineers too optimistic . At the broadest level, people at the companies have consistently said that scaling isn’t hitting a wall, that more powerful capabilities will be here soon, and they’ve been right. Today, all three leaders seem to be saying pretty broad beyond-human AGI will be here in 1-3 years. Rather than dismissing what the companies tell us as “hype”, talking about stochastic parrots https://x.com/ben j todd/status/2112229949071344036 , or continually pointing out “fundamental” limitations which get overcome months later https://benjamintodd.substack.com/p/gary-marcus-says-ai-cant-do-things , the best AI forecasters take what the companies say seriously, but tend to assume things will take just a bit longer. For instance, Ryan Greenblatt was ranked second of 400 in the 2025 AI forecasting competition https://ai2025.org/ , and his forecasts often have this character. As does Daniel Kokotajlo, whose 2021 “ what 2026 looks like https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-looks-like ” is one of the most prescient things ever written about AI. Situational Awareness https://situational-awareness.ai/racing-to-the-trillion-dollar-cluster/ AI revenue , Leopold Aschenbrenner’s 2024 essay, which built on insider access to OpenAI before 2024 and the views of Carl Shulman https://www.dwarkesh.com/p/carl-shulman , also looks prescient. It erred a little slow on investment, and a little optimistic about revenue, but overall feels on track. The best forecasters I know https://www.aifuturesmodel.com/forecast/daniel-08-16-26?timeline=AC%2CTED-AI all https://situational-awareness.ai/from-gpt-4-to-agi/ expect https://x.com/RyanGreenblatt/status/2097839501309861952 beyond-human AI across the board in 2028-2029 as their median . I think there’s a good chance they’re right, though I’d put my median a shade later, around 2030. In the updated article https://benjamintodd.substack.com/p/the-case-for-agi-by-2030 , I’ve moved a little away from the “by 2030 or bust” framing and put more weight on a third scenario, where progress gradually slows after 2028 and AGI arrives in the early 2030s. 1 footnote-1 The difference isn’t very decision-relevant: either way, we should be planning for AGI within a few years. Even the people building AI keep getting surprised I find it shocking that five years into the AI boom, even people who are highly informed and bullish on rapid progress can underestimate it so badly. I was as aware as anyone that AI has been surprising for years https://carnegieendowment.org/research/2025/01/ai-has-been-surprising-for-years , and I tried to project the existing trends forward, but was still wrong-footed by the acceleration in maths and agents. With the Hugging Face incident, even OpenAI’s own staff claimed the agent’s coordination skills “ surprised the fuck out of us https://x.com/ben j todd/status/2104598707542921606 ”. Most discussions of how to adapt to AI implicitly assume the capabilities are fixed, or a little ahead of today. Instead, we need to plan for not only continued rapid progress, but also progress that’s even faster than expected. AI is just extremely hard to predict, but that doesn’t mean it’ll come slow – it means it could come much faster than we expect. What should be done? After the Hugging Face incident https://80000hours.org/hugging-face/ , AI safety went https://www.politico.com/news/2026/09/16/poll-ai-technology-risks-humanity-trump-voters-01078087 mainstream https://www.fastcompany.com/91616915/over-70-of-americans-are-concerned-about-ais-existential-risks-polling-shows . Dario Amodei published We Must Pace the Frontier https://darioamodei.com/post/we-must-pace-the-frontier , and Altman and Musk agreed https://www.itv.com/news/2026-09-12/musk-and-altman-back-calls-to-slow-down-development-of-ai with it. In light of these developments, I updated my post outlining 11 key interventions on AI risk https://benjamintodd.substack.com/p/ai-risk-interventions . I put pacing at the top: slowing down AI, especially before recursive self-improvement starts, is the biggest thing we can do to reduce a wide range of risks. I also beefed up several other sections. 1 footnote-anchor-1 Other updates to the article: I underweighted how compute shifting into RL would slow the pretraining trend, but maybe overemphasised test-time compute too much it mainly brings capabilities forward 1-2 years, so isn’t a driver in the same way as the other three . I didn’t emphasise enough the possibility that a new driver or paradigm will be discovered between now and 2030, although it seems quite likely to me especially if LLMs hit stronger diminishing returns .