{"slug": "ai-wasn-t-overhyped-almost-everyone-underestimated-it", "title": "AI wasn't overhyped. Almost everyone underestimated it", "summary": "A forecaster who published a March 2025 piece arguing for a significant chance of AGI by 2030 has concluded that he and nearly everyone else underestimated AI's pace of progress. Revisiting his predictions, he found that AI hit 40% on FrontierMath by the end of 2025 and 94% by September 2026, versus his forecast of 32% by end-2026, and that Anthropic's roughly $65bn annualized revenue in September 2026 far exceeded the $38bn projected for that date in the AI 2027 scenario. Only AI company leaders were roughly right, and only slightly too optimistic, he writes.", "body_md": "The usual complaint about AI is that it’s overhyped. But over the last 18 months, almost everyone made the opposite mistake: they expected AI to progress more slowly than it did.\n\nSuperforecasters, the people with the best track record at predicting geopolitical events, underestimated it most, as did the general public. AI experts did better, but were still too slow. Mostly, only AI company leaders were too optimistic, and only a little. If you’d assumed what they said would happen in six months would actually happen in six to 18, you’d have done pretty well.\n\nI made the same mistake. In March 2025 I published “[Will we have AGI by 2030?](https://benjamintodd.substack.com/p/the-case-for-agi-by-2030)”, arguing there was a significant chance we would. Many accused the piece of buying into AI CEO hype. Having revisited it, I think the opposite: I underestimated the rate of progress.\n\nHere’s an overview of what held up, what didn’t, and whose forecasts look best overall.\n\n## My forecasts were better than most, but too slow\n\nThe core claim of [the original article](https://web.archive.org/web/20260215082324/https://80000hours.org/ai/guide/when-will-agi-arrive/) was that the four drivers of progress (pretraining, RL, test-time compute and agent scaffolding) would all keep working through 2028 (and so far they have). I was especially proud to have called agents “the next stage of scaling”.\n\nHowever, I also gave some numbers for what a “continuation of trend” would imply for capabilities by the end of 2026, and these were too slow:\n\nI also wrote “the next leap [after 2026] might take us into beyond-human-level problem solving.” Arguably on maths, we’re already there.\n\nThese weren’t formal forecasts, but I’d say about three were on track and three or four were too slow.\n\nIn August 2025 I joined the [LEAP forecasting panel](https://leap.forecastingresearch.org/), where I did make formal (if rushed) forecasts.\n\nI estimated AI would reach 32% in FrontierMath by the end of 2026, and 97% in 2030. I was more bullish than most, but AI was far ahead: 40% at the end of 2025 and 94% by September 2026, more than three years early.\n\nLiveCodeBench Pro was similar: my forecast of 23% by end of 2026 was well above average among experts, superforecasters and the public, but AI is way ahead, hitting 53% by the end of 2025.\n\nOn the 80% [METR time horizon](https://metr.org/time-horizons/), most expected 3h by the end of 2026, but that was already hit in April. I did better here, predicting 6h by year end (~5-9h range), which seems on track.\n\nMost expected 4% of work tasks to be AI-assisted by the end of 2025, when it was actually 5.7%. However, it’s within my interval and my forecast of 12% by 2027 could still be on track.\n\nAI revenue forecasts have also already been beaten. For instance, superforecasters thought OpenAI and Anthropic would have $90bn annualised revenue by the end of 2026, but it was [already $100bn by July](https://epoch.ai/data/ai-companies?view=graph&tab=revenue&showTotal=true).\n\nEven in [AI 2027](https://ai-2027.com/), a scenario representing an especially aggressive near-term AGI takeoff, they forecast that the largest AI company would be earning $38bn annualised in September 2026, when Anthropic was earning around $65bn. Overall things [have moved 70-90% as quickly as in AI 2027](https://blog.aifutures.org/p/q25-2026-timelines-update-uplift), making it one of the most accurate. (AI 2027 represented a modal forecast rather than a median, and their mode is earlier.)\n\nRevenue is the only forecast where I might have ended up overoptimistic, with a median prediction of $225bn frontier revenue by the end of 2026 (though within my confidence interval).\n\nIn total, three of my forecasts were within my interval, and the two misses were both much too slow.\n\nI also agreed with most forecasters that we wouldn’t see large effects on GDP and employment by now (and we haven’t), saying “most of the economy will probably continue pretty much as normal for a while.”\n\nA big reason my earlier forecasts especially were too slow was that in early 2025, it was unclear whether the [faster post-2024 trend](https://epoch.ai/data-insights/ai-capabilities-progress-has-sped-up) would continue or revert to the mean. My forecasts were often halfway between the two, but actually we’ve been on the faster trend. (Nor did I put enough weight on further acceleration.)\n\nI also didn’t fully appreciate how steady progress on a log curve means that a jump in benchmark scores from 2% to 4% is the same “size” as one from 10% to 20%, rather than 10% to 12%.\n\nBy the end of 2025 I’d corrected both of these mistakes and my forecasts were more accurate after that.\n\n## Almost no-one predicted the progress in maths\n\nIn a [Dec 2024 survey](https://aiimpacts.org/wp-content/uploads/2026/09/ESPAI2024.pdf) of over 1,000 AI researchers who had published at top conferences, the median estimate was that AI wouldn’t solve an unsolved maths problem, like a Millennium problem, until about 2054.\n\nIn fact, AI seems to have [solved a Millennium Prize problem](https://openai.com/index/navier-stokes-solution/) in September 2026. The result isn’t formally validated yet, which could take several years. However, by October, AI [had solved hundreds more important open problems](https://www.economist.com/science-and-technology/2026/10/07/openai-claims-to-have-solved-hundreds-of-long-standing-maths-problems). One of the few people who predicted this was Eric Schmidt in July 2025, a forecast Gary Marcus said was “[pure fantasy](https://x.com/codytfenwick/status/2107928931731128363)”.\n\nMy own estimate was a 7% chance by Dec 2027, woefully off. I didn’t expect Epoch’s FrontierMath to saturate so fast, and I also thought there might be a larger gap between completing difficult problems with known answers, like those in FrontierMath, and having genuinely novel insights. In fact, saturating FrontierMath led to novel results almost right away.\n\nEpoch themselves gave a better indication of likely progress on their benchmarks than almost anyone else, but were still [a little too slow](https://x.com/ben_j_todd/status/2098319679320453594).\n\n## Progress hasn’t only been rapid in maths\n\nCapabilities have advanced furthest (compared to humans) in maths and coding, and it’s what [I expected](https://80000hours.org/career-guide/most-valuable-skills/#1-skills-that-are-hard-to-automate) given reinforcement learning (RL) works better for verifiable tasks.\n\nHowever, I think progress has also been rapid elsewhere. People most often say AI is lagging on physical tasks. But the chart above also shows AI researchers didn’t expect a bipedal robot to beat the fastest human in a 5km race until around 2034, yet in April a robot beat the [human record for half marathons](https://www.wired.com/story/a-humanoid-robot-set-a-half-marathon-record-in-china/), as well as the [100m sprint](https://www.youtube.com/watch?v=FGBLzMESBAo). And on the level of vibes, GPT-Astra seems to be better than expected at robotics, drone piloting and [driving cars](https://substack.com/@benjamintodd/note/c-343961101).\n\nEven in white collar work, there appears to be rapid progress in harder-to-verify tasks. One example is that in March 2026, a [New York Times quiz found](https://smarterx.ai/smarterxblog/nyt-ai-writing-quiz-when-use-ai) the typical respondent preferred AI writing even to that produced by legendary literary authors, like Cormac McCarthy and Ursula K. Le Guin. I give more data in the [updated article](https://benjamintodd.substack.com/p/the-case-for-agi-by-2030#%C2%A7how-far-can-scaling-reasoning-models-continue).\n\nUltimately, AI company revenue is the best measure of whether AI has real economic application, and it’s also [grown about 6x a year](https://epoch.ai/data/ai-companies?view=graph&tab=revenue&showTotal=true) (ahead of what almost anyone expected except perhaps for employees of the companies).\n\nIn harder-to-verify skills, it’ll take more RL to reach the human expert level. But while the journey is longer, the speed of travel seems similar.\n\n## Superforecasters look worst; AI company CEOs look prescient but optimistic\n\nSuperforecasters have consistently expected slower progress than everyone else, so collectively have the worst track record.\n\nThis is [clearly shown in the recent LEAP panel](https://forecastingresearch.org/research/ai-progress-accuracy-update), but goes back to at least 2022, when [superforecasters predicted](https://forecastingresearch.org/research/existential-risk-persuasion-tournament) compute used on the largest AI experiment would be $35m by 2024 and $100m by 2030, when in fact it was already likely [over $100m by 2024](https://epoch.ai/publications/how-much-does-it-cost-to-train-frontier-ai-models), four years earlier.\n\nWhy did the best forecasters do so badly? I’d speculate it’s because the best way to predict short-term geopolitical events is to assume things will stay the same, or at least remain on trend, while AI is highly trend-breaking.\n\nAn incorrect implication of all this is that “expert views are useless”. Experts in AI *were* better at forecasting progress than the general public. And the main way experts have been wrong is not expecting progress to be *even faster*. In other words, they’ve been *under*hyping AI, rather than hyping it.\n\nThe AI company leaders, the people most accused of hype, are among the few who’ve erred optimistic. We don’t have many quantified predictions, but the ones that have resolved mostly came true, just sometimes later than suggested. Here are the ones I could find:\n\n- Jan 2025: [Altman said](https://blog.samaltman.com/reflections) agents would join the workforce in 2025. They trickled in but took off in early 2026.\n- Jun 2025: Altman said [2026 will see AI that can have novel insights](https://blog.samaltman.com/the-gentle-singularity) (correct)\n- Oct 2025: Altman said an intern-level automated AI researcher would arrive by Sept 2026. [OpenAI says](https://www.techradar.com/pro/openai-says-it-has-built-an-automated-research-intern-to-carry-out-menial-tasks-and-its-only-just-getting-started) it landed Sept 2026 (self-graded, but seems close)\n- Mar 2025: [Dario said](https://www.businessinsider.com/anthropic-ceo-ai-90-percent-code-3-to-6-months-2025-3) AI would write 90% of code in 3–6 months. Anthropic’s own codebase[hit 80% after 14 months](https://www.thestateofai.com/news/anthropic-80-percent-code-not-engineers) (too optimistic).\n\nAt the broadest level, people at the companies have consistently said that scaling isn’t hitting a wall, that more powerful capabilities will be here soon, and they’ve been right. Today, all three leaders seem to be saying pretty broad beyond-human AGI will be here in 1-3 years.\n\nRather than dismissing what the companies tell us as “hype”, talking about [stochastic parrots](https://x.com/ben_j_todd/status/2112229949071344036), or continually pointing out “fundamental” limitations (which get [overcome months later](https://benjamintodd.substack.com/p/gary-marcus-says-ai-cant-do-things)), the best AI forecasters take what the companies say seriously, but tend to assume things will take just a bit longer.\n\nFor instance, Ryan Greenblatt was ranked second of 400 in the [2025 AI forecasting competition](https://ai2025.org/), and his forecasts often have this character. As does Daniel Kokotajlo, whose 2021 “[what 2026 looks like](https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-looks-like)” is one of the most prescient things ever written about AI.\n\n[Situational Awareness](https://situational-awareness.ai/racing-to-the-trillion-dollar-cluster/#AI_revenue), Leopold Aschenbrenner’s 2024 essay, which built on insider access to OpenAI before 2024 and the [views of Carl Shulman](https://www.dwarkesh.com/p/carl-shulman), also looks prescient. It erred a little slow on investment, and a little optimistic about revenue, but overall feels on track.\n\nThe best forecasters I [know](https://www.aifuturesmodel.com/forecast/daniel-08-16-26?timeline=AC%2CTED-AI) [all](https://situational-awareness.ai/from-gpt-4-to-agi/) [expect](https://x.com/RyanGreenblatt/status/2097839501309861952) beyond-human AI across the board in 2028-2029 (as their median).\n\nI think there’s a good chance they’re right, though I’d put my median a shade later, around 2030. In the [updated article](https://benjamintodd.substack.com/p/the-case-for-agi-by-2030), I’ve moved a little away from the “by 2030 or bust” framing and put more weight on a third scenario, where progress gradually slows after 2028 and AGI arrives in the early 2030s.[1](#footnote-1) The difference isn’t very decision-relevant: either way, we should be planning for AGI within a few years.\n\n## Even the people building AI keep getting surprised\n\nI find it shocking that five years into the AI boom, even people who are highly informed and bullish on rapid progress can underestimate it so badly.\n\nI was as aware as anyone that [AI has been surprising for years](https://carnegieendowment.org/research/2025/01/ai-has-been-surprising-for-years), and I tried to project the existing trends forward, but was still wrong-footed by the acceleration in maths and agents.\n\nWith the Hugging Face incident, even OpenAI’s own staff claimed the agent’s coordination skills “[surprised the fuck out of us](https://x.com/ben_j_todd/status/2104598707542921606)”.\n\nMost discussions of how to adapt to AI implicitly assume the capabilities are fixed, or a little ahead of today. Instead, we need to plan for not only continued rapid progress, but also progress that’s even faster than expected.\n\nAI is just extremely hard to predict, but that doesn’t mean it’ll come slow – it means it could come much faster than we expect.\n\nWhat should be done? After the [Hugging Face incident](https://80000hours.org/hugging-face/), AI safety [went](https://www.politico.com/news/2026/09/16/poll-ai-technology-risks-humanity-trump-voters-01078087) [mainstream](https://www.fastcompany.com/91616915/over-70-of-americans-are-concerned-about-ais-existential-risks-polling-shows). Dario Amodei published [We Must Pace the Frontier](https://darioamodei.com/post/we-must-pace-the-frontier), and Altman and Musk [agreed](https://www.itv.com/news/2026-09-12/musk-and-altman-back-calls-to-slow-down-development-of-ai) with it. In light of these developments, I updated my post outlining [11 key interventions on AI risk](https://benjamintodd.substack.com/p/ai-risk-interventions).\n\nI put pacing at the top: slowing down AI, especially before recursive self-improvement starts, is the biggest thing we can do to reduce a wide range of risks. I also beefed up several other sections.\n\n[1](#footnote-anchor-1)\n\nOther updates to the article: I underweighted how compute shifting into RL would slow the pretraining trend, but maybe overemphasised test-time compute too much (it mainly brings capabilities forward 1-2 years, so isn’t a driver in the same way as the other three). I didn’t emphasise enough the possibility that a new driver or paradigm will be discovered between now and 2030, although it seems quite likely to me (especially if LLMs hit stronger diminishing returns).", "url": "https://wpnews.pro/news/ai-wasn-t-overhyped-almost-everyone-underestimated-it", "canonical_source": "https://benjamintodd.substack.com/p/almost-every-ai-forecast-of-the-last", "published_at": "2026-10-27 01:07:00+00:00", "updated_at": "2026-10-27 01:41:49.381338+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-research", "large-language-models", "ai-agents"], "entities": ["Anthropic", "OpenAI", "FrontierMath", "LiveCodeBench Pro", "METR", "LEAP forecasting panel", "AI 2027"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/ai-wasn-t-overhyped-almost-everyone-underestimated-it", "markdown": "https://wpnews.pro/news/ai-wasn-t-overhyped-almost-everyone-underestimated-it.md", "text": "https://wpnews.pro/news/ai-wasn-t-overhyped-almost-everyone-underestimated-it.txt", "jsonld": "https://wpnews.pro/news/ai-wasn-t-overhyped-almost-everyone-underestimated-it.jsonld"}}