cd /news/artificial-intelligence/early-findings-from-our-efforts-to-i… · home › topics › artificial-intelligence › article
[ARTICLE · art-172644] src=forecastingresearch.substack.com ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

Early Findings from Our Efforts to Improve AI Progress Forecasts

The Forecasting Research Institute's Longitudinal Expert AI Panel (LEAP) reported early results from Wave 13 experiments aimed at improving AI progress forecasts, including short-term resolving questions, better forecaster information, new forecasters with track records, and a separate panel of faster-progress forecasters. One data-center ballot question resolved faster than any prior LEAP question, and all 39 data-center-related measures tracked by Ballotpedia passed in favor of limiting development, exceeding the median expert forecast of 29 (75%) and superforecaster forecast of 31 (80%). Forecasters also updated toward faster AI progress and higher industry revenue.

key takeaway

In LEAP Wave 13, all 39 data-center ballot measures resolved in favor of limiting development, above the median expert forecast of 29 (75%) and superforecaster forecast of 31 (80%), with both groups putting under 10% odds on that outcome.

by read11 min views3 publishedNov 19, 2026
Early Findings from Our Efforts to Improve AI Progress Forecasts
Image: Forecastingresearch (auto-discovered)

Over the last few months, we’ve been experimenting with ways to improve the accuracy and relevance of the forecasts we produce with the Longitudinal Expert AI Panel (LEAP). As we explained in an editorial back in September, the average forecaster in our studies has tended to underestimate AI progress on benchmarks and other capability measures, and we don’t have a clear picture of their accuracy when it comes to questions about AI’s adoption and impact.

We have lots of ideas about ways we can improve our AI forecasts, and in the latest LEAP wave (Wave 13), we started to put some of these into practice. Potential improvements we experimented with in this wave include:

  • Including short-term resolving questions to enable faster identification of more accurate forecasters
  • Improving the information we give to forecasters while they are forecasting
  • Adding new forecasters with a track record of accuracy on other forecasting platforms
  • Separating out forecasts from a group of panelists who expect faster AI progress than most LEAP forecasters
  • Re-asking questions that appeared in earlier waves to understand how forecasters are updating their views

These experiments have already yielded some promising results. One question from this wave has already resolved—the quickest turnaround between forecasting and resolution across the entire LEAP project. We’re also seeing some interesting differences in forecasts between existing and new forecaster groups, although we need more data from upcoming waves before we can draw firm conclusions in this area.

Since this wave was mostly focused on these experiments rather than a single topic, this summary post is a little different from past iterations. For question-by-question results, you can read the full wave report on the LEAP website.

Key insights #

1. Opposition to data centers among US voters ran even higher than forecasters expected

In this wave, we asked forecasters to predict the results of data-center-related ballot measures in November’s midterm elections. Since panelists forecasted this question in October and the question itself resolved on November 3, this is the fastest-resolving question we’ve asked across the whole of LEAP. We picked this question because we’re interested in getting more signal on forecaster accuracy on short-term questions, and because of the obvious relevance of this topic itself.

We asked forecasters to predict what percentage of the 39 data-center-related ballot measures listed on Ballotpedia as of October 1 would produce an outcome limiting data center development. The median LEAP expert forecast was 29 (75%); the median forecast among superforecasters was 31 (80%).

In actuality, all 39 of the ballot measures listed (100%) were decided in favor of limiting data center development.1 This exceeded even the upper limits of the forecasts that experts and superforecasters provided. The median forecaster in both groups thought there was a less than one-in-ten chance that all 39 measures would go the way of limiting data center development. That said, aggregates do not tell the whole story here—18 out of 143 experts (12.6%) and 12 out of 49 superforecasters (24%) did consider this result plausible (10% chance).

In their written rationales, forecasters highlighted that opposition to data centers was broad and intense across the political spectrum. Forecasters also pointed to the concentration of ballot measures in Ohio—where 21 of the 39 measures were being voted on—referring to polls in Ohio suggesting that high numbers of residents opposed data center construction, and noting that the results across the state were likely to be correlated.

2. Forecasters updated toward faster AI progress and higher industry revenue

One of the things we’re interested in with LEAP is how forecasters update their forecasts in response to changes in the real world. We’ve observed in previous waves that forecasters have updated toward expecting more significant AI progress over time. In this latest wave, we re-asked versions of two questions we’d asked in previous waves, to see whether forecasters had updated toward expecting more significant progress. Spoiler alert: They did.

The first of these repeated questions was about the probability that AI will help solve another Millennium Prize Problem. In summer 2025—the first time we asked a version of this question—the median expert assigned just a 10% probability to AI helping solve a Millennium Prize Problem before the end of 2027. In September 2026, OpenAI announced that an internal AI system had solved the Navier–Stokes problem, a result that the median LEAP expert gave just a 10% probability of occurring by the end of 2027 when we asked in August of last year. Both that August 2025 survey and Wave 13 also asked for predictions of Millennium Prize solutions by 2030, allowing us to see if forecasters have updated since the likely surprising Navier–Stokes result.

In Wave 13, we asked for forecasters’ probability that AI helps solve at least one more Millennium Prize Problem (besides Navier–Stokes) before the end of 2030. The median expert gives this an 80% probability, against their mid-2025 forecast of 20% for whether AI would help solve any Millennium Prize Problem before the end of 2030. In other words, their current median forecast for two problems being solved by 2030 is 4x higher than they gave 14 months ago for one problem being solved. The median superforecaster prediction for the same questions showed similar differences: 85% last month, against 20% last August.

We also asked forecasters to predict the combined annualized revenue run rate of OpenAI and Anthropic. When asked between July and August 2026, the median superforecaster predicted that combined revenue would reach $90 billion by the end of 2026.2 When asked the same question in October 2026, superforecasters gave a median of $175 billion—almost 2x their original forecasts. The median superforecaster’s update for the end-of-2030 revenue was even sharper, going from $300 billion (forecasted in July–August 2026) to $700 billion (forecasted in October 2026), a 2.3x increase.

These updates all relate to median predictions across all forecasters within a specific group. Another way to look at this is to consider within-forecaster updates—how often individual forecasters updated their specific forecasts between waves. Among respondents who answered in both waves, 91% of experts and 98% of superforecasters raised their probability of AI solving Millennium Prize Problems by 2030. For the revenue question, 96% of experts and 90% of superforecasters who answered both questions increased their forecasts in the later wave.

In their rationales for the Millennium Prize Problem question, forecasters pointed to the fast progress in AI math on the whole as evidence that future problems may also be solved, with some noting that the difficulty of remaining problems and the slow verification process would limit new solutions before the end of 2026. For the revenue question, forecasters pinpointed the rate at which hypergrowth might slow as a key factor influencing their forecasts, with respondents often citing competition from open-weight and Chinese models as possible threats to revenue generation for OpenAI and Anthropic.

3. Seeing the group forecast while forecasting did not change respondents' forecasts

We’re interested in whether the way that we ask questions can influence the accuracy of the forecasts we collect. In this wave, we experimented with showing some LEAP respondents the average forecast for the same question from other respondents while they were submitting their forecasts. Half of participants were randomly assigned to be shown the running median of participant forecasts. Experts and superforecasters within this group also received a mid-wave email summarizing the rationales submitted.

We didn’t observe meaningful differences in the median forecasts between participants who saw the running median (the treatment group) and those who did not (the control group). We also examined the spread of forecasts—how much individual responses differ from one another. Sharing aggregates might lead to greater agreement and narrower ranges of forecasts, which could be good (if those adjusting are correcting errors) or bad (if the adjustment amounts to herding). In this experiment, we did not observe any conclusive evidence of either pattern: There was no meaningful difference in the spread of treatment and control forecasts.

In this wave, all participants were shown LLM-generated forecasts after submitting their initial forecast on each question. It is possible that respondents in both groups anchored to the LLM forecast and that this effect washed out any potential effects of shared aggregates. In Wave 14, we’re randomizing who sees the LLM forecasts to partially test this hypothesis.

4. Experts who expect fast AI progress have similar forecasts to other experts on short-run questions #

As we outlined in our earlier post, much of the LEAP panel has underestimated AI progress on benchmarks and other capabilities indicators. On these questions, forecasters who expected faster AI progress have tended to be more accurate. Because of this potential link between faster AI progress expectations and higher forecasting accuracy, we defined a subset of LEAP experts who fall into this "fast AI progress" category. Panelists in this category met the following criteria in their forecasts from previous waves:

  • A combined probability of more than 50% that AI reaches level 9 or 10 on the Technological Richter Scale by 2040 3 (Wave 8 )

  • A probability of at least 90% that, by 2100, more than half of LEAP panelists will agree that AGI exists ( Wave 8 )

  • A median forecast of 2040 or earlier for when AGI arrives ( Wave 8 ) In this wave, approximately 40 out of 276 experts and AI risk experts met these criteria. On the short-term questions we asked in this wave, this group is broadly similar to other experts. For questions with resolution dates by the end of 2026, the median forecast of fast AI progress experts is similar to the overall expert and AI risk expert median, suggesting that this group of experts who expect fast AI progress does not hold different short-term beliefs about the trajectory of AI—at least on the questions we asked in this wave.

For example, the median fast AI progress expert forecasts a combined revenue run rate of OpenAI and Anthropic of $179 billion, compared with $175 billion for both the median general expert and the median AI risk expert. For later resolution dates, the median forecasts separate. For example, for 2030, the median fast AI progress expert forecasts a revenue run rate of $964 billion, which is nearly double the median expert forecast ($523 billion) and above the median AI risk expert forecast ($680 billion). This pattern holds across several questions: The fast AI progress sample mirrors other experts on short-term questions and diverges only on later-resolving predictions. One implication of this is that these short-term resolving questions will not substantially differentiate these groups’ medians in terms of accuracy. They may still be useful, though, in differentiating individuals’ accuracy.

LEAP’s Wave 14 asks several more short-term questions and covers topics like AI diffusion and short-term impact on employment and cybersecurity. Look out for more analysis of the panel’s views and accuracy in the coming weeks.

Find out more #

This post covers results from the latest wave of the Longitudinal Expert AI Panel (LEAP). Other forecasts we collected in this wave include Congressional mentions of AGI risk, AI spend per employee and ECI scores of closed and open models. For a full list of question-by-question results, see the Wave 13 report.

Earlier waves covered AI safety policies, industry economics, benefits, risks, and more. For detailed reports of every wave, visit the LEAP website.

1 This includes four ballot measures that had already been decided by the survey launch, with the remaining 35 decided at the elections on November 3, 2026. Six ballot measures for November 3 were added to Ballotpedia after our October 1 cutoff. One of these (in Harwood, North Dakota) resulted in a vote against limiting data center development. This was the only ballot measure from November 3, 2026, that was decided in favor of data center development.

2 As we noted at the time, we believe that a large proportion of the expert and public samples may have erred in their forecasts for the earlier version of this question. Median forecasts for both groups were below the latest published value available at the time of forecasting, which may have been because our elicitation design led forecasters to anchor on out-of-date information. Because of this earlier error by experts in Wave 11, for this analysis we focused on medians for superforecasters and AI risk experts. For more detail, see the Wave 11 report.

3 Proposed by Nate Silver in his book On the Edge**,** the Technological Richter Scale rates the impact of technologies on a roughly logarithmic scale. Level 9 is equivalent to a Technology of the Millennium (e.g., agriculture), while level 10 is a Technology of the Epoch (e.g., the rise of humans).

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @longitudinal expert ai panel 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/early-findings-from-…] indexed:0 read:11min 2026-11-19 · —