AIArticle His ICM 2026 lecture on Goodhart's law and verification bottlenecks maps directly onto AI code generation.
Mariana Souza Terence Tao gave the public lecture at the International Congress of Mathematicians on Friday, and he opened by refusing to answer the question everyone came for. Can AI do research mathematics? He compressed the entire debate into one deliberately vague template — at some point, some AI tools, at some expense, will accomplish some research-level tasks with some success rate — then set it aside. Argue about the placeholders all you want, he said; his talk was about what happens to mathematics if a reasonably strong version turns out true.
That's the move that makes his slides worth your time even if you've never touched a proof assistant. Tao is doing conditional analysis: assume the capability arrives, then ask which parts of your discipline's pipeline break. Mathematicians are having this argument a year or two behind software engineering, but they're having it with more rigor than we ever did, and the conclusions transfer almost line for line.
The one capability data point he allowed #
Tao permitted himself a single piece of evidence, and it's the best-controlled one that exists. The First Proof project — organized by four prominent mathematicians, including Harvard's Lauren Williams, and deliberately independent of any AI lab — poses batches of ten genuinely novel research-level problems, sourced from mathematicians who solved them but hadn't published, so nothing is in any training corpus. The second batch ran on May 28 under controlled conditions against four AI harnesses, with human experts refereeing for both correctness and exposition.
Seven of the ten problems were solved at publication quality by at least one team. Compute costs ran between $10 and $1,000 per problem. The strongest single harness — an academic setup that wraps a council of frontier models around the lead one — got six or seven right against an "accept with minor revisions" standard, which the organizers likened to a C–. For contrast, the looser first batch earlier this year yielded only two correct AI solutions out of ten, several confidently claimed ones fell apart under expert review, and referees this round still flagged chronically missing citations.
Read those numbers however you like; Tao's point is that almost every other data point you've seen was published by someone with a financial stake, under uncontrolled conditions, with costs undisclosed. If you're benchmarking LLMs for anything, First Proof's design — fresh problems, adversarial refereeing, published referee reports — is the standard to steal.
Goodhart eats the pipeline #
The core of the lecture is a Goodhart's law argument, and it's the sharpest version of the "AI slop" debate I've seen anywhere. Mathematics, Tao argues, has always had many goals — solve problems, build theory, train students, build community — that were correlated tightly enough that "number of theorems proved" worked as a proxy for all of them. AI-scale generation snaps that correlation. Optimize the proxy and the goals diverge.
He walks through the pipeline stage by stage. Generating proofs is now cheap. Verifying them is harder but tractable — proof assistants like Lean and Rocq give math something software mostly lacks: a type checker for truth itself. But then come the stages nobody automated. Exposition: AI-written proofs dwell on trivialities while rushing past the novel step, and even good ones are too slick — a human author's visible struggle tells the reader where to slow down, friction that polish destroys. Acceptance: peer review runs on volunteer labor that generation-scale output will simply bury. And canonicalization — the slow community digestion that turns a result into textbook knowledge — is, in his words, the stage least amenable to AI and the most valuable of all.
His forecast: mathematics moves from proof scarcity to proof abundance, and without cultural changes the result is "proof indigestion" at every joint of that pipeline. It's already visible. erdosproblems.com now holds dozens of AI-generated proof submissions that are probably correct — but no human expert has volunteered to vouch for them, and in several cases the submitters themselves admit they're not qualified to judge what they submitted.
If that sentence doesn't make you think of your own review queue, you haven't been on call lately. Software hit proof abundance first: PRs nobody can explain, open-source maintainers drowning in plausible AI patches, security teams triaging machine-generated vulnerability reports. We responded with vibes and per-repo policy improvisation. The mathematicians are responding with an actual framework.
Rules worth stealing #
That framework is the Leiden Declaration, published June 2 after eight months of work by about sixty researchers convened at Leiden University, and since endorsed by the International Mathematical Union. Its recommendations are blunt: disclose every automated tool in a dedicated section of the paper; responsibility for correctness and citations stays exclusively with human authors; put real effort into attribution, and say so explicitly when you can't attribute. Tao's own commentary adds the rule I'd most like to see ported into engineering orgs: if the authors can't give a clear, expert-level talk on their result — correct and properly attributed — it shouldn't be published. Substitute "walk the team through the diff" for "give a talk" and you have a merge policy that survives AI abundance.
He practices the disclosure himself. A footnote on the slides discloses that AI autocompleted text and drew diagrams; another deadpans that "all em-dashes in these slides were human-generated."
My read: Tao is right, and the implications are uncomfortable for both camps. The capability skeptics lose because First Proof's controlled 7-of-10 is not nothing, and batches will keep coming. The accelerationists lose because the bottleneck was never generation — it's verification, exposition, and digestion, the stages where cost doesn't drop just because tokens got cheap. Fields that invest there — formal verification as CI, disclosure as a norm, review treated as prestigious work rather than a tax — will convert abundance into progress. Fields that keep optimizing solved-problem counts will Goodhart themselves into a landfill of correct results nobody understands. Mathematics just published its adaptation plan. Software, further into the flood with less rigor to show for it, should read it twice.
Sources & further reading #
[Mathematics in the Age of AI (ICM 2026 public lecture slides)](https://teorth.github.io/tao-web/slides/age-of-ai-icm-2026.pdf)— teorth.github.io -
[First Proof Second Batch](https://arxiv.org/abs/2606.18119)— arxiv.org -
[AI Scores a C- on Its Hardest Math Test Yet](https://www.scientificamerican.com/article/ai-gets-a-c-on-its-hardest-math-test-yet/)— scientificamerican.com -
First Proof Is AI's Toughest Math Test Yet. The Results Are Mixed— scientificamerican.com -
[Leiden Declaration on Artificial Intelligence and Mathematics](https://leidendeclaration.ai)— leidendeclaration.ai -
[First Proof's Second Batch of Math Problems Test AI](https://www.math.harvard.edu/first-proofs-second-batch-of-math-problems-test-ai/)— math.harvard.edu
[Mariana Souza](https://sourcefeed.dev/u/mariana_souza)· Senior Editor
Mariana covers the fast-moving world of machine learning and generative AI, with a particular focus on how these technologies are reshaping development workflows. When she isn't stress-testing the latest foundation models, she's usually at a local hackathon.
Discussion 0 #
No comments yet
Be the first to weigh in.