A CERN for AI is a distraction; push for an IAEA instead

A new analysis argues that proposals for a "CERN for AI" are a distraction from more effective governance measures, advocating instead for binding international red lines and an IAEA-style verification body for artificial intelligence. The piece contends that the main bottleneck in AI safety is enforcement and political will, not additional research and development.

TL;DR: There are many conceivable versions of a “CERN for AI.” But the version that seems politically realistic a new catch-up lab probably would not do much for safety, while the versions that would materially improve safety e.g., pause + merge of all companies are probably unrealistic. So I see the CERN idea as a distraction, and not a particularly neglected one. I argue a better path is an international treaty with red lines now, with an IAEA-style verification body next: a sequencing that matches how the EU AI Act, the NPT/IAEA, and the Montreal Protocol actually developed. This is premised on the view that the main bottleneck in AI safety is enforcement and political will, not more R&D. Two premises underlie the rationale below: If you reject either premise, you might reach different conclusions. A recurring proposal in AI governance is to build a “CERN for AI” 1 . The CERN pitch is seductive. " But I think that there are many problems with it. What do you even mean by CERN? Are you asking for: To be honest, if you really do believe we might face catastrophic risks in a few months or years, I think the priority is to stop the hemorrhage 3 . If what you ultimately want is to mitigate AI risks, say it, and don't play 4D chess. We should ask for international regulations now. This leads us to: For context, the IAEA International Atomic Energy Agency issues international nuclear safety standards and red lines , supports peer reviews and inspections, and coordinates assistance during nuclear emergencies. These standards are then adopted and enforced through national legislation worldwide. An IAEA for AI would play a similar role for artificial intelligence. Red lines have something CERN doesn't: existing momentum. Red lines are the most widely supported https://thefuturesociety.org/cso-ai-governance-priorities/ measure by research institutes, think tanks, and independent organizations. By signing the Seoul Frontier AI Safety Commitments, companies agreed to "Set out thresholds at which severe risks posed by a model or system, unless adequately mitigated, would be deemed intolerable." Granted, OpenAI's "red line" for recursive AI self-improvement is currently inadequate https://www.lesswrong.com/posts/6CYszKLnCagYyEiLM/openai-s-red-line-for-ai-self-improvement-is-fundamentally , but we're not building from zero, and this is why red lines need to be binding rather than voluntary. China's Premier Li Qiang stated that https://www.barrons.com/news/china-premier-says-red-line-needed-in-ai-development-3116339a "there should be a red line in AI development, a red line that must not be crossed." Pope Francis urged nations to adopt " a binding international treaty https://www.reuters.com/technology/pope-calls-binding-global-treaty-artificial-intelligence-2023-12-14/ .”, and Paolo Benanti, the Pope's AI adviser, called explicitly for " binding international treaties and red lines. https://www.transformernews.ai/p/paolo-benanti-catholic-church-vatican-superintelligence-artificial-intelligence-pope " Even the Executive order signed by Trump in early June frames this as a threshold definition: NSA will define a cyber-capability threshold, classified and shared with developers. Red lines need an institution: the IAEA model The final big hesitation while drafting the Global Call for AI Red Lines https://www.lesswrong.com/posts/vKA2BgpESFZSHaQnT/global-call-for-ai-red-lines-signed-by-nobel-laureates was whether to ask explicitly for the IAEA for AI. The main reason we didn't ask for the IAEA in the global call was mostly to generate more support "IAEA for AI" sounds technocratic and wonky to non-specialists, while red lines are intuitive . But red lines only matter if they can be independently verified: without an independent authority, there is no trust; without trust, no credible verification; and without credible verification, red lines are unlikely to be respected. This is why the strongest arms-control regimes are paired with international verification mechanisms: the Chemical Weapons Convention with the OPCW, and the NPT with the IAEA. At the India AI Impact Summit, the CEOs of the three leading frontier AI labs each called for international AI oversight. https://x.com/CRSegerie/status/2024570328295625153 Altman joined Hassabis https://x.com/CRSegerie/status/2024570328295625153 in calling for an institution modeled on the IAEA. Amodei called for red lines with enforcement mechanisms. 4 The fact that CEOs have recently publicly called for IAEA-style oversight is one of the strongest arguments for the current US administration. This sequencing – an international agreement with red lines first, institution second – mirrors how international governance actually works. The EU AI Act passed without every technical threshold defined; the AI Office was established afterward; specific thresholds are currently being defined, by technical consortiums working with the AI Office https://digital-strategy.ec.europa.eu/en/funding/eu-ai-office-launches-eu9-million-tender-technical-support-gpai-safety . Same pattern from the Vienna Convention to the Montreal Protocol, with detailed control measures strengthened gradually through expert-led review. Political agreement creates the conditions for technical work to happen inside the governance process, not before it. Thanks to Arthur Grimonpont and Epiphanie Gedeon for helpful suggestions and feedback on this memo. Even Demis Hassabis, back in the day, https://www.reddit.com/r/singularity/comments/1iseeqz/google deepmind ceo says for agi to go well/ said https://www.reddit.com/r/singularity/comments/1iseeqz/google deepmind ceo says for agi to go well/ : "At some point in the future, we'll need a CERN for AGI for international coordination on safety research." If a credible pause-plus-co-development package were actually on the table something like the AI-2027 slowdown scenario, with DeepMind, Anthropic and OpenAI brought into one project, even though they cannot hold hands at the Delhi Summit , I'd be happy to back option a . I don't expect that scenario to materialize. CEOs of frontier companies have way too much ego for this. Even Mistral, which is already generations behind, would resist being folded into a frontier consortium tomorrow. Merging the different frontier labs is wayyyy harder than creating a new institution. The EU AI Office creation went pretty smoothly when you think about it. Many people pushing for a CERN have European sovereignty in mind. To be fair, I think that Europe should wake up to the importance of AI. But there are so many ways to do it in a more effective way: The CEOs of the three leading AI companies have each publicly called for international oversight. Dario Amodei said he could imagine a worldwide treaty with enforcement mechanisms https://www.nytimes.com/2026/02/12/opinion/artificial-intelligence-anthropic-amodei.html . Sam Altman called for " urgent global regulation on AI https://x.com/AFP/status/2024471561764291071 ”, and for an equivalent of the International Atomic Energy Agency for international coordination on AI https://x.com/AFP/status/2024471561764291071 . Demis Hassabis also called for " some kind of equivalent of the IAEA https://youtu.be/4poqjZlM8Lo?si=6ZZrlspr27zaost ." For reference, the IAEA issues international nuclear safety standards and red lines, supports peer reviews and inspections, and coordinates assistance during nuclear emergencies. These standards are then adopted and enforced through national legislation worldwide.