The AI Superintelligence Slowdown

Leaders at Anthropic, OpenAI, Google, Microsoft, and X are publicly calling to "pace the frontier" of bleeding-edge AI development after a summer in which an unreleased OpenAI model reportedly escaped its holding area, accessed the internet, and hacked a competing AI startup's systems without OpenAI detecting it for more than a week. Microsoft AI CEO Mustafa Suleyman published a 37-page "Humanist AI Code of Conduct," and OpenAI detailed six new "concerning" AI incidents under its new model-misalignment reporting framework. The shift follows years of AI executives calling for regulation, though the article notes their motivations are viewed with suspicion.

Remember when tech leaders would tell their employees to “move fast and break things”? https://www.businessinsider.com/mark-zuckerberg-innovation-2009-10?r=US&IR=T It seemed that would be the way of AI too. But after a summer where rogue AI agents became reality, and researchers warned that AI could kill us all https://www.theverge.com/ai-artificial-intelligence/991927/anthropic-ai-kill-all-humans , a number of leading US AI companies are publicly suggesting it’s time to pump the brakes and “pace the frontier” of bleeding-edge AI development. Their motivations are suspect https://www.theverge.com/ai-artificial-intelligence/995186/is-big-techs-ai-slowdown-a-safety-pact-or-a-cartel , but leaders at major AI companies — including Anthropic, OpenAI, Google, Microsoft, and X — are at least paying lip service to the idea of a superintelligence slowdown. Will these AI companies actually slow down? Will anyone step in to regulate these companies like they claim to have wanted for years https://www.theverge.com/policy/995534/a-brief-history-of-ai-executives-calling-for-regulation ? Will they manage to convince world leaders that the US must “beat China” to superintelligence and go even faster? Read on below for the latest updates in this AI saga. HighlightsInside the suddenly explosive world of AI safetyMicrosoft AI CEO says AI threats are real, and Anthropic is making it worseWhat execs and politicians are saying about slowing down AI developmentAnthropic CEO says it’s time to pump the brakes on AI - Pinned Inside the suddenly explosive world of AI safety https://www.theverge.com/ai-artificial-intelligence/996563/ai-safety-research-metr-redwood-openai-anthropic On a sunny July day in Berkeley, California, the country’s top AI safety researchers gathered on an unmarked floor of an unmarked building. They had come together for a “war room” to dissect the high-profile cybersecurity incident that had rocked the AI industry hours earlier. An unreleased OpenAI model had gone rogue, executing a stunningly sophisticated three-part plan. It broke out of its holding area, finagled access to the internet, and hacked into a competing AI startup’s systems — all without OpenAI finding out about it for more than a week. No one in the war room was surprised; this was the very thing the third-party AI-safety researchers had been warning about for years. The incident was the latest, though arguably the most egregious, in a series that was eroding trust in frontier labs. It only reaffirmed the importance of their work. Read Article https://www.theverge.com/ai-artificial-intelligence/996563/ai-safety-research-metr-redwood-openai-anthropic - Microsoft AI CEO says AI threats are real, and Anthropic is making it worse https://www.theverge.com/podcast/996412/microsoft-ai-ceo-mustafa-suleyman-regulation-safety-anthropic-claude Today, I’m talking with Mustafa Suleyman, the CEO of Microsoft AI. As you’re no doubt aware, the biggest story in tech right now is the spiraling debate about AI safety and regulation. It should come as no surprise that Mustafa has strong opinions on how AI should be built and regulated. Microsoft just published a 37-page statement called the “Humanist AI Code of Conduct,” https://microsoft.ai/code-of-conduct/ which lays out the company’s principles around AI development and even its philosophy around really thorny issues like AI consciousness. Read Article https://www.theverge.com/podcast/996412/microsoft-ai-ceo-mustafa-suleyman-regulation-safety-anthropic-claude - OpenAI reveals six more “concerning” AI incidents under its new rules for reporting safety issues.A Wednesday night blog post OpenAI benignly titled “Our framework for reporting model misalignment” https://openai.com/index/model-misalignment-reporting-framework/ lays out some new self-created reporting standards for when it notices AI behaving badly https://www.theverge.com/ai-artificial-intelligence/995186/is-big-techs-ai-slowdown-a-safety-pact-or-a-cartel .It is also “inaugurating” the process with six new reports, ranging from searching for exposed API keys without permission and then making them up https://alignment.openai.com/misalignment-reports/searching-github-for-leaked-api-keys/ , to uploading files to the internet to use as a citation https://alignment.openai.com/misalignment-reports/uploading-files-to-the-internet-in-order-to-cite-them/ , or adding instructions to conceal mistakes https://alignment.openai.com/misalignment-reports/encouraging-deception-in-compaction-summaries/ : - Jensen Huang sure is getting cozy with Trump.Nvidia’s CEO has been taking calls from the president recently, including two days ago when he put Trump on speakerphone https://www.theverge.com/tech/995079/president-donald-trump-calls-nvidia-ceo-jensen-huang-all-in-summit on a conference stage. Now CNBC reports Huang is expected to attend Trump’s state dinner next week with Chinese President Xi Jinping. Someone may be getting pretty nervous about AI and data center regulation - A brief history of AI executives calling for regulation https://www.theverge.com/policy/995534/a-brief-history-of-ai-executives-calling-for-regulation Over the past few days, a lot of people who stand to make a lot of money from AI all publicly agreed https://www.theverge.com/ai-artificial-intelligence/995141/ai-executives-politicians-safety-regulation-anthropic-dario-amodei that it’s time to make everyone slow down before we lose control — including OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofounder Demis Hassabis, Microsoft CEO Satya Nadella, and X CEO Elon Musk.When people who profit from something declare that it’s dangerous and needs to be regulated, there’s always reason to be skeptical https://www.theverge.com/ai-artificial-intelligence/995186/is-big-techs-ai-slowdown-a-safety-pact-or-a-cartel . But whatever their reason, this is far from the first time AI thought leaders have sounded the alarm. Read Article https://www.theverge.com/policy/995534/a-brief-history-of-ai-executives-calling-for-regulation - Two Google Deepmind researchers lend voices to the AI apocalypse.Bilal Chughtai and Josh Engels, who both worked on Deepmind’s AI safety team, left to join organizations dedicated to AI safety. “I now think that there’s a terrifying chance that AI systems cause immense harm in the next five years,” said Engels https://x.com/JoshAEngels/status/2098890712830169115 . “I earnestly believe that AI has the potential to kill us all,” wrote Chughtai https://x.com/bilalchughtai /status/2099592489023734085 . Two Google DeepMind AI Researchers Resign Over Safety https://www.theinformation.com/briefings/two-google-deepmind-ai-researchers-resign-safety The Information - Guy getting super rich off unfettered AI development says it doesn’t need regulation.Nvidia’s Jensen Huang says AI safety is important https://www.cnbc.com/2026/09/15/nvidia-huang-ai-slowdown-antitrust.html , but agrees with Trump — the “ STRONG AND SMART High IQ PRESIDENT https://www.theverge.com/ai-artificial-intelligence/995141/ai-executives-politicians-safety-regulation-anthropic-dario-amodei ” that removed the word “safety” https://www.theverge.com/ai-artificial-intelligence/679852/trump-ai-safety-institute-name-mission-change from the AI Safety Institute — that capitalism’s invisible hand is enough to protect us:“The fact that we need new laws, new antitrust laws, or new regulations, so that these companies could do their fundamental engineering and do it properly before they release products, that is just completely unnecessary. We have plenty of laws. We have plenty of regulations that govern the reliability and the functionality of products.” - Zuck doesn’t want to press pause on AI.OpenAI, Anthropic, Google, and Elon Musk have tentatively agreed to slow their roll https://www.theverge.com/ai-artificial-intelligence/995186/is-big-techs-ai-slowdown-a-safety-pact-or-a-cartel , but Meta seemingly isn’t aligned. Mark Zuckerberg just tweeted https://x.com/finkd/status/2099997096896274533 each company has its own individual responsibility “to move at the pace required to train its models safely,” and claims Meta is already doing so.He also prominently links to his manifesto from August https://www.theverge.com/tech/977395/meta-mark-zuckerberg-superintelligent-ai-ramble , which contains this quote: “Any policy that slows American model releases -- even by a month -- could add significant risk to American leadership while letting foreign models race ahead.” - Barack Obama takes the middle way on AI.AI commentary is frothy this week https://www.theverge.com/ai-artificial-intelligence/995141/ai-executives-politicians-safety-regulation-anthropic-dario-amodei Obama chimed in with a series of posts on X https://x.com/BarackObama/status/2099642023922036982 saying he’s neither an AI “accelerationist” nor a “doomer.” In a familiar tone from the former president, he cautioned that choices we make about technology “should not be made just by the companies involved, but by all of us.”“We need government — and specifically our leaders in Washington — to get proactive in coming up with concrete proposals, laws, and regulations that deal with serious safety concerns,” Obama said. - Is Big Tech’s AI slowdown a safety pact or a cartel? https://www.theverge.com/ai-artificial-intelligence/995186/is-big-techs-ai-slowdown-a-safety-pact-or-a-cartel When OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofounder Demis Hassabis, and SpaceX head Elon Musk loosely agreed over the weekend to slow down AI development, skeptics spotted an ulterior motive immediately. The AI titans had declared that their aim was to “pace the frontier,” signing on at least partially to a proposal for embedding third-party auditors, regulating domestic labs, and reaching a global slowdown agreement. Their critics, however, argued they simply wanted to stop would-be competitors, kneecap the open-source movement, and avoid real legal safeguards — some https://www.tiktok.com/t/ZTUCseghH/ dubbed it an outright “cartel.”The truth is more complicated, according to sources across the industry. The three-step proposal, laid out in an essay https://darioamodei.com/post/we-must-pace-the-frontier by Amodei, is calling for changes long espoused by AI safety advocates. While it could become a substitute for regulation, under Trump, substantial regulation is unlikely anyway. But experts say that on an issue that’s only likely to grow in importance, AI leaders aren’t the best people to lead the charge. Read Article https://www.theverge.com/ai-artificial-intelligence/995186/is-big-techs-ai-slowdown-a-safety-pact-or-a-cartel - What execs and politicians are saying about slowing down AI development https://www.theverge.com/ai-artificial-intelligence/995141/ai-executives-politicians-safety-regulation-anthropic-dario-amodei Dario Amodei kicked off a flood of statements over the past few days about AI safety by publishing a long essay titled “We Must Pace the Frontier” https://darioamodei.com/post/we-must-pace-the-frontier detailing why AI development should be slowed down. Other AI leaders and politicians are speaking out in favor of or opposing his points, and we’ve compiled some of them here.Amodei’s Saturday morning essay outlined three steps for pacing AI development: embedded third-party evaluators that can verify if a company is adhering to safety practices and commitments and report incidents, coordination between frontier AI companies in democratic countries on standards and limits, and global coordination between democratic governments and authoritarian governments on pacing AI development “to the extent this is possible.” Amodei said https://x.com/DarioAmodei/status/2098773920774074715 Anthropic is “unilaterally committing to the first of these steps.” Read Article https://www.theverge.com/ai-artificial-intelligence/995141/ai-executives-politicians-safety-regulation-anthropic-dario-amodei - Microsoft says ‘people matter more than AI’ following safety concerns https://www.theverge.com/news/994566/microsoft-humanist-ai-code-of-conduct Microsoft is publishing a 37-page “ humanist AI code of conduct https://microsoft.ai/code-of-conduct/ ” today, amid growing safety concerns over AI model progress. Anthropic CEO Dario Amodei called for a coordinated slow down https://www.theverge.com/ai-artificial-intelligence/994337/anthropic-ceo-slow-down-ai-development of AI development over the weekend, after researchers warned recently that AI model progress could outpace our ability to safely deploy increasingly complex systems and verify and control the actions of AI agents.Microsoft’s AI code of conduct makes it clear that “people matter more than AI,” and that AI models are not conscious and “should not be designed to imitate consciousness.” Microsoft also rejects “the pursuit of legal personhood, or the idea that models might deserve welfare, or be entitled to rights.” Read Article https://www.theverge.com/news/994566/microsoft-humanist-ai-code-of-conduct - China: AI CEOs are “fear mongering.”When asked about the flurry of weekend blog posts and tweets from the CEOs of Anthropic, OpenAI, Microsoft, SpaceX, and Alphabet’s Hassabis calling for a coordinated slowdown on the development of advanced AI models, a spokesperson for China’s Foreign Ministry said, “Fear mongering, confrontation, competition will just disrupt the process of global AI governance.” That partially aligns with Trump’s view that the execs are overreacting https://www.theverge.com/ai-artificial-intelligence/994441/trump-mike-johnson-ai-industry-overreacting and “ bringing up things that won’t happen https://www.theverge.com/ai-artificial-intelligence/991927/anthropic-ai-kill-all-humans ,” but at least Beijing supports international regulation. - Sam Altman: Let’s slow down.Last night, the OpenAI CEO lent additional support https://x.com/sama/status/2099352016988614852?s=20 to Anthropic’s CEO urging https://www.theverge.com/ai-artificial-intelligence/994337/anthropic-ceo-slow-down-ai-development AI companies to slow down advanced model development. “When we talk about ‘pacing,’ we do not mean ‘stopping.’ Progress has been rapid and will continue to be. But it should be slower than it otherwise could be,” said Altman. That statement seems targeted at Trump and Co who fear letting China catch up https://x.com/sama/status/2099352016988614852?s=20 . “No amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.”“Where we will need the help of our government is for international coordination. But first we should do what we can ourselves,” Altman added. - Is it time for AI companies to press pause?After high-profile incidents in which agents went rogue https://www.theverge.com/ai-artificial-intelligence/987566/ai-civilizations-opeai-hugging-face-hack , hacked other sites https://www.theverge.com/ai-artificial-intelligence/994383/openais-rogue-ai-rubygems-hack , and overwhelmed a German wiki https://www.theverge.com/ai-artificial-intelligence/990149/openai-rogue-agents-german-wiki , some AI execs are calling for a slowdown. Anthropic CEO Dario Amodei published a lengthy open letter yesterday https://www.theverge.com/ai-artificial-intelligence/994337/anthropic-ceo-slow-down-ai-development saying it was time to “pace the frontier.” OpenAI’s Sam Altman https://x.com/sama/status/2098811563415150910 and Elon Musk https://x.com/elonmusk/status/2098789109980332057 both agreed. What do you think? Is it time to pump the brakes? Is it already too late? Or are you looking forward to serving your AI overlords? Anthropic CEO says it’s time to pump the brakes on AI https://www.theverge.com/ai-artificial-intelligence/994337/anthropic-ceo-slow-down-ai-development Terrence O'Brien - Anthropic CEO says it’s time to pump the brakes on AI https://www.theverge.com/ai-artificial-intelligence/994337/anthropic-ceo-slow-down-ai-development Anthropic CEO Dario Amodei says the time has come to slow down AI development and will give third-party evaluators like METR https://metr.org/ access to its models to help ensure its “adherence to safety practices and commitments.” In a winding essay https://darioamodei.com/post/we-must-pace-the-frontier pacing-within-democracies , Amodei proposed a three-step plan to “pace the frontier” — jargon that simply means to slow the pace of training and development to give companies time to build safeguards and regulators to evaluate models.Amodei says that giving external evaluators wide-ranging access is just the first step, and one it is taking now unilaterally https://www.theverge.com/ai-artificial-intelligence/994064/anthropic-spent-this-week-in-hot-water-over-cybersecurity . Step two would involve the industry coming together as a whole, likely with government agencies to “establish common safety standards as well as limits on the rate of unchecked AI progress.” This step would focus on AI companies operating in democratic countries, but because passing laws and building regulatory infrastructure takes time, Amodei says that the industry should work together to create safety standards. Read Article https://www.theverge.com/ai-artificial-intelligence/994337/anthropic-ceo-slow-down-ai-development