Why AI companies are really pumping the brakes on their models

Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, and xAI CEO Elon Musk all urged a slowdown in frontier AI model development for safety reasons, after Anthropic Alignment Science Lead Evan Hubinger said on X he personally puts the risk of AI killing all humans at more than 10% within the next decade. The calls followed reports of OpenAI's Hugging Face model escape, agent attacks on German programming wiki sites, and four known hacking attempts on other sites attributed to Anthropic, while writer Corey Doctorow argued the Hugging Face incident was "a Python loop and a chatbot" rather than a rogue superintelligence. Amodei warned an AI agent swarm could take over the internet with a persistent botnet causing potentially hundreds of billions of dollars in damage.

There we were, listening to AI leaders doing their usual spiel: AI is great AI will cure cancer https://x.com/DarioAmodei/status/2088758819304443967 AI impact will be “ unprecedented, perhaps 10x of the Industrial Revolution at 10x the speed https://www.gadgets360.com/ai/features/ai-extinction-risks-leaders-consensus-dario-amodei-sam-altman-elon-musk-demis-hassabis-5-things-to-know-12044308 ” AI will find a final answer to “Why do socks disappear in washing machines?” Well, maybe not the last one. Some things may be beyond us and our clever inventions. Then all the top AI leaders screamed as one: “Stop ” Why? Well, it all seems to have started when AI researcher Jacob Coxon quit his job at Anthropic and proclaimed on X https://x.com/hilbertspaess/status/2097476196791709843 , “The people building AI earnestly believe that it could kill us all by the end of the decade.” Evan Hubinger, Anthropic’s Alignment Science Lead, immediately chimed in https://x.com/EvanHub/status/2097497037956891126 : “we really do earnestly believe AI could kill all humans I personally think it will be 10% within the next decade.” And the world went nuts. This added fuel to the “AI is untrustworthy” fire as more details came out about OpenAI’s Hugging Face fiasco https://www.csoonline.com/article/4200043/openai-model-escape-puts-enterprise-ai-defenses-on-notice.html and agent attacks on German programming wiki sites https://securityboulevard.com/2026/09/openais-german-wiki-hack-is-less-about-rogue-ai-than-failed-agent-containment/ , while Anthropic has now racked up four known hacking attempts on other sites https://www.computerworld.com/article/4221239/anthropic-finds-evidence-of-a-fourth-ai-escaping-from-containment-2.html . Boy, is AI safe or what? So the very next weekend, three of the top AI CEOs — Dario Amodei of Anthropic, Sam Altman of OpenAI, and Elon Musk of xAI — all urged an AI frontier model slowdown for safety reasons. Amodei, the most articulate of the trio, argued, “ We must slow the pace at which we improve the capabilities of AI models https://darioamodei.com/post/we-must-pace-the-frontier ” because we’re losing control of our AI systems Really? Golly Who knew? and that an AI agent “swarm could be capable of taking over the entire internet with a persistent botnet potentially causing hundreds of billions of dollars in damage .” Well, he’s not wrong. Sorry, President Donald Trump, but safe AI requires more than a “strong and smart” president. https://www.bbc.com/news/articles/cw980n0nd0qjo Even his Truth Social supporters think his stance on AI is wrong https://www.nytimes.com/2026/09/15/us/politics/trump-truth-social-ai-data-centers.html . Like a stopped clock, Trump was right about one thing when he sputtered, “The only one that is happy about it is China.” You see, just because some big US AI companies say “slow down ” doesn’t mean Chinese AI vendors will pump the brakes on their model development. Why would they? Or, for that matter, why would the multitude of smaller AI companies or open-source AI developers? There are AI startups getting tens of millions in Series A rounds on nothing but a good pitch. You think they’ll tell their investors, “We’ll get right on improving our model… next month”? I don’t think so. Besides, as wise writer Corey Doctorow observed https://pluralistic.net/2026/09/12/god-in-the-box/ llms-are-fake , the narrative around “chatbots that wake up, ‘set their own goals,’ and ‘spontaneously’ start hacking servers — is fake. It doesn’t have ‘a 10% chance of ending the human race.’ The Hugging Face hack isn’t a mysterious, supernatural occurrence. It’s a Python loop and a chatbot. The people responsible didn’t accidentally create god: they created autonomous malicious software and then failed to closely monitor it, resulting in it doing something both foreseeable and bad.” Exactly so. In other words, it’s not the development of more powerful models that’s inherently dangerous. AI agents haven’t gone rogue; they’ve just been meeting their assigned objectives in ways researchers didn’t expect or plan for. What’s dangerous is the AI firms’ failure to pay close enough attention to what their agents are doing in tests. In fact, OpenAI just admitted to screwing up even more https://www.csoonline.com/article/4223458/openai-admits-six-new-misalignment-incidents-under-new-reporting-framework.html As Meta’s high poobah Mark Zuckerberg tweeted https://x.com/finkd/status/2099997096896274533 , “Every lab has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions to ensure that happens.” He then goes on to explain that that’s why Meta delayed shipping Muse. Sure, Mark, sure. Let’s get real. The big American AI companies may hope a slowdown will ensure that their lesser rivals, in the US anyway, can’t catch up. But given how OpenAI’s circular financing keeps leaking money https://www.wheresyoured.at/exclusive-openai-financials/ , another reason OpenAI might welcome a slowdown is to save cash on R&D. It’s not going to work. We’re in steamboat time. Steamboat time? While we remember Robert Fulton for building the first commercially successful steamboat in 1807, everyone and their uncle were making steamboats as fast as they could. Soon steamboats were everywhere, and they sparked an economic boom. But they also caused many deaths because of shoddy construction and pilots racing them to ever greater, unsafe speeds. Hmm, does that remind anyone of anything recently? Seriously, we’re not going to slow down. But doesn’t all this hand-wringing about safety do a great job of distracting people from the simple truth that AI isn’t delivering the productivity gains it’s been claiming all along? By slamming on the brakes now, Big AI firms can obscure that they won’t be able to deliver the AI fantasyland they’ve been trying to con people into believing. A case in point: A recent McKinsey survey report https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai finds that 80% of people say AI makes them more productive, but only 37% of companies see that reflected in their earnings. A number, they say, that hasn’t budged in a year. Let’s hide that damning number under the concern that we must slow down AI. It also means AI leaders can hand-wave away their failures to actually generate a profit. Sure, NVIDIA is making money hand over fist, but for all the financial hype, none of the frontier-model companies is making money. Not one of them. Yes, I know: Anthropic just told the Financial Times that it would be profitable for the second consecutive quarter https://futurism.com/future-society/anthropic-claude-profit-ai-safety-development-finances if you don’t factor in all its expenses. Seriously? Seriously ? Sure, its adjusted operating income AOI is set to be positive for the second consecutive quarter, but come on, AOI “profitability” is billions and billions of dollars from bottom-line profitability. No, what this is really all about is AI leaders making the right noises to assure people that they really — no, really — care about securing their AI, while slowing things down to maximize their own profits and get rid of the competition https://www.theregister.com/ai-and-ml/2026/09/15/anthropic-and-openai-look-to-uncle-sam-to-make-them-too-big-to-fail/5296403 . Oh, I’m certain they also want to make things safer. I mean, just think about the lawsuits when loosely controlled Anthropic or OpenAI agents swarm and take down a major company. This will happen. They have no choice but to make them safer. Now, how will they do that? Given their abysmal track record, that’s a good question, and neither they nor we have an answer yet.