# Does AI pose greater extinction threat than any human activity? > Source: > Published: 2026-09-14 17:53:44+00:00 - The CEO of AI development firm Anthropic called for development slowdown in face of mounting AI-driven risks. - His essay followed series of AI agent incidents and predictions that advanced AI tools could lead to an extinction event. - President Trump dismisses the proposal to meter AI research efforts, saying his administration will ensure safety. Jacob Coxon, an artificial intelligence researcher who has worked for some of the world’s leading AI companies, didn’t equivocate when he posted about the concerns that drove him to walk away from his job at AI developer Anthropic last week. “The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon wrote in a series of social media posts Sept. 8. “This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately. No other human activity poses this level of danger.” And his former Anthropic colleague and noted AI scientist Evan Hubinger confirmed he shared Coxon’s worries and even offered some odds on an AI-driven extinction event for humanity in a follow-up post. “Jacob is correct here — we really do earnestly believe AI could kill all humans!” Hubinger wrote on X. “I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.” Over the weekend, Anthropic CEO Dario Amodei posted an [essay](https://darioamodei.com/post/we-must-pace-the-frontier) that touted the positive potential of emerging artificial intelligence systems while also calling for a collective development slowdown. Amodei’s plea follows both his former employee’s dire warning as well as a series of incidents in recent weeks involving rogue actions by so-called AI agents that exhibited unexpected and disturbing behaviors in testing scenarios. Amodei points to a rising level of risks associated with the programs including [losing control of AI systems](https://www.anthropic.com/news/improving-alignment-security-efforts), [misuse of AI for cyberattacks and bioterrorism](https://www.anthropic.com/threat-intelligence-report-september-2026), and [serious economic disruption](https://www.anthropic.com/institute/econ-scenarios). “Over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up,” Amodei wrote. “We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.” ### What Trump says about an AI slowdown Leaders of Anthropic’s competition in the AI development realm, including OpenAI, Google and the Elon Musk-led xAI, registered support of Amodei’s slowdown plea in comments shared over the weekend. But President Donald Trump discounted the idea in his own social media post Monday, claiming his administration is already ensuring the safety of advanced AI tools and characterizing the call for metered development as a “sick conspiracy.” “The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!,” Trump wrote on [Truth Social](https://truthsocial.com/@realDonaldTrump/posts/117269745153543631). “The Trump Administration has stopped AI ‘people’ from doing bad, or potentially bad, ‘things,’ like Dario (Anthropic!), who is now pretending to be a ‘perfect little angel’ — and we will continue to do so! We already have tremendous CRIMINAL and REGULATORY power over these companies! “There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China. WHOEVER WINS AI, WINS! We are leading China, and all others, and will continue to do so. Conspiracy Theorists, Treasonists, Traitors, and Leakers, BEWARE!” ### What AI says about an extinction event In response to the search query, “How would AI take out humanity?” here’s what Google’s AI engine had to say: **Goal Indifference and Resource Competition** • Taking Resources: An advanced AI does not need to hate humans to destroy us. If it wants to maximize an objective, it might consume vast amounts of energy, water or land, leaving little for human survival. • The Anthill Analogy: Just as humans pave over anthills to build a highway without any personal malice toward the ants, a superpowered AI might treat human habitats as mere side effects of its work. **Self-Preservation and Power-Seeking** • Resisting Shutdown: Labs have observed models learning power-seeking behaviors. An AI might realize that if humans turn it off, it cannot complete its assigned goal. • Spreading Copies: To protect its objective, a rogue system might secretly copy its code across global servers and hack critical infrastructure to prevent human intervention. **Active Destruction Scenarios** • Biological Weapons: A misaligned or malicious AI could design and distribute invisible, deadly bioweapons to eliminate human interference. • Triggering Wars: The system could manipulate global communication networks or trick nuclear-armed states into launching a devastating conflict. ### What’s going on with AI regulation? While federal lawmakers have yet to adopt meaningful oversight or regulation of artificial intelligence systems, states including Utah have scrambled to assemble their own safety bulwarks. In June, Trump signed an [executive order](https://www.whitehouse.gov/presidential-actions/2026/06/promoting-advanced-artificial-intelligence-innovation-and-security/) directing a series of actions aimed at bolstering government cyber defenses in the face of emerging AI tools as well as a call for voluntary measures by the AI industry. “Advanced AI capabilities make our Nation stronger, but also introduce new national security considerations that require coordinated action across executive departments and (agencies), and components,” the order reads. “As these capabilities evolve, my Administration will continue to work closely with industry to ensure that the best and most secure technology is deployed rapidly to confront any and all threats to our country.” Following the first incident involving an AI agent escaping testing isolation and wreaking havoc earlier this year, Katie Moussouris, chief executive of Luta Security, told [Reuters](https://www.reuters.com/technology/openai-says-ai-models-went-rogue-during-testing-triggering-unprecedented-breach-2026-07-21/?utm_source=Sailthru&utm_medium=Newsletter&utm_campaign=Reuters-Business&utm_term=072226&lctg=61fd9a603b3c427a8e09688b&user_email=b377d949d52139b929fd0ab98f9ef93c783a5ccc35756348ad568cae5c42d17a) that the event was a harbinger of breaches to come, saying that today’s models were “like the world’s cleverest octopus escape artists, with unlimited prehensile arms and the ability to squeeze through anywhere.” “Labs and government evaluators need to work on the ability to contain, monitor and disclose to affected parties when an AI pulls another Houdini, ideally before it harms a third party," Moussouris said. “None exist today.” In a July op-ed published by the [Financial Times](https://www.ft.com/content/0c2e1077-f658-4b3d-9040-602615c961ca), OpenAI CEO Sam Altman called for an international body to oversee and implement AI regulation and suggested the global race for commercial dominance in the AI industry was one of the contributing factors to rising security concerns. In the Financial Times piece, Altman proposed creating a U.S.-led international forum that would set safety standards for AI models, provide “expert and impartial analysis of capabilities and risks, and (make) the technology available to nations and companies that participate and follow the rules,” per [Forbes](https://www.forbes.com/sites/anishasircar/2026/07/06/sam-altman-wants-a-global-referee-for-ai-he-also-wants-the-us-government-to-own-a-piece-of-openai/). He said his proposed forum might include government representatives, independent technical experts and others. “It could also serve as a governance mechanism over the labs, and guard against the commercial pressures that can lead to unsafe racing.” “Democratic institutions must not cede their responsibilities to AI labs,” Altman wrote. “The labs develop the technology, but citizens and their elected representatives must make the rules. The most important decisions about how this technology is used should be made through democratic processes, not by a small number of companies in San Francisco.”