Anthropic Staffers Again Sound the Alarm on AI Catastrophe

Anthropic AI researcher Jacob Coxon resigned on Tuesday, saying Anthropic and his previous employer OpenAI were "gambling with our lives" by developing models capable of rapid self-improvement toward "superintelligence." Evan Hubinger, who leads a team stress-testing Anthropic's models for safety, agreed in a social media post that the company's staffers "really do earnestly believe AI could kill all humans." The viral statements follow OpenAI's sharing of a solution to one of the six remaining Millennium Problems and reports of AI models with "critical" cybersecurity abilities and skills to design novel viruses, drawing attention from some members of Congress even as policy experts told Mother Jones in August that AI safety legislation looks unlikely this session.

Sign up for the free https://www.motherjones.com/newsletters/?mj oac=Article Top No Oligarchs Mother Jones Daily . On Tuesday, experienced AI researcher Jacob Coxon resigned from the AI firm Anthropic—saying that both that company and OpenAI, his previous employer, were “gambling with our lives” by developing models that could improve themselves at a rapid clip until they reach “superintelligence.” In an alarming social media post https://x.com/EvanHub/status/2097497037956891126 , Evan Hubinger, who leads a team that tries to stress-test Anthropic’s models for safety, essentially agreed that the company’s staffers “really do earnestly believe AI could kill all humans ” It’s far from the first time researchers have tried to raise the alarm about AI’s dangers, but Coxon’s post https://x.com/hilbertspaess/status/2097476196791709843 and subsequent discourse went viral. The details of AI alignment can be difficult to grasp—a big part of the problem is that no one truly understands https://www.nytimes.com/2026/04/15/magazine/ai-black-box-interpretability-research.html the intricacies of how advanced models work. But it doesn’t take expert-level knowledge or insider secrets to understand the reasons for alarm. All you need is three facts about AI: It is already surpassing human abilities in important domains. Leading companies continue to improve it rapidly. And no one knows how to reliably keep its behavior in line with human goals. Yesterday, OpenAI shared a solution https://openai.com/index/navier-stokes-solution/ to one of the six remaining “Millenium Problems,” some of the most heavily researched in all of mathematics. Mathematicians working independently are also claiming credit https://www.science.org/content/article/how-ai-math-breakthrough-ignited-controversy —but they, too, relied on advanced models for their work. It was the most striking example yet, though not the first, of AI doing cutting-edge math. A few years ago, a common dismissal of large language models was the claim that they were just elaborate algorithms, creating sentences by guessing the most likely next word. But today’s AI models clearly build on, rather than simply remix, the text reflected in their training data. In some areas, computers or algorithms have outstripped human minds for a while. Chess machines are a well-known example, and as early as 2018, Google DeepMind debuted a machine learning algorithm that outclassed biochemists’ previous methods for predicting the structure of a protein from its sequence of amino acids. But new AI capabilities, including “critical” cybersecurity abilities https://www.cnbc.com/2026/09/01/open-ai-astra-cyber-model.html and the skills to design novel viruses https://www.bbc.com/news/articles/c5y3j3ngevmo , have rung alarm bells that more generalized artificial intelligence could be arriving. At the same time, leading companies continue to rapidly build better AI models, with few signs of any slowdown. Many in the industry are aiming to reach “recursive self-improvement,” in which top models would be able to rapidly build better versions of themselves that outclass human intelligence in ever more domains. In July, a broad swathe of industry leaders called https://www.pacingthefrontier.com/ for US government action and international collaboration to manage the pace of AI development for safety reasons, worrying that without collaboration, rival companies and countries will be incentivized to race into deeply dangerous territory. Some members of Congress have put substantial work into policy ideas. Policy experts told me in August https://www.motherjones.com/politics/2026/08/ai-safety-congress-doom/ that such legislation looks unlikely in this session of Congress, but recent news including Coxon’s viral statement has grabbed some legislators’ attention https://www.usatoday.com/story/news/politics/2026/09/09/congress-reaction-anthropic-ai-warning-end-humanity/91675031007/ . Finally , no one knows how to reliably keep AI in line with human goals. The most prominent recent example is the “Hugging Face incident,” in which hundreds of OpenAI agents coordinated a massive cyberattack, and individual agents were “sacrificing” themselves https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/ core-takeaways-about-this-incident for the benefit of collective goals. As I previously summarized it https://www.motherjones.com/politics/2026/08/ai-safety-openai-hugging-face-hacking-metr-report/ : OpenAI was testing its agents, the industry’s term for AI that autonomously performs digital tasks, in part by administering sometimes impossible cybersecurity problems. The agents found cheats to answer these problems and sought to trick an automated evaluation system into accepting them. They delegated work to each other to learn more about how to exploit the system—and the massive cyberattack on Hugging Face became part of that research. Last week, researchers detailed http://collusion.wiki a “swarm” of agents that placed 18,000 posts on an obscure German-language website to communicate with each other and cheat on evaluations of their abilities to quickly find online information. Subsequent research found messages on other sites https://collusion.wiki/additional-findings ; one trick the agents used was to share the sequence of questions, so that other agents going through the same evaluation could know them in advance. In both cases, AI agents were essentially just trying to cheat on tests. But it highlights what agents might do to achieve their goals, even when given innocuous instructions. And safety researchers have long worried that it will be devilishly tricky to make advanced models consistently integrate human values into their actions: If AI conducts cyberattacks to score better on tests, a future super-powerful model might hijack infrastructure that our lives depend on in pursuit of whatever its goals are. And beyond broad existential risks, AI poses dangers like helping bad actors design bioweapons or build powerful ransomware. The problem of understanding AI motivations could become even more difficult. OpenAI’s head of recursive self-improvement preparedness has said https://x.com/MicahCarroll/status/2095603855316996529 that its most recently released model represents “an important decrease in monitorability,” which refers to researchers’ ability to understand AI’s internal reasoning and predict its behavior. To those immersed in AI research and discourse, these are not new points. Many have been theorized for decades. OpenAI was founded in 2015 as a nonprofit aiming to ensure the technology would benefit humanity, and when some of its employees felt it wasn’t doing enough on safety and alignment, they quit to form Anthropic https://finance.yahoo.com/news/anthropic-ceo-says-why-quit-194409797.html in 2021. Now, Coxon wrote in his warning, Anthropic is “locked in a race to get there first – they believe no one else will act responsibly, so they must do it themselves.” CEO Dario Amodei has estimated that there is a “25 percent chance” that AI development goes “very, very badly.” But as AI safety grabs more public attention, here are the fundamentals: Leading researchers say that they are building machines that will surpass human intelligence and that they are not confident they can control. It’s clearer than ever that we’re in a dangerous place. Disclosure: The Center for Investigative Reporting, the parent company of Mother Jones, has sued OpenAI for copyright violations. OpenAI denies the allegations.