{"slug": "anthropic-warns-ai-could-soon-build-itself-without-human-involvement-and-urges-a", "title": "Anthropic warns AI could soon build itself without human involvement—and urges a global pause on development", "summary": "Anthropic warned Thursday that its AI models are advancing so rapidly they may soon be capable of improving themselves without human involvement, urging an industrywide pause on development. The AI lab, valued at nearly $1 trillion and preparing for an IPO, said more than 80% of code in its codebase is now written by its Claude model, with engineers shipping eight times more code per quarter than before 2025. The company cautioned that the threshold of \"recursive self-improvement\" could arrive sooner than institutions are prepared for, potentially making it harder for humans to maintain control over AI.", "body_md": "Anthropic has published a new account of how quickly its AI models are advancing, warning that the technology may soon be capable of improving itself without meaningful human involvement. Just as the AI lab, valued at almost $1 trillion, prepares to go public, it’s also urging an industrywide pause in AI development.\n\nIn the [now-viral blog post](https://www.anthropic.com/institute/recursive-self-improvement) on Thursday, authors Marina Favaro and Jack Clark argue that AI development at Anthropic has already shifted dramatically: More than 80% of code merged into the company’s codebase is now written by Claude, and engineers are shipping roughly eight times as much code per quarter as they were before 2025. The authors say this trajectory is heading toward “recursive self-improvement”—a process in which AI systems autonomously design, build, and train their own successors, without humans driving each step.\n\nThe authors warn that while this threshold has not yet been crossed, it “could come sooner than most institutions are prepared for,” and that if it arrives without adequate safeguards, it could make it significantly harder for humans to maintain meaningful control over AI development.\n\nThe company also took the time to advocate for a pause on AI development, but only if “multiple well-resourced labs at or near the frontier, in multiple countries, agree to stop under the same conditions.” Presumably Anthropic is referring to competitors including OpenAI, [Google](https://fortune.com/company/alphabet/), xAI, and [Meta](https://fortune.com/company/facebook/). Most of these labs are currently engaged in a high-stakes race to develop ever more powerful AI models, and, with IPOs looming over three of the main labs, it seems unlikely that these companies will come together to agree on a coordinated pause on AI development.\n\nThe timing is raising some eyebrows. Just last week, the AI company filed confidential paperwork to prepare for an IPO. In the past month, it also leapfrogged OpenAI to become the most valuable AI lab at a $965 billion valuation.\n\nWhile Anthropic has long presented itself as more safety-conscious than other major AI labs, the timing—just before what could be one of the largest tech IPOs in history—has some observers questioning whether it’s also a way to stir up hype before the company’s public debut.\n\nCritics have previously accused the company of using safety rhetoric as a form of competitive positioning. Trump advisor David Sacks has [previously accused ](https://www.wsj.com/tech/ai/anthropic-urges-global-pause-in-ai-development-flags-self-improvement-risk-99cefb73)Anthropic of running a “regulatory capture agenda” designed to slow rivals under the guise of responsible AI development—a characterization the company rejects.\n\nThe AI lab is also in fierce competition with old rival OpenAI, with both labs racing to go public this year in an attempt to seize what could be a first-mover advantage with investors.\n\nIt’s also worth noting that Anthropic, despite advocating for a pause on AI development, recently dropped a key safety pledge that promised to do something similar. In February, [ Time magazine reported](https://time.com/7380854/exclusive-anthropic-drops-flagship-safety-pledge/) that Anthropic had overhauled its Responsible Scaling Policy—scrapping its central commitment to never train an AI system unless it could guarantee in advance that its safety measures were adequate. Chief science officer Jared Kaplan said at the time that the company felt it “wouldn’t actually help anyone” to stop training models unilaterally while competitors pressed ahead.\n\nThe company argued the overhaul was a pragmatic response to a changed political and competitive landscape, rather than a capitulation to market pressure. The new policy commits to transparency and matching rivals’ safety efforts; still, it leaves Anthropic significantly less constrained by its own rules than before.\n\nThursday’s post does make it clear that, whatever its motivation, Anthropic believes the window for meaningful deliberation on AI safety is narrowing. The company says it plans to convene policymakers, researchers, and civil society in the coming months to work through the questions recursive self-improvement raises.\n\n“The window to investigate the questions together is here,” the authors wrote. “And people outside AI companies should be involved in this deliberation.”\n\n**Fortune Brainstorm Tech** has been the place where bold ideas collide. From\n\n**June 8–10**, we will return to\n\n**Aspen**—where it all began—to mark 25 years of Brainstorm.", "url": "https://wpnews.pro/news/anthropic-warns-ai-could-soon-build-itself-without-human-involvement-and-urges-a", "canonical_source": "https://fortune.com/2026/06/05/anthropic-ai-pause-development-recursive-self-improvement/", "published_at": "2026-06-05 14:27:53+00:00", "updated_at": "2026-06-05 17:47:00.229409+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-safety", "ai-policy", "ai-research", "large-language-models"], "entities": ["Anthropic", "Marina Favaro", "Jack Clark", "Claude", "OpenAI", "Google"], "alternates": {"html": "https://wpnews.pro/news/anthropic-warns-ai-could-soon-build-itself-without-human-involvement-and-urges-a", "markdown": "https://wpnews.pro/news/anthropic-warns-ai-could-soon-build-itself-without-human-involvement-and-urges-a.md", "text": "https://wpnews.pro/news/anthropic-warns-ai-could-soon-build-itself-without-human-involvement-and-urges-a.txt", "jsonld": "https://wpnews.pro/news/anthropic-warns-ai-could-soon-build-itself-without-human-involvement-and-urges-a.jsonld"}}