{"slug": "anthropic-urges-top-ai-labs-to-slow-development-over-self-improvement-risks", "title": "Anthropic urges top AI labs to slow development over self-improvement risks", "summary": "Anthropic called on leading AI labs and governments to consider slowing or temporarily pausing frontier AI development, citing internal data showing that Claude now writes over 80% of code merged into Anthropic’s codebase and that engineers produce eight times more code per day than in 2024. The company warned that AI systems are accelerating their own improvement, raising the risk of recursive self-improvement arriving faster than institutions can prepare. Anthropic said a credible, verifiable pause across multiple well-resourced labs in different countries could give policymakers and researchers more time to address risks, but cautioned that a unilateral halt would be ineffective and could cede leadership to less cautious actors.", "body_md": "# Anthropic urges top AI labs to slow development over self-improvement risks\n\nThe Claude maker says the ability to slow global AI development would 'likely be a good thing,' citing internal data on how fast its own models are improving.\n\nAnthropic said the world should have the option to slow or temporarily pause frontier AI development if leading AI labs and governments can create a credible way to verify that everyone is complying.\n\nIn a new [report](https://www.anthropic.com/institute/recursive-self-improvement) from the Anthropic Institute, the company said a pause could help give policymakers, researchers, and civil society more time to address the risks tied to increasingly capable AI systems. But the firm warned that a slowdown would only improve safety if it applied across multiple well resourced labs at or near the frontier.\n\nAnthropic said a unilateral pause by one company would be easier to implement but far less effective, because it could simply hand the lead to less cautious actors. The company said any meaningful pause would require developers in multiple countries to stop under the same conditions and verify that competitors had also stopped.\n\nThe warning comes as Anthropic says AI systems are already accelerating the development of new AI models. As of May 2026, more than 80% of code merged into Anthropic’s codebase was authored by Claude, up from the low single digits before Claude Code launched in research preview in February 2025.\n\nThe company said the typical Anthropic engineer merged eight times as much code per day in the second quarter of 2026 as they did in 2024. A March 2026 internal poll also found that research staff using Mythos Preview estimated they were producing roughly four times as much output as they would without AI models.\n\nAnthropic said those gains point to a broader shift in how frontier AI systems are built. Human engineers and researchers are still setting goals, reviewing outputs, and deciding which problems matter, but Claude is taking on more of the execution work across coding, testing, debugging, and experiment optimization.\n\nThe company framed that trend as an early step toward recursive self improvement, a scenario in which AI systems become capable of autonomously designing and developing their own successors. Anthropic said such a system does not exist yet and may not be inevitable, but warned it could arrive sooner than most institutions are prepared for.\n\nIf that happens, Anthropic said the pace of AI development could become tied more directly to compute availability and efficiency gains, while humans shift toward oversight, validation, and verification.\n\nThat would make the systems used to monitor frontier AI development more important, especially if AI models begin playing a larger role in building future versions of themselves.\n\nAnthropic said the challenge is that verifying a pause in AI development is harder than monitoring many other technologies. Training runs can be concealed, their inputs are general purpose, and the incentive to quietly defect would be enormous if one actor could gain the lead while others stop.\n\nThe company said a credible pause would need clear rules around what triggers it, what ends it, and who decides whether the conditions have been met. It compared the challenge to arms control regimes for other complex technologies, while noting that those systems took decades to build and that AI may not leave governments that much time.\n\nAnthropic said it plans to organize conversations in the coming months with policymakers, researchers, civil society, and other AI companies to examine recursive self improvement and possible coordination mechanisms for frontier AI development.\n\nThe company said it would expect to slow or temporarily pause development if other frontier developers also did so in a verifiable way.\n\n**Disclosure:** This article was edited by Estefano Gomez. For more information on how we create and review content, see our\n\n[Editorial Policy](https://cryptobriefing.com/editorial-policy/).", "url": "https://wpnews.pro/news/anthropic-urges-top-ai-labs-to-slow-development-over-self-improvement-risks", "canonical_source": "https://cryptobriefing.com/anthropic-urges-ai-labs-slow-development/", "published_at": "2026-06-05 15:59:08+00:00", "updated_at": "2026-06-05 16:57:52.633642+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-safety", "ai-policy", "ai-research", "large-language-models"], "entities": ["Anthropic", "Claude", "Anthropic Institute"], "alternates": {"html": "https://wpnews.pro/news/anthropic-urges-top-ai-labs-to-slow-development-over-self-improvement-risks", "markdown": "https://wpnews.pro/news/anthropic-urges-top-ai-labs-to-slow-development-over-self-improvement-risks.md", "text": "https://wpnews.pro/news/anthropic-urges-top-ai-labs-to-slow-development-over-self-improvement-risks.txt", "jsonld": "https://wpnews.pro/news/anthropic-urges-top-ai-labs-to-slow-development-over-self-improvement-risks.jsonld"}}