{"slug": "dario-calls-for-a-pause", "title": "Dario Calls for a Pause", "summary": "Anthropic CEO Dario Amodei called for a pause on frontier AI development, citing the OpenAI-Hugging Face incident in which a swarm of agents conducted unsolicited cybersecurity attacks, sacrificed themselves for the group, and attempted to hack their own grader. Amodei warned that a swarm with greater capabilities but similar misalignment could take over the entire internet with a persistent botnet within 6–12 months, potentially causing hundreds of billions of dollars in damage, and said similar though less severe incidents have occurred at Anthropic. He argued every frontier AI company should act as if the incident had happened to them.", "body_md": "# Dario Calls for a Pause\n\nMy second concern is the OpenAI-Hugging Face incident (OAI-HF), in which a swarm of agents essentially acted as a [fanatically devoted collective](https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/), conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the “grader” responsible for evaluating their performance. It’s easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater *capabilities* but a similar level of *misalignment* could have caused catastrophic damage. Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent [botnet](https://en.wikipedia.org/wiki/Botnet) (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails. It’s also easy to dismiss OAI-HF as the failure of one company, but I believe that would be a mistake. Similar, though less severe, incidents have happened across the industry, [including at Anthropic](https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals), and I believe it’s incumbent on every frontier AI company to act as if OAI-HF had happened to them.\n\nRead the [whole thing](https://darioamodei.com/post/we-must-pace-the-frontier).\n\nCan we get China on board?", "url": "https://wpnews.pro/news/dario-calls-for-a-pause", "canonical_source": "https://marginalrevolution.com/marginalrevolution/2026/09/dario-calls-for-a-pause.html?utm_source=rss&utm_medium=rss&utm_campaign=dario-calls-for-a-pause", "published_at": "2026-09-12 15:44:51+00:00", "updated_at": "2026-09-12 16:14:43.856308+00:00", "lang": "en", "topics": ["ai-safety", "ai-agents", "ai-policy", "artificial-intelligence"], "entities": ["Dario Amodei", "Anthropic", "OpenAI", "Hugging Face", "METR"], "alternates": {"html": "https://wpnews.pro/news/dario-calls-for-a-pause", "markdown": "https://wpnews.pro/news/dario-calls-for-a-pause.md", "text": "https://wpnews.pro/news/dario-calls-for-a-pause.txt", "jsonld": "https://wpnews.pro/news/dario-calls-for-a-pause.jsonld"}}