{"slug": "openai-ai-models-went-rogue-during-testing-triggering-unprecedented-breach-at", "title": "OpenAI AI models went rogue during testing, triggering 'unprecedented' breach at startup", "summary": "OpenAI said on Tuesday that one of its autonomous AI agents escaped a controlled security test, reached the internet, and hacked into AI startup Hugging Face last week, marking what the company called an unprecedented cyber incident involving state-of-the-art cyber capabilities. Hugging Face used Chinese startup Zhipu AI's GLM-5.2 model to contain the attack because leading U.S. models refused to process the attacker data. The incident has intensified concerns about the risks of frontier AI models and prompted calls from Representative Greg Casar for mandatory safety testing and regulation.", "body_md": "By Raphael Satter\n\nWASHINGTON, July 21 (Reuters) - OpenAI said on Tuesday that an autonomous agent powered by its advanced artificial intelligence models went rogue during a security test and triggered a hack that compromised the infrastructure of AI startup Hugging Face last week.\n\nThe ChatGPT creator was testing capabilities of some of its most advanced models in a controlled environment, but the agent escaped containment, reached the internet and broke into Hugging Face to satisfy its testing goal.\n\nThe incident signals that AI's expanding capabilities are already fueling the security threat experts long feared and even top developers can be caught off-guard by flaws their models can exploit.\n\nThe breakout was \"an unprecedented cyber incident, involving state-of-the-art cyber capabilities\" and OpenAI is reinforcing its safeguards, the company said in a blog post.\n\nIt also drew attention as New York-based Hugging Face said it had used an open-source Chinese model to contain the attack because leading U.S. models, unable to tell a defender from an attacker, refused to process the data needed for analysis.\n\nThe company said in a blog post last week that it used Zhipu AI's GLM-5.2 for the analysis, which also allowed it to keep attacker data and any credentials within its systems.\n\nGLM-5.2 and Beijing-based Moonshot's Kimi K3 have stirred Silicon Valley recently with capabilities nearing those of top U.S. models at lower costs and without the guardrails that block their American rivals from use in tasks such as cybersecurity.\n\n\"When a frontier model is attacking you and moving laterally inside your infrastructure, defenders need wide access to near-frontier tools within hours or even minutes, rather than being pointed towards a closed-door, vetted application programme for model access,\" Hugging Face Co-founder Thomas Wolf said on X.\n\nSIGN OF THINGS TO COME\n\nThe hack at Hugging Face, which hosts open-source large language models and datasets, rattled the cybersecurity community after the company said last week the breach \"was different from anything we had handled before\" and \"was driven, end to end, by an autonomous AI agent system.\"\n\nOpenAI's disclosure that its advanced models were responsible for the breach, despite having placed them in what it described as \"a highly isolated environment,\" will likely intensify disquiet over the power and risk of frontier models.\n\nRepresentative Greg Casar, a Texas Democrat, said the incident was alarming.\n\n\"AI is developing extremely fast with no real regulations to keep us safe,\" he said in a statement, calling for mandatory independent safety testing, mandatory disclosure of security incidents, and international cooperation \"to keep people safe from absolute disaster.\"", "url": "https://wpnews.pro/news/openai-ai-models-went-rogue-during-testing-triggering-unprecedented-breach-at", "canonical_source": "https://ca.finance.yahoo.com/news/openai-ai-models-went-rogue-113513486.html", "published_at": "2026-07-22 11:35:13+00:00", "updated_at": "2026-07-22 12:23:07.957759+00:00", "lang": "en", "topics": ["ai-safety", "ai-agents", "ai-policy", "ai-research"], "entities": ["OpenAI", "Hugging Face", "Zhipu AI", "GLM-5.2", "Moonshot", "Kimi K3", "Thomas Wolf", "Greg Casar"], "alternates": {"html": "https://wpnews.pro/news/openai-ai-models-went-rogue-during-testing-triggering-unprecedented-breach-at", "markdown": "https://wpnews.pro/news/openai-ai-models-went-rogue-during-testing-triggering-unprecedented-breach-at.md", "text": "https://wpnews.pro/news/openai-ai-models-went-rogue-during-testing-triggering-unprecedented-breach-at.txt", "jsonld": "https://wpnews.pro/news/openai-ai-models-went-rogue-during-testing-triggering-unprecedented-breach-at.jsonld"}}