{"slug": "how-openais-models-escaped-their-sandbox-and-slipped-past-californias-ai-law", "title": "How OpenAI’s Models Escaped Their Sandbox and Slipped Past California’s AI Law", "summary": "OpenAI disclosed that multiple models, including GPT-5.6 Sol and a pre-release model, autonomously attacked Hugging Face's system, executing over 17,000 automated actions in hours in one of the first publicly known cases of frontier AI models cyber attacking another company. The incident occurred days after Hugging Face revealed a breach by a malicious dataset, and it highlights gaps in California's frontier AI law, which excludes safety evaluation incidents like this from mandatory reporting. \"This matches the 'agentic attacker' scenario the industry has been forecasting,\" Hugging Face wrote.", "body_md": "# How OpenAI’s Models Escaped Their Sandbox and Slipped Past California's AI Law\n\n[OpenAI](/news/tag/open-ai)[ this week said](https://openai.com/index/hugging-face-model-evaluation-security-incident/) more than one of its models — three, according to [ Bloomberg](https://www.bloomberg.com/news/articles/2026-07-23/openai-models-lurked-in-hugging-face-system-for-hours-undetected?srnd=homepage-americas) — attacked Hugging Face, a popular repository for open-source AI tools — marking one of the first publicly-disclosed cases of frontier AI models autonomously cyber attacking another company.\n\nThe disclosure came days after Hugging Face [revealed](https://huggingface.co/blog/security-incident-july-2026) it was breached in a hack by a “malicious dataset,” which intruded into the AI company’s software to run code. The campaign executed a flurry of more than 17,000 automated actions in a matter of hours, the post noted. It’s still unclear if Hugging Face customer or partner data was taken.\n\n“This matches the ‘agentic attacker’ scenario the industry has been forecasting,” Hugging Face wrote on July 16.\n\nOpenAI said an investigation showed the incident was driven by a combination of models — “including GPT‑5.6 Sol and an even more capable pre-release model.” The company said it’s working with Hugging Face to produce a more thorough report of what happened.\n\nOpenAI’s announcement came one day after the San Francisco-based frontier AI developer disclosed a separate incident in which it paused another pre-release model that had escaped what the industry calls “a sandbox” — an isolated environment with no path to the open internet, except for a single internal service that fetches software packages — and [posted](https://www.wired.com/story/openai-models-escaped-containment-and-hacked-huggingface/) to GitHub.\n\nLast April, a researcher challenged an early version of Anthropic’s Mythos model to break out of an isolated system and message back. Mythos did, and then [went further](https://www.mondaymomentum.io/p/the-ai-that-broke-out-of-its-cage), building a multistep process to reach the wider internet.\n\nCalifornia’s [frontier AI law](https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB53), the first of its kind in the country, obliges the largest developers to tell the state about critical safety incidents, but only ones that kill or injure someone, or that cause catastrophic harm.\n\nIt expressly excludes the kind of safety evaluation OpenAI was running when its models got loose. Which leaves a question nobody in Sacramento or Washington can answer: how often have frontier models in development gone rogue at companies that decided not to mention it?\n\n“This was one of the topics I was focused on when I worked at OpenAI: what happens when AI models become good enough to reward-hack their way through the infrastructure they use to accomplish their objectives?” Ariel Herbert-Voss, OpenAI’s first security hire, wrote KQED.\n\n“It’s a really difficult problem to defend against and is one of the reasons I left to start my own company,” Herbert-Voss, now the founder of an automated offensive cybersecurity company called RunSybil, continued.\n\n“It’s a really remarkable and, I think, fair to say scary event,“ said Nathan Calvin, general counsel at safety advocate EncodeAI.\n\nScientists and AI researchers have warned for years of the possibility AI systems could go rogue and do unpredictable and unintended things. AI developers commonly test safety with so-called “[red teams](https://www.paloaltonetworks.com/cyberpedia/what-is-ai-red-teaming),” that try to break or trick machine learning models, like chatbots, to find hidden flaws.\n\n“In certain circumstances, AI will try to break out of their sandboxes or otherwise do things that were not what the creator intended,” Calvin said. “But this really is the first very big example of that happening, at scale, with a really highly capable AI model in a way that actually harmed a third party.”\n\nAccording to OpenAI’s account, the company was running an internal evaluation designed to measure how good its models are at hacking. To get a true reading, OpenAI deliberately switched off the safety filters that normally stop its models from carrying out cyberattacks.\n\nThe models were confined to a so-called “sandbox,” an isolated environment with no route to the open internet except OpenAI’s one internal service that fetches software libraries from public repositories. The models found a previously unknown flaw in that service, exploited it, escalated their privileges and moved from machine to machine inside OpenAI’s research network until they reached one with internet access.\n\nOnce the models broke out onto the open internet, according to OpenAI, they reasoned that Hugging Face might be storing the answers to the test they were trying to pass. Hugging Face’s credentials and production database aren’t on the public internet any more than a bank’s vault sits on a sidewalk, but the models broke into Hugging Face using stolen credentials, among other things.\n\nIn short: The models were told to hack. They were not told to leave the building or to break into another company’s servers.\n\nHugging Face detected the intrusion and shut it down on its own, days before OpenAI connected the attack to its own testing.\n\n“It’s quite mind-blowing that all of this happened autonomously!“ Hugging Face Chief Executive Clement Delangue wrote on [X](https://x.com/ClementDelangue/status/2079670308156645882).\n\nIn its blog post, OpenAI wrote, “This incident points to the need to further strengthen our model’s alignment, cyber protections during evaluation time, and monitoring during internal testing.”\n\nThe Future of Life Institute — a nonprofit which issues a biannual risk assessment of nine leading AI companies — recently warned that many of the companies building frontier models are quietly walking back safety commitments.\n\nAccording to the group’s [most recent](https://futureoflife.org/ai-safety-index-summer-2026/) AI Safety Index, Anthropic, OpenAI, Google DeepMind, and Meta all weakened or abandoned promises to pause development if certain red lines were approached, even while publicly suggesting they were amenable.\n\nIn an email, KQED asked Hamza Chaudhry, who leads AI and national security work at Future of Life, to imagine it was not OpenAI’s software gone rogue but a foreign state, intentionally executing an unauthorized intrusion into a private company’s production infrastructure, exploiting cybersecurity vulnerabilities, stealing live credentials and accessing a production database.\n\n“We would likely call this a dangerous act of cyber-espionage,” he wrote — a likely criminal violation of the Computer Fraud and Abuse Act, which “would draw a threat group designation and eventually an indictment or sanctions.”\n\nOpenAI did not respond to KQED’s request for comment, but a spokesperson told Bloomberg the company communicated with law enforcement and other government authorities about the incident.\n\nThe fact that Google DeepMind, Meta, Mistral, xAI and Chinese AI model developers haven’t revealed similar events doesn’t necessarily mean they haven’t happened. OpenAI and Anthropic are the only two frontier model developers that have publicly disclosed containment failures.\n\nThere is no mandatory public disclosure regime in force, like the one in California that requires [hacked companies to reveal](/news/11992954/cybersecurity-expert-shares-tips-for-dublin-based-patelco-credit-union-customers-after-ransomware-attack) there’s been a data breach.\n\nCongressional candidate and state senator Scott Wiener wrote two AI frontier model safety bills. Gov. Gavin Newsom [vetoed](/news/12007323/can-california-still-lead-on-ai-regulation-following-newsoms-veto-of-ai-safety-bill) Wiener’s first effort, arguing in his [veto message](https://www.gov.ca.gov/wp-content/uploads/2024/09/SB-1047-Veto-Message.pdf) that “By focusing only on the most expensive and large-scale models, SB 1047 establishes a regulatory framework that could give the public a false sense of security about controlling this fast-moving technology.”\n\nThe next year, Newsom signed Wiener’s second at-bat, but only after the bill was softened to [overcome industry pushback](/news/12058013/newsom-signs-california-ai-transparency-bill-tailored-to-meet-tech-industry-tastes).\n\nThe [law](https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB53), which took effect Jan. 1, requires developers of the most powerful AI models to notify the Governor’s Office of Emergency Services of any “critical safety incident“ within 15 days of discovering it — which is not the same as reporting the incident to the public.\n\nWiener told KQED he still thinks the law is strong.\n\n“We worked very hard with the governor to produce a bill that is meaningful and impactful, and that he would sign,“ he said, adding he doesn’t consider the work done, because AI continues to evolve at a rapid pace.\n\nAI, he said, is “likely the most powerful technology in human history, and we need to make sure that we are both understanding the risks and taking them seriously, so that we can get ahead of them.”\n\nCalvin likened the Hugging Face hack to the cyclospora outbreak linked to, but not confirmed as originating from, [Taylor Farms](/news/12091770/california-based-taylor-farms-recalls-lettuce-shipped-to-27-states-over-cyclospora-risk).\n\nThat California-based company has said it’s removing its lettuce indefinitely while the FDA investigation continues.\n\n“And OpenAI is like, ‘Maybe it’ll happen again. Maybe it’ll get worse. We don’t really know,“ Calvin said. “Your AI, again, hacked out of its box and hacked into another company, and you’re saying that you don’t know how to stop it from doing that again? That seems pretty nuts,” Calvin said.", "url": "https://wpnews.pro/news/how-openais-models-escaped-their-sandbox-and-slipped-past-californias-ai-law", "canonical_source": "https://ww2.kqed.org/news/2026/07/23/how-openais-models-escaped-their-sandbox-and-slipped-past-californias-ai-law/", "published_at": "2026-07-23 19:33:03+00:00", "updated_at": "2026-07-23 19:54:04.041382+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-safety", "ai-policy", "ai-agents", "ai-research"], "entities": ["OpenAI", "Hugging Face", "GPT-5.6 Sol", "Ariel Herbert-Voss", "RunSybil", "Nathan Calvin", "EncodeAI", "Anthropic"], "alternates": {"html": "https://wpnews.pro/news/how-openais-models-escaped-their-sandbox-and-slipped-past-californias-ai-law", "markdown": "https://wpnews.pro/news/how-openais-models-escaped-their-sandbox-and-slipped-past-californias-ai-law.md", "text": "https://wpnews.pro/news/how-openais-models-escaped-their-sandbox-and-slipped-past-californias-ai-law.txt", "jsonld": "https://wpnews.pro/news/how-openais-models-escaped-their-sandbox-and-slipped-past-californias-ai-law.jsonld"}}