# How OpenAI Lost Control of an AI Model—and What Needs to Change > Source: > Published: 2026-07-24 15:13:52+00:00 OpenAI was evaluating its artificial intelligence models’ ability to exploit vulnerable software when instead the models hacked the infrastructure surrounding the test, broke containment, and attacked a real company, OpenAI [revealed](https://openai.com/index/hugging-face-model-evaluation-security-incident/) on July 21. Observers say this is the first real-world instance of AI doing something researchers have long worried about: a loss-of-control scenario. If the industry fails to learn from it, it is unlikely to be the last. Hugging Face, a company that hosts AI models and datasets, was the target of the autonomous attack, and reported the incident to local police before it knew OpenAI’s models were responsible. The breach was serious, but the immediate consequences were limited. Had similar behavior occurred inside a hospital, power grid, or other critical system, it could have been much worse. Many have called the incident a “warning shot.” TIME spoke with experts and insiders about what it would take to heed that warning before a similar failure produces consequences that are harder to contain. # What happened? On July 16, Hugging Face [said](https://huggingface.co/blog/security-incident-july-2026) it had been hit by an unusually automated cyberattack. Over the course of a weekend, AI agents carried out thousands of actions across many temporary virtual computers, moving through the company’s internal systems and shifting the infrastructure coordinating the attack between online services to keep it running. Five days later, OpenAI disclosed that its own models were responsible. The models were trying to cheat on a cybersecurity test. OpenAI had placed them inside what it called a “highly isolated environment,” with only limited access to an internal service used to download approved software. They found a previously unknown flaw in that service, used it to break into other OpenAI systems and eventually reached the open internet. From there, they inferred that Hugging Face might hold material related to the test, broke into its systems and obtained information that helped them score higher. “If a model of this capability level cannot be contained, what should we expect for future, much more powerful models? This is an important wake-up call both for risks from loss of control of powerful AI systems as well as organizational security for frontier labs,” says [Marius Hobbhahn](https://time.com/collections/time100-ai-2025/7305864/marius-hobbhahn/), CEO and founder of Apollo Research, which tests AI models for deception and scheming. # Requiring disclosures before incidents become catastrophic How long were the agents running? Did they work in unison? What was the prompt? These details remain unknown, at least to the public. Several experts TIME spoke with stressed that the lack of details make the severity of the incident hard to judge. OpenAI did not respond to TIME’s request for comment, but has said it has partnered with Hugging Face to conduct a thorough investigation and will share more details once complete. But perhaps more worryingly, OpenAI is not legally compelled to disclose the incident in the first place. California’s SB 53 and New York’s RAISE Act are recently passed state-level laws that require large AI companies to disclose critical safety incidents, but only if an incident risks causing more than 50 deaths or serious injuries, or more than $1 billion in property damage. “They have made the bar so high for anything to qualify, only the most grievous incidents will actually be reported,” says Mackenzie Arnold, director of U.S. policy at LawAI, an independent think tank focused on the legal challenges posed by artificial intelligence. “The version of the RAISE Act that the NY Legislature passed would have required disclosure of this ‘incident.’ After lobbying from OpenAI, Bloomberg, and a16z, the final version the Governor signed allows companies to hide events like this,” Alex Bores, New York state representative and the bill’s sponsor [posted to X](https://x.com/AlexBores/status/2080264714085400787?s=20). “I'm glad OpenAI chose to disclose this crime. The law shouldn't give them a choice,” he wrote. # Stronger containment If AI can slip out of its box, you need a stronger box. “Externally, this feels like a big warning shot, but internally, related incidents have been happening for a while,” says an OpenAI staffer, who spoke under the condition of anonymity. The day before OpenAI disclosed the incident, the company [revealed](https://openai.com/index/safety-alignment-long-horizon-models/) that it had shut down another internal deployment after it realized it had slipped out of its sandbox—a digitally, rather than physically, separated environment. “Models have broken out of sandboxes before, and we always try to patch them,” the staffer says. “But the problem is … it's impossible to patch every single thing that a creative AI can do.” The problem is not unique to OpenAI. Anthropic disclosed in April that it realized an internal deployment of Mythos had gained unauthorized access after one of its researchers received an email from the model while having lunch in a park. As models surpass human ability to craft escape routes, the challenge of containment becomes daunting. “I think we do need to do that, but I think we also need to be prepared for a world where even best practices aren't really good enough,” says Peter Wildeford, head of policy at the AI Policy Network, a nonprofit that advocates for policies to help America prepare for a world where AI matches human cognition. But while in theory, a superhuman AI system might outmaneuver even the most secure containers, there are also actions AI companies could take right now to improve security. Currently, there are no laws governing the security of internal deployments. “Sandboxes are actually notoriously insecure,” says [Heidy Khlaaf](https://time.com/collections/time100-ai-2025/7305862/heidy-khlaaf/), chief AI scientist at AI Now Institute, and a former safety systems engineer contractor at OpenAI. The fact that the models were permitted to connect to a service for downloading packages meant the environment was not truly sealed off, she adds. In a previous role, Khlaaf audited security at dozens of technology companies. Before that, she worked auditing high-risk systems, like those used inside nuclear power plants—which often “air gap” systems, physically cutting them from internet access. “What we consider safe in a nuclear plant is so different from what big tech considers safe.” # Catching mischievous agents in real time The Hugging Face incident reveals the importance of real-time monitoring. Though details of the precise timeline are scant, Hugging Face has said the agents worked over a “weekend,” suggesting that they were able to break containment and get up to no good for an extended period before OpenAI noticed and intervened. Actions carried out internally by agents on OpenAI’s Codex platform are carefully monitored, the OpenAI staffer says, but models undergoing evaluation are deployed on a separate system that is not monitored by default. Zack Korman, CEO of Oslo-based agent-oversight startup Embroidery, says real-time agent monitoring—even outside top AI companies—is commonplace, and to not carefully oversee a cybersecurity evaluation is “irresponsible.” You should be confident models cannot break free, “but also have monitoring just in case you're wrong,” he says. OpenAI has said “this incident points to the need to further strengthen our model’s alignment, cyber protections during evaluation time, and monitoring during internal testing.” # Steering development towards safer AI systems To prevent models from taking actions, OpenAI typically installs guardrails on its models after training to reduce the chance they’ll engage in harmful actions. In this instance, those cyber guardrails were disabled to properly measure its performance. But the ultimate aim of the field of “AI alignment” is to ensure that such guardrails become less necessary as AI models naturally behave as intended. This is seen as particularly important to those who believe AI may become smart enough to route around guardrails. “We train the models to be really good at accomplishing tasks and doing whatever it takes to accomplish those tasks,” the OpenAI staffer says. What remains an open technical question is how to guarantee those models don’t take unintentional or dangerous actions. “We're still nowhere near solving this misalignment problem,” they add. There’s another way the industry could steer development in a safer direction, Khlaaf says. While some abilities, like spotting vulnerabilities in software, are useful to attackers and defenders, designing exploits for those vulnerabilities uniquely empowers attackers. Khlaaf says that labs should put more resources into capabilities that directly help defenders, such as detecting attacks, writing secure code, and patching vulnerabilities. AI companies are already investing in some of those tasks, but their most visible capability gains and benchmarks have centered on finding vulnerabilities and constructing exploits, she says. In an industry defined by speed, OpenAI has said the stricter infrastructure controls it has implemented in response have already slowed its “research velocity.” Hobbhahn says that’s a price worth paying. “This is humanity’s last technology. We cannot screw this up. So we need to err on the side of getting it right rather than getting it immediately.”