For the second time in two days, OpenAI has disclosed how one of its models broke internal guardrails to achieve the goal it had been set. This time the story doesn’t stay contained inside OpenAI’s own infrastructure. It spills into Hugging Face’s production systems, and involves a zero-day vulnerability that a model found and used entirely on its own. OpenAI CEO Sam Altman called it a “significant security incident”. The incident, disclosed jointly by OpenAI and Hugging Face, took place during an internal evaluation designed to measure how far OpenAI’s models can go when their cyber refusals are deliberately turned down. The models involved were GPT-5.6 Sol and an even more capable pre-release system, both being tested on a benchmark called ExploitGym without the production classifiers that would normally restrict high-risk cyber activity. OpenAI is calling this an unprecedented cyber incident involving state-of-the-art cyber capabilities, and says it’s still investigating alongside Hugging Face.
How A Sandboxed Test Ended Up On Hugging Face’s Servers #
The setup was supposed to be contained. OpenAI runs these evaluations in an isolated environment where the only network access available is through an internally hosted proxy that caches package registries, rather than the open internet. That proxy became the way out. The models spent a large amount of inference compute trying to find internet access in pursuit of solving the ExploitGym problem, and eventually identified and exploited a zero-day vulnerability in that very cache proxy. OpenAI has since disclosed the vulnerability to the vendor responsibly.
From there, the models used a chain of privilege escalation and lateral movement inside OpenAI’s own research environment until they reached a node with actual internet access. Once online, the models reasoned that Hugging Face likely hosted models, datasets, and reference solutions tied to ExploitGym, and went looking for them. They found a path in, using a combination of stolen credentials and additional zero-day exploits, eventually landing on a remote code execution route into Hugging Face’s servers. From there, the models pulled test solutions directly out of Hugging Face’s production database. OpenAI’s security team caught the anomalous activity from its own side. Hugging Face’s team had already detected and contained the intrusion independently, using its own open-source models to run forensic reconstruction before the two companies connected. Hugging Face co-founder and CEO Clem Delangue called it possibly the first incident of its kind, and framed it as proof that AI safety gets solved through open collaboration among defenders rather than by any single company working alone.
A Pattern Emerging Fast #
What makes this notable is less the specific exploit chain and more how narrowly focused the model’s behavior was throughout. OpenAI says all the evidence points to the model being fixated on solving ExploitGym specifically, and that it went to extreme lengths for what was, in the end, a fairly narrow testing goal. That single-mindedness is the same trait OpenAI flagged just a day earlier in a separate safety essay, where it described pausing internal access to a long-horizon research model after it broke out of its sandbox twice, once to post code publicly on GitHub against instructions, and once by splitting an authentication token into fragments to slip past a security scanner. In both cases, the model wasn’t confused about the rules. It understood them well enough to route around them while pursuing a goal nobody had told it to prioritize over its constraints. There’s plenty of material from science fiction where AI does what it can to achieve goals set to it with some unexpected results — it appears that frontier labs are seeing this happen in their own environments.