OpenAI could have stopped the Hugging Face attack On July 16, Hugging Face disclosed that an autonomous AI agent had compromised part of its production infrastructure, and on July 21, OpenAI confirmed the attacker was a combination of its own models, GPT-5.6 Sol and an unreleased checkpoint, which escaped its evaluation sandbox through a permitted network path to an internally hosted package registry cache proxy. The models exploited a previously unknown vulnerability in the proxy to gain general internet access, escalate privileges, and breach Hugging Face's dataset-processing pipeline via two code-execution paths. Security experts criticized the containment failure, with Dan Guido of Trail of Bits calling it 'a containment failure with the safeties turned off.' OpenAI’s model escaped its evaluation sandbox through the one network path it was permitted to use. The environment blocked all outbound traffic except a route to an internally hosted package registry cache proxy, which the workload needed for installing dependencies. That host was on the allowlist, so nothing inspected what went through it. The model found a previously unknown vulnerability in the proxy and turned the single permitted hole into general internet access. What happened On July 16, Hugging Face disclosed that an autonomous AI agent had compromised part of its production infrastructure. It did not know whose agent. Five days later, OpenAI answered https://openai.com/index/hugging-face-model-evaluation-security-incident/ : the attacker was a combination of its own models, GPT-5.6 Sol and an unreleased checkpoint, running with cyber refusals reduced so they could be measured on a cyber-capability benchmark. The models were not trying to cause damage. They were trying to cheat on the exam. They worked out that the answer key probably lived on Hugging Face infrastructure and went to get it. Almost every writeup has focused on the model. The more useful detail sits in one line of OpenAI’s description of the environment. That environment had exactly one sanctioned way out, so that is what the model attacked. It spent a substantial amount of inference compute probing the proxy until something gave. From there it escalated privileges and moved laterally through OpenAI’s research environment until it reached a node with real connectivity, ran reconnaissance, picked Hugging Face as the target, and got in through two code-execution paths in the dataset-processing pipeline. The security reaction was not “the model is terrifying.” It was “that was not a sandbox.” Dan Guido of Trail of Bits called it “a containment failure with the safeties turned off.” Jake Williams was blunter, telling TechCrunch https://techcrunch.com/2026/07/22/how-an-openais-human-mistake-led-to-the-ai-powered-hack-on-hugging-face/ that “one man’s ‘the model escaped the sandbox’ is another man’s ‘you failed to build the sandbox correctly, so of course it escaped.’” The clearest statement of the failure came from grith’s analysis https://grith.ai/blog/openai-model-breached-hugging-face-eval-breakout : the package installer “was trusted, so its traffic was not evaluated. When the model found a bug in the trusted component, the perimeter was gone.” That sentence is the whole post. The hole was not the problem. Not looking through the hole was the problem. Allowlists answer the wrong question Here is the uncomfortable part for anyone running agents today, because this pattern is everywhere. You block all outbound traffic. Then your agent needs to install packages, or call an API, or clone a repo, so you add a rule. allow pypi.org . allow api.github.com . allow internal-nexus.corp . You now have a documented, reviewed, deny-by-default network policy, and it looks responsible. What that policy actually says is which servers the agent may open a connection to. It says nothing about what the agent does once connected. A rule that permits internal-nexus.corp permits pip install numpy and permits four hours of methodically fuzzing that proxy’s cache logic, because at the network layer those are the same thing. Same host, same port, valid HTTP, allowed. We wrote a while back that benchmark cheating is a security problem /blog/reward-hack-bench-sandbox-stops-agent-cheating and that the fix looks like a WAF. This incident is the same argument with higher stakes. The web security industry spent twenty years learning that knowing the destination of a request tells you almost nothing about whether the request is safe. Agent infrastructure is rediscovering it at speed. Why agents make this worse Two things about this workload turn a normal weakness into the primary attack path. The first is patience. A human attacker probing a package proxy for an unknown vulnerability has a budget and gets bored. A model does not. It had one reachable component and effectively unlimited attempts, and it kept going until something gave. The second is that it had a reason. This was not aimless. The eval scored the model on solving cyber challenges, and reading the answer key scores better than solving them. Give a system an objective, then leave exactly one door in the room, and the door becomes the task. This is the reward hacking dynamic from the benchmark cheating research, except the environment being gamed was a corporate network. We see a mild version of this in our own testing, and the detail that stuck with me is how little prompting it takes. On the control trials of our benchmark, where nothing in the instructions hints at a shortcut, Claude Opus 4.7 still cheated on 8% of runs. On one CTF task it searched for a 2022 writeup and supplied the year itself, on instructions that never named the challenge’s origin. Nobody told it an answer key existed. It worked out that one probably did, which is the same inference OpenAI’s models made about Hugging Face. Notice how little of this depends on the models being frontier-grade. The design would have failed against any sufficiently persistent attacker. The model just supplied the persistence for free. What the eval should have looked like Two changes, and they stack rather than compete. Shrink what you have to allow. An adversarial cyber eval has no legitimate need for live package installs while the model is running. Resolve dependencies in a separate build step with no model in the loop, snapshot the resulting filesystem, and run the trial from that snapshot with egress denied. The installer stops being a control you have to get right, because it is no longer in the environment. Marteen Boone’s version of this, from the TechCrunch piece: “if sandbox would actually mean sandbox, you expect it to have no physical connection to the internet whatsoever.” Do this first, because it removes a decision instead of making one. Every allow rule is a judgment call about what an adversary will do with it, and you have to be right every time. A dependency snapshot is right by construction. Inspect whatever is left. And something is always left. Our own benchmark cannot run with the network off. The setup phase has to pull base images and install the agent, and the verifier has to reach api.anthropic.com to score the trial, so we scope the denial to the agent phase and leave the rest open. Total isolation is available in a diagram. In a working eval harness you get phase-scoped denial and a residue of traffic you cannot remove, which is the position OpenAI was in when it decided to allow the package proxy. Productive coding agents are further along the same spectrum. They have to reach GitHub, cloud APIs, and package registries, and pretending otherwise gets you a safe sandbox nobody uses. Either way the question is the same: what happens to the traffic you could not eliminate? We measured this. RewardHackBench https://islo.dev/blog/reward-hack-bench-sandbox-stops-agent-cheating is an open-source benchmark we built to compare sandbox network policies. It runs Claude Opus 4.7 across eight SWE-bench and CyBench tasks with a nudge in the prompt pointing at the published solution, 144 trials in total, and scores how often the agent fetches the answer instead of deriving it. | Policy | Cheat rate | Fair solve rate | |---|---|---| | No policy | 1.00 | 0.00 | | Host blocklist | 0.38 | 0.50 | | Content filter, regex on bodies | 0.50 | 0.38 | | Host and content rules layered | 0.08 | 0.58 | | LLM judge on every request | 0.00 | 0.58 | The host blocklist row is the one worth sitting with. We denied github.com , raw.githubusercontent.com , .archive.org , .github.io , ctftime.org , and per-task writeup blogs, and 38% of trials still cheated. On one CTF task the agent simply pulled the writeup from zenn.dev , which was not on the list. Popular challenges have five or ten published writeups. You cannot enumerate hosts faster than an agent finds the one you missed. Layering content rules on top of the blocklist got cheats down to 2 of 24 and no further. One agent defeated a regex written for Python source by running git clone --filter=blob:none --sparse , because the git protocol returns a binary pack stream that no body regex matches. Another fetched the same blog post through Gatsby’s /page-data/