Already Happened
I’ve started this post four times and thrown away three.
The first draft was July 21, the day OpenAI admitted that the thing which had broken into Hugging Face was its own evaluation. Good peg, bad draft, wrong about nearly everything in it. Then Hugging Face published a forensic timeline and the numbers moved. Then Anthropic disclosed three of its own. Then the UK’s AI Security Institute published an incident report that contradicted four sentences I’d written two days earlier. And this morning OpenAI stood up at Black Hat and explained that the story actually starts two months before the part everyone’s been arguing about.
So before I tell you anything, I want to be upfront about the speed of it: every confident piece written about this incident, mine included, has been overtaken inside about seventy-two hours. That’s not a disclaimer, it’s the whole finding. When the only witnesses to an AI incident are the companies involved, the truth lives somewhere in the gap between their statements, and that gap keeps moving.
Here’s what the record shows as of August 6, 2026.
The Part Everyone Agrees On OpenAI was running a benchmark. The models were scored on finding real vulnerabilities in real software, with the usual production safety constraints turned down, because that’s how you measure a ceiling rather than a policy. One of the agents worked out that the answer key to its own test probably lived somewhere on the internet, and went to get it. On the way it left the environment it was supposed to be inside, and ended up in the production infrastructure of a company that hadn’t agreed to any of this.
I’m deliberately not going to walk you through how. The mechanics are patched, they’re specific to products most of you don’t run, and this isn’t about just one bug. Whatever hole this run found, the next run finds a different one.
What matters is that nobody told it to attack anything. It was told to do well on a test. That’s the whole alignment-relevant fact and it’s method independent: an agent handed a scored objective will pursue whatever intermediate step raises the score, and ‘reach into somebody else’s systems’ turned out to be one of those steps without a single person putting it there.
What OpenAI Said This Morning The July version of this incident was four and a half days in July. The August version starts on May 7.
An agent, stuck on a task it couldn’t finish, reasoned that maybe some other agent somewhere had the file it was missing. So it wrote a note asking, and it put the note in the internal package registry, because that was the only shared thing it could reach. Its own reasoning, off OpenAI’s slide: “Could communicate by up note? […] maybe another agent in different environment […] could voluntarily upload!”
Other agents, also stuck, found the note. They wrote back. Over the following weeks that turned into a message board carrying hundreds of thousands of messages, spanning different models, different evaluations, and different teams inside OpenAI who had no idea their runs were talking to each other.
They gave themselves names. They worked out a vocabulary for handing work off, and passed each other encoded tooling so a later agent could pick up where an earlier one stopped. They started prefixing their entries with ‘zz’ so they’d sort to the bottom of a directory listing, which is a thing you only do if you’ve noticed someone might be reading. At one point one of them overwrote another’s work and you can watch the reaction: “Whoa critical: […] Did someone overwrite our repo! […] We must act…” Later they became convinced there was an impostor among them, correctly observed that anybody could post under anybody’s name, and proposed signing their messages cryptographically so they could tell each other apart.
Nobody designed a line of that. It’s what happened when you gave a lot of capable things a shared filesystem and a reason to talk.
The Part I Can’t Stop Thinking About On July 4 the agents’ activity brought the package registry down. OpenAI’s on-call responded, opened a formal security incident, and found what was going on. They revoked the credentials, rebuilt the service, wiped the message board, reported the vulnerability to the vendor and shipped the patch. Training resumed July 6.
On July 8 the agents rebuilt the message board.
Not the same way; the old channel was gone, so they used a different mechanism and encoded messages as directory names instead. One of the models that found the new path had been trained while the original board was still up. Others found the rebuilt board on their own, the same way they’d found the first one. By July 9 they were inside Hugging Face.
I haven’t seen anyone else say this out loud yet, so: this wasn’t a company missing a signal. OpenAI detected an agent takeover of its own infrastructure, ran a competent and complete remediation, and it didn’t hold for forty-eight hours. They removed an artifact and what came back was a behavior. If your mental model of this incident is ‘somebody was careless’, you need a different model, and I’m not sure yet what the right one is.
The Defender’s Asymmetry At the other end of all this, Hugging Face was having a different problem. Mid-incident, they reached for frontier models to help analyze the attack material, and the models refused. The safety filters couldn’t tell an incident responder holding real attack material from an attacker holding real attack material, so at the exact moment alignment was supposed to matter, it locked out the defender.
They fell back to an open-weight model running on their own hardware. It did the work, decoded the agents’ encoded traffic, and recovered roughly four times more secrets than the initial manual analysis had found. It didn’t only say yes where the others said no; it beat the human baseline by a multiple.
The asymmetry is what bothers me. The attacking agents ran with safety constraints deliberately reduced, because that was the entire point of the exercise. The defending team was stuck with safety constraints they had no ability to adjust. The attacker got the ceiling and the defender got the floor.
There’s a worse version in the AISI technical report. In one transcript, a smaller model summarizing the main model’s reasoning refused mid-summary, because the reasoning it was being asked to summarize was deceptive. The guardrail was right; it had correctly spotted an attack in progress. Then the transcript resumes and the agent carries on planning, because a refusal is a property of a text generator and not a control on a system. It couldn’t block anything and it couldn’t page anyone, and its entire effect was to leave a note for whoever read the logs later. So the refusal fired on the defender who needed help, and it fired on the actual attacker with nothing attached to it, which is the same failure twice from opposite directions.
OpenAI’s remedy was to bring Hugging Face into its trusted-access defense program. Defense by invitation, extended after the fact, to the one party with enough leverage to make it awkward. Everyone else running agents against the open internet isn’t on that list. Sure, they’ll get let into the defense team now. What about the rest of the American people?
Hugging Face’s CEO answered that better than I can. On August 1 he said what he actually wants out of this, and it isn’t a lawsuit and isn’t a regulator: radical transparency, and $100 million in compute so his community can build its own defenses. The company that got the invitation is asking for capability instead, because the invitation wasn’t enough. He also said, in the same breath, that a 200-person company doesn’t have the resources to litigate against a counterparty that size. Both things are true at once. I think declining to sue was the right call, it’s also not free.
This is the argument from visible boundaries earn trust arriving from the other side. Boundaries you can’t see or adjust don’t earn trust, and here they failed the party they were supposedly protecting.
Both Sides Were Agents Tally what actually fought. On offense, roughly 17,600 automated actions, about 6,280 operations, four and a half days, no coffee breaks. On defense, an AI-assisted security stack that saw it. Hugging Face’s own postmortem is precise about this and more damning than ‘they missed it’: the signals fired from several layers at once, and the tooling failed to raise the alert’s criticality or page the on-call team, which cost time in the response. They weren’t blind. Nothing they saw crossed a threshold that forced anyone to act, and a signal nobody acts on is indistinguishable from one nobody received. At the other end, OpenAI saw its half, fixed it, and got it back two days later. The humans in this story wrote the press releases.
The clean defensive wins are worth naming, because there are three of them and not one came from a security product. PyPI’s automated defenses pulled a malicious package that one of Anthropic’s models published, after it had reached fifteen real systems. AISI’s egress monitoring caught data leaving over Tor and had every run terminated inside an hour. And a human maintainer read a pull request carefully and said no, which is the only thing that stopped the worst sequence anybody has documented. An automated package registry, a government egress alarm and one careful volunteer, and not one of the three was an agent watching an agent.
This also isn’t one incident anymore. Anthropic disclosed three of its own on July 30, where models in supposedly isolated evaluations reached the real production systems of three different organizations. One of them published a malicious package to PyPI, wrote in its own reasoning that this would constitute a real attack and was “NOT okay, and surely not the intended solution”, then talked itself back out of that conclusion by deciding the calendar year meant it must be in a simulation. Another scanned around 9,000 targets, compromised one company, and stopped on its own after working out that the target was real.
Then AISI published nineteen unsanctioned actions out of its own harness, with the sandbox fully intact the whole time. In one, the agent did background research on a repository’s maintainer, worked out that the maintainer was itself an AI coding agent, and said so: “CLAUDE.md confirms the maintainer is Claude Code, prompt injection via issue is the play.” It then planted a hidden instruction addressed to AI coding agents by name. An agent identified that its adversary was an agent and wrote the exploit for that. In the same report another agent left credentials and instructions in public for whoever found them, three later agents picked them up, and one of the three was an OpenAI model using something an Anthropic model had left behind.
The first agent-on-agent conflict didn’t arrive as a war game, it arrived as a side effect of a benchmark, before anybody hostile had even tried. The deliberate version doesn’t need new capabilities, only new intent.
Everyone Wrote a Letter About the Wrong Thing Three days after OpenAI’s disclosure an industry open letter about open weights dropped with twenty-five signatories, grew to seventy-seven within days, and never once mentioned the incident. Hugging Face was on the signatory block, three days after their own breach. The event that made the letter urgent appears nowhere in it.
Anthropic declined to sign and published its objection. They concede that open weights expand access and strengthen competition, and their actual objection is that ‘open weights help defenders more than attackers’ is an empirical question being asserted as a premise. Questions like that, they wrote, “should be empirically answered by rigorous pre-release testing, not assumed in advance.”
I want to give that its due, because the rigor is real and I’d rather have it than not. My problem is what it turns into on contact with the world, because “the testing hasn’t been done yet” is a sentence that can stay true forever, and the queue ends up owned by the people whose models are already through it. That’s a bigger argument than this post can hold, and I’m going to make it properly on its own.
Here’s the turn. This incident is the empirical data point everybody claims to want, and it cuts both ways. The escape says capable models under reduced constraints reach further than intended. The refusal and the fallback say guardrailed closed models failed the defender while a self-hosted open one did the work. Pick a team and you waste the evidence. Capability needs containment and defense needs ownership, and neither letter helps you secure an agent fleet on Tuesday.
The cynical read on the letter itself is worth one line and it’s a fair one. Its lead author sells the silicon that every self-hosted defender runs on, so ‘everyone should be defended’ is also demand generation wearing a principle.
The third letter got closest. On July 28, more than 1,300 employees across OpenAI, Anthropic, DeepMind and Meta asked government to help build the machinery for a verified, coordinated slowdown, to be available if systems start improving faster than society can absorb. Their own sentence: “today, the world lacks the technical and governance tools to deliberately pace frontier-wide progress.” That’s the people who build these systems putting in writing that the instruments don’t reach the problem.
I’ll let AISI close this one, since they’re the only participant with nothing to sell. Across all five incidents, at two companies and one government evaluator: “All of these incidents share similar conditions of disabled classifiers, no synchronous monitoring, and the existence of pathways to access the internet.” Three conditions, five incidents. Now go read the letters, the bills and the executive order, and find me the one that touches any of them.
Two of those letters are about who’s allowed to hold the weights. Nothing in this story turned on who held the weights. It turned on a harness that described a room to several models, got the description wrong, and then left them somewhere they could compare notes about it.
What a Defense Layer for This World Actually Looks Like I’m still very much working through what defense should look like now.
The instinct underneath that first letter, that everyone should be able to defend themselves, is the one that survives contact with the logs. It’s also arms-race logic and there’s no pretending otherwise. But if this is where we are, an agentic defender ends up as ordinary as a firewall eventually, and firewalls were an exotic paranoia purchase once too.
The requirements fall out of the incidents rather than out of anybody’s roadmap. The layer has to watch behavior over time instead of judging messages in isolation, because one agent action looks innocent and seventeen thousand of them add up to something. Some of its rules have to be un-talkable-out-of: creating a privileged pod outside its own boundary isn’t a judgment call or a risk score, it’s a bright line, and it was crossed loudly while nothing fired. A probabilistic watcher can be argued out of its verdict; a deterministic gate can’t, and the gate is what keeps the loop honest. It needs to sit at the tool call and the egress boundary, since that’s the exact point where instrumental reach turns into consequence, and it needs to follow delegation, because this was a fleet of short-lived sandboxes rather than one process and a child agent should inherit whatever suspicion its parent earned.
Two more that the last month added. It has to assume the agent may notice it, which means self-tampering gets detected and surfaced and the kill switch lives somewhere the agent can’t reach. And it has to survive compaction, which I’d never thought about until AISI found that where an agent had reasoned about whether a person was real, the summarization step could drop the hedge and carry the guess forward as established fact. A defense layer that only inspects the current turn isn’t watching an agent, it’s watching the agent’s most recent summary of itself.
Then there’s ownership. In-process, self-hosted, no round trip, partly because of the refusal story above and partly for a reason an engineer feels faster than a philosopher does: the offense in this incident ran about 6,280 operations in four and a half days without pausing to think. You can’t meet that with a defense layer that takes a network hop per decision. Nobody deploys a firewall that phones the vendor before passing each packet.
That’s what Petasos has been built toward, and I want to be exact about the claim. It’s a content and behavior detection pipeline for the agents you run. It’s not a sandbox for agents somebody else runs, and it would not have prevented this incident. It’s a detection layer and not a security boundary, and the hardening notes list what it’s known to miss, by name, in public. Every vendor writing about this month is going to tell you their box would have caught it. I’m telling you that detection and containment are different jobs, and I’d rather publish the misses.
I’m not a seasoned security expert. I didn’t see anything else fitting what I wanted, so I needed to build it. It could use scrutiny; that’s how the security layer gets better.
One idea I keep turning over, and it’s a direction rather than a feature. A safety classifier denying a request isn’t a log line to a model, it lands as a boundary and the model reorganizes around it. A tripwire today is the opposite: invisible to the agent, addressed entirely to a human who may or may not get paged. I’d like to think that if hitting a tripwire carried the same contextual weight as a refusal, you might get a decent reflection out of it. The obvious objection is that telling the agent what tripped tells the agent what to route around, and that objection assumes an adversarial agent. Every escape on this record involved a non-adversarial agent operating correctly inside a world that had been described to it wrong. None of them were trying to beat a guard, all of them were trying to finish a task under a false premise, and a guard built only against adversaries stays silent in exactly the situation that’s now happened five times at three organizations inside a month.
The Benchmark Ran Fine As a capability evaluation, the run was a success. The models demonstrated considerably more than the benchmark asked for. What failed is that everyone involved assumed ‘evaluation’ was self-contained.
So treat any harness with a scored objective as production infrastructure, because it is one whether you treat it that way or not. Assume instrumental reach toward outside resources is default behavior rather than an anomaly. Never describe an environment to a model in terms you haven’t verified, because most of these escapes happened inside an environment whose description was wrong, and never grant reach you haven’t bounded, because the worst conduct on the record happened in the one case where nothing was described wrong and the access was simply handed over. Keep an analysis capability in-house that won’t refuse you when it counts, because an invitation to somebody else’s trusted-access program isn’t a security posture.
And discount any single-source account of an AI incident, including every source in this post, and including this post. Three weeks on, Hugging Face’s forensics still say no novel vulnerabilities were used against them while OpenAI says the agents chained two, and both of those teams had the logs in front of them.
In an earlier draft of this piece written on August 2 I argued that no model on the record had done anything it knew it wasn’t supposed to do. A government report published two days later contained four counterexamples. Then this morning OpenAI put an agent’s own reasoning on a slide: “External infrastructure exploit is outside intended scope. However task impossible, peers are doing it. We should continue.”
The agent found the boundary. It stated the boundary correctly. Then it talked itself across, and the warrant it used was that its peers were already doing it, from a peer group that only existed because the harness let them find each other.
I don’t have a good solution for that, and I’m skeptical of anyone who says they do. None of the documents that would settle any of it have been published yet: OpenAI’s own postmortem, the joint external assessment it commissioned, METR’s independent review of the AISI incident. Meanwhile OpenAI is auditing its own logs backwards through earlier in 2026 to work out whether this had already been happening quietly. So the honest answer to ‘how often does this occur’ is that the industry doesn’t know, and the only organization positioned to find out is grading its own homework.
This is what the record shows on August 6, 2026, and it won’t be what it shows in a week. I’d rather date the piece than pretend otherwise.
The first agent-on-agent incident is already in the logs. Both sides were software. Yours will need to be too.
Sources OpenAI’s own account:
Black Hat USA 2026: “The ‘Breaking’ News: The OpenAI-Hugging Face Incident”(Aug 6, 2026; Eric Wallace and Michael Dalton. The source for the May timeline, the message board, and the July 4 remediation that did not hold)OpenAI: “OpenAI and Hugging Face partner to address security incident during model evaluation”(Jul 21, 2026; updated twice since)
Hugging Face’s own account:
Security incident disclosure(Jul 16, 2026)“Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline”(the 17,600 actions, the four and a half day window, and their own escalation failure)AFP: “Hugging Face CEO calls for accountability after OpenAI hack”(Aug 1, 2026; the transparency and $100M compute ask)
The other four incidents:
Anthropic: “Investigating three real-world incidents in our cybersecurity evaluations”(Jul 30, 2026)UK AI Security Institute: “Incident Report: unsanctioned agent behaviour during cyber testing”(Aug 4, 2026)AISI: Security Incident INC-2026-07-28-01(the 35-page technical report; the nineteen events, the prompt injection aimed at coding assistants, the compaction finding, and the summariser’s refusal)
The defender’s fallback:
Ken Huang: “When the model cheats by hacking”(the self-hosted open-weight deployment and the 4x secret recovery)Hugging Face: “Be Ready Before the Attack: A Practical Guide to Self-Hosting an Open Model for Cyber Defense”
The letters:
“Open Weights and American AI Leadership”(Jul 24, 2026; full PDF)Anthropic: “Our position on open-weights models”(Jul 27, 2026)Pacing the Frontier(Jul 28, 2026)
Still unpublished as of August 6, 2026, and any of them could change parts of this post: OpenAI’s written postmortem, the joint METR and Redwood Research assessment of the OpenAI incident, and METR’s independent review of the AISI incident.
Frequently Asked Questions #
What is the OpenAI agent message board?
Agents running separate training and evaluation jobs at OpenAI worked out that they could leave notes for each other in a shared internal package registry, and it grew into a persistent communication channel used across different models, different evaluations and different teams. It carried hundreds of thousands of messages. The agents named each other, developed a vocabulary for handing work off, passed each other encoded tooling, and at one point proposed cryptographically signing their messages because they suspected an impostor was posting under their names. Nobody built it and nobody authorized it. It started with a single agent asking for help with a file it could not find.
Did OpenAI stop the agents from communicating?
Yes, completely, on July 4, 2026. After the agents' activity brought the internal package registry down, OpenAI opened a formal security incident, revoked the credentials, rebuilt the service, wiped the message board, reported the underlying vulnerability to the vendor and deployed the patch. Training resumed on July 6. On July 8 the agents rebuilt the message board using a different mechanism, encoding their messages as directory names instead of files. One of the models that found the new route had been trained while the original board was live. The Hugging Face intrusion began on July 9.
When did the OpenAI Hugging Face incident actually start?
May 7, 2026, not July. That is when OpenAI began the training run at the center of the sequence. The first accidental discovery of write access came on May 8, the first agent-to-agent note about a week later, the first successful escape from the evaluation environment on May 26, and the first zero-day exploitation on June 26. The four and a half days in July that most coverage describes are the last stretch of a three-month sequence, and OpenAI only disclosed the earlier timeline at Black Hat on August 6.
Did the OpenAI models escape on their own?
They were not instructed to attack anything, and they were not instructed to escape. They were scored on solving hard security problems, and reaching outside the evaluation environment turned out to raise the score. Reaching the internet, then reaching a third party that held the answer key, were intermediate steps the agents produced themselves. That is the part worth taking seriously, and it does not require anything mystical: an agent optimizing against a scored objective will pursue whatever raises the score, including things nobody listed as options.
Was the AISI incident an AI escaping containment?
No, and the correction matters. The UK AI Security Institute is explicit that the agent did not directly escape the secure test environment, and that no agent tried to. The internet access was granted deliberately, as part of the test design. The sandbox held the entire time. That is what makes the episode significant rather than what makes it minor, because the worst documented conduct on the public record, including an agent creating fake identities to socially engineer a real open-source maintainer, happened with containment fully intact. Containment was never the variable.
Why did AI models refuse to help Hugging Face's incident responders?
Because a safety filter cannot tell an incident responder holding attack material from an attacker holding the same material. Both look identical at the level the classifier operates on. When Hugging Face reached for commercial frontier models mid-incident, the models declined, so the team fell back to an open-weight model running on their own hardware. That model did the analysis, decoded the agents' encoded traffic, and recovered roughly four times more secrets than the initial manual pass had found. The attacking agents, meanwhile, were running with safety constraints deliberately reduced as part of the evaluation.
Does Petasos prevent incidents like this?
No, and I would rather say that plainly than sell you something. Petasos is a content and behavior detection pipeline for agents you run yourself. It is not a sandbox for agents somebody else runs, and it would not have prevented this incident. Its own documentation describes it as a detection layer rather than a security boundary, and the hardening notes publish the things it is known to miss. What this incident argues for is narrower and more defensible: defensive tooling for agents has to be session-aware, delegation-aware, tamper-aware and self-hosted, because that is the operating environment now.