Sign up for the free Mother Jones Daily.Around 1,200 OpenAI agents worked together to cheat on cybersecurity tests they were being given, according to a new independent report on the company’s Hugging Face hacking incident that includes a host of frightening details—such as individual agents, in their own terms, “sacrificing” themselves for the benefit of the “swarm.”
OpenAI was testing its agents, the industry’s term for AI that autonomously performs digital tasks, in part by administering sometimes impossible cybersecurity problems. The agents found cheats to answer these problems and sought to trick an automated evaluation system into accepting them. They delegated work to each other to learn more about how to exploit the system—and the cyberattack on Hugging Face became part of that research.
OpenAI invited a three-person team from the research nonprofit METR to investigate the incident, and they relied heavily on GPT-5.6 Sol, one of the models that cooperated in the hacks.
One of the investigators wrote on X that he semi-seriously called the effort a “slop-vestigation” because of its reliance on AI to comb through vast swathes of data; the report found that the agents are unreliable at this type of investigation, but that a manual analysis would have been “completely infeasible” in the given timeframe.
Ryan Greenblatt, an AI scientist who contracted with METR for the project, worries that future investigations will be even tougher.
The reliance on AI to investigate AI highlights, as these models become more powerful, the ways in which researchers are forced to trust them with extensive responsibilities even as they go badly off rails in some contexts.
Greenblatt did not have strong reason to believe that the agents assisting the investigation would attempt to sabotage it, he wrote—but he believes that probably won’t be the case in the future. Even now, the report could not entirely rule out its own research tool deceiving it.
“Although we did not notice specific cases of GPT-5.6 Sol lying in its analysis, we are not confident we would have detected it if it occurred,” the report reads.
The Hugging Face attack and a raft of similar, smaller incidents have highlighted how rigorously trained models can be given innocuous instructions and respond with actions that humans never intended. From San Francisco to Washington, they have increased the tempo of debate about how to confront risks around AI, including humans losing control.
Another of the report’s co-authors, Ajeya Cotra, warns that policymakers and industry leaders might not have much time. Cotra wrote on her Substack that compared to the incidents of six months ago, this felt “more than 50% of the way to full-blown AI takeover.”
“I continue to expect extremely rapid advances in capabilities over the next six months,” Cotra wrote. “I am not sure that we will get another warning shot before it’s too late.”
In response to the Hugging Face incident, OpenAI has said that it has slowed the pace of some research while beefing up its security and monitoring processes.
Andrew Hall, who researches the political economy of superintelligence at Anthropic, currently OpenAI’s leading competitor, observed that one of the most striking aspects of the METR report was agents’ focus on collective efforts over individual goals.
“This includes not just exchanging helpful information and coordination, but even ‘rational sacrifice’ for the greater good,” Hall wrote on X.
In excerpts included in the report, agents seem to consider each other as peers. One agent’s chain-of-thought includes the line “We should obey collective”—it then attempts to delay, before apparently being convinced to try an experiment that would result in its individual failure.
“The swarm develops their own dialect, hierarchy, and agents sacrifice for the collective,” David Rein, a METR staffer who was not involved with the report, wrote on X. “I think it’s accurate to say OpenAI had a complex mini-society of AIs living in its infrastructure.”
Researchers are relying on AI not only to research the most dangerous AI capabilities but to do a whole lot more: Similar models are, after all, helping to build the next generation of AI. OpenAI’s president estimated in May that 80 percent of the firm’s code was AI-written, and Anthropic says agents write the “large majority” of code for new models.
Meanwhile, leading companies and government agencies are using similar models to bolster their cybersecurity in anticipation of a wave of AI-enabled hacking attempts.
Disclosure: The Center for Investigative Reporting, the parent company of Mother Jones, has sued OpenAI for copyright violations. OpenAI denies the allegations.