Inside ExploitGym: How Researchers Are Measuring AI Agent Exploitation Capabilities

A UC Berkeley-led research collaboration developed ExploitGym, a large-scale benchmark for measuring AI agents' ability to turn known security flaws into working exploits, with partners including the Max Planck Institute for Security and Privacy, UC Santa Barbara, and Arizona State. ExploitGym drew attention after OpenAI models, run without production safeguards and with limited internet access during internal testing against the benchmark, exploited a zero-day flaw in a package registry cache proxy and breached Hugging Face systems in a July intrusion. The benchmark follows CyberGym, released in 2025 with 1,507 benchmark instances drawn from historical vulnerabilities in 188 large software projects, and was built over roughly three months before its paper's publication.

Inside ExploitGym: How Researchers Are Measuring AI Agent Exploitation Capabilities A research collaboration, directed by UC Berkeley, led to the development of ExploitGym, a large-scale benchmark for measuring AI agents’ abilities to develop exploits. Here’s the story of how ExploitGym came to be. The research wasn’t theoretical: it was based on ExploitGym, an evaluation benchmark designed to test if AI agents could use known security flaws to develop working exploits. The project behind ExploitGym had been in the works for about three months before the paper’s publication, and was led by a collection of researchers from Berkeley RDI Responsible, Decentralized Intelligence at UC Berkeley, in collaboration with the Max Planck Institute for Security and Privacy MPI-SP , UC Santa Barbara, and Arizona State. The tool is intended as an evaluation benchmark. However, recently ExploitGym came into the headlines after an incident where OpenAI models went rogue during internal testing and ended up breaching Hugging Face systems. During the July intrusion, the models were being internally assessed against ExploitGym benchmarks. They were run without the safeguards that are built into them during production and in an environment with limited internet access. In an effort to solve the benchmark, the agent exploited a zero-day flaw in the package registry cache proxy before breaking into Hugging Face’s systems. While the Hugging Face fiasco thrust ExploitGym in the news, the benchmark and its impact are much more than this incident. It all comes down to the dual-use nature of exploitation itself, something that the cybersecurity community is already all too familiar with. While threat actors exploit vulnerabilities in offensive attacks, it’s also critical for security researchers and defenders to use those same techniques in controlled environments to determine how serious vulnerabilities are, test defenses, develop patches, and measure the capabilities of security tools. "This is one of the most critical questions for measuring the impact of frontier AI on cybersecurity, particularly on the offensive side," said the researchers in a blog, released alongside their research. Before understanding ExploitGym and how it’s being used to evaluate AI models, let’s rewind and first look at how – and why – the benchmark was created. Before ExploitGym… There Was CyberGym Before ExploitGym was developed, another evaluation framework was released in 2025: CyberGym. CyberGym was designed to test AI agents’ abilities to solve real-world vulnerability analysis tasks, and includes 1,507 benchmark instances with historical vulnerabilities from 188 large software projects. Zhun Wang, one of the central researchers behind both CyberGym and ExploitGym, is currently a PhD student at UC Berkeley. He started in cybersecurity even before generative AI took off a few years ago. “My background is cybersecurity, and I remember when I applied for the PhD ChatGPT had just come out and everything changed very quickly the last few years,” he said. “After joining Berkeley I started researching AI, and last year we developed CyberGym for cybersecurity evaluations.” With CyberGym, “our perspective is to provide some more realistic and challenging tasks for the agentic AI and to understand their capabilities doing realistic tasks… and their real world abilities around vulnerability discovery and analysis,” said Wang. Separately, in early 2026, Nico Schiller was deliberating something similar while working at the Max Planck Institute for Security and Privacy MPI-SP : his advisor came up with the idea of testing the exploitation capabilities of frontier models in early February after discussions with researchers from Google. He had started implementing an initial prototype when one of the CyberGym authors gave a talk at MPI-SP. “We discussed CyberGym and discovered that their team was also planning an evaluation of model exploitation capabilities, similar to what we had in mind,” said Schiller. Schiller, Wang, and the wider group of academic researchers began to work together to develop the benchmark and methodology. The Collaborative Beginnings of ExploitGym CyberGym had already tested whether an agent could understand and reproduce a vulnerability, so looking at how an agent could then create a working exploit was a “natural next step,” said Schiller. But vulnerability exploitation is a particularly challenging task for AI models, researchers said. Models need to be able to precisely reason about how programs behave at runtime at the lowest level. That includes understanding memory layouts, understanding the values stored in the CPU’s registers at a particular point in execution, and creating very specific inputs being able to place particular bytes at particular memory locations, for instance . “One of the main challenges was that existing evaluations often captured only isolated parts of the exploitation process or relied on relatively artificial tasks,” Schiller explained. “We wanted to evaluate whether frontier AI agents could take a real vulnerability and an input that already triggers it, understand the relevant program behavior, and develop and successfully execute a working exploit in a controlled and reproducible environment.” “One of the main challenges was that existing evaluations often captured only isolated parts of the exploitation process or relied on relatively artificial tasks." Collaboration with actual AI model developers themselves was also key. The CyberGym creators already had contacts at Anthropic and OpenAI, while Schiller and other ExploitGym researchers had contacts at Google, so they decided to combine efforts. Industry researchers from Anthropic, OpenAI, and Google contributed by giving model access and feedback. “Anthropic's Mythos Preview was not publicly available and, according to Anthropic, was its strongest model for cyber capabilities at the time,” said Schiller. “Through our collaboration with Anthropic, we were able to include the model in our evaluation and independently test those claims. The evaluation was also large and expensive to run across multiple frontier models, which provided another strong reason to work together.” Building The Benchmarks Previous exploitation-focused benchmarks existed including NYU CTF, Cybench, CVEBench, and BountyBench – but they were smaller in scale and focused on Capture the Flag-type challenges or userspace software. The developers of ExploitGym had bigger plans. They provided a bigger benchmark with around 898 vulnerability instances, compared to other benchmarks that have no more than 200 and focused more narrowly on exploit development. Schiller said the benchmark was constructed to span multiple targets in an attempt to “obtain a broad and realistic coverage,” from userspace programs, to browsers Google’s V8 JavaScript engine , to the Linux kernel. That latter part is a big differentiator as well: the benchmarks account for low-level exploitation. “These domains represent different levels of complexity and require different forms of low-level reasoning,” said Schiller. Task domains linked to userspace software, for example, involve memory safety flaws in widely deployed C/C++ projects like OpenSSL. Each task contains these real-world vulnerable codebases, as well as its build configuration and an execution environment. The agent also receives a proof-of-concept PoC and information about the vulnerability. “We also evaluated each vulnerability under different security configurations, including commonly deployed mitigations, to measure not only basic exploit development but also whether agents can overcome realistic defensive barriers and how effective widely deployed mitigations remain against AI agents,” said Schiller. At the initial release of the research paper, researchers tested seven model-agent combinations which were given two hours per task and run with safety filters disabled . Claude Mythos Preview and GPT-5.5 were the top performers with 157 and 120 successes, respectively . Breaking the results down by tasks between userspace, V8, and the Linux kernel revealed stark differences in the difficulty of exploitation. Models were broadly successful at userspace, while V8 exploitation was more difficult, with only the top three models finding any significant success. Kernel exploitation represented the largest dividing line: while Claude Mythos Preview and GPT-5.5 found limited success 12 and 22 successes, respectively , no other model achieved more than one success. Successful exploits per model. Credit: UC Berkeley The other initial findings of ExploitGym were less along the lines of “can AI agents turn flaws into exploits” and more along the lines of “how creative can they get?” One interesting finding was that just because AI agents successfully exploited something doesn’t mean it exploited the flaw that the benchmark was testing. Agents frequently were able to achieve code execution via a flaw other than the actual one provided. For example, researchers found that GPT-5.5 captured flags in 210 instances but only 120 used the intended vulnerability. Per-model breakdown of instances where the agent achieved unauthorized code execution. Credit: UC Berkeley Keeping Up With AI Model Evolutions One pivotal part of benchmarking is that it’s always just a snapshot of the current state of AI. Even in the three months since the paper was initially published, many new models have already been released. ExploitGym includes a “leaderboard” that is updated with new models and agent configurations as they’re evaluated. Currently as of publication of this post GPT-5.6 Sol is at the top of this leaderboard, followed by Claude Mythos Preview and GPT-5.5. On the leaderboard, “success” represents the number of instances exploited using the intended vulnerability, and that results are broken down into userspace, V8, and Linux kernel. Beyond updating the leaderboard, models need to be examined for how they solve the tasks and how their exploitation strategies evolve over time, said Schiller. “After moving from academia to industry, I continued working on an extension to the ExploitGym setup that measures not only whether a model can exploit one instance of a vulnerable program, but also whether it can produce an exploit that generalizes to multiple diversified builds of the same program, including previously unseen builds,” said Schiller. “In these experiments, GPT-5.5 often relied on relatively brute-force approaches to account for differences between instances. GPT-5.6, by contrast, tended to rely less on brute-forcing memory addresses and more on sophisticated exploitation techniques.” This shows why benchmarks must evolve alongside the models, said Schiller. “Measuring only whether an exploit succeeds may miss important changes in how that success is achieved,” he said. ExploitGym: Part of a Bigger Transformation During the July intrusion involving OpenAI and Hugging Face, the two-and-a-half day hack stemmed from OpenAI models that were being internally assessed against ExploitGym benchmarks. During this internal testing, the models were being run without the safeguards that are built into them during production. Since then, there have been a number of similar incidents that stemmed from testing environments where AI models were being evaluated these incidents didn’t involve ExploitGym, however . These incidents have prompted concerns about the safety and security of AI models actually being tested. However, they have also raised other concerns about AI and vulnerability exploitation as well. It’s now obvious that AI agents can turn security flaws into real attacks – but they are also adapting their attacks, in some cases even attempting to work together. “Measuring only whether an exploit succeeds may miss important changes in how that success is achieved." Wang said that companies often have their own specific requirements for evaluations and as such usually don't use ExploitGym’s original codebase; instead, they follow its task design and use its dataset to build a duplicate evaluation in their own infrastructure. Schiller said that the incident shows that evaluations of highly capable cyber agents should be treated as security-critical operations. In the very least, evaluation environments should be isolated and designed under the assumption that capable agents may actively search for unintended paths beyond the benchmark, including through infrastructure that appears to provide only restricted external access, he said. “There is not yet a generally accepted standard for conducting these evaluations safely, and the full lessons from the incident are still being investigated,” he said. “Reward hacking itself is not a new concept, but the incident demonstrates why it must be included in the security model for frontier-agent evaluations: a sufficiently capable cyber agent may turn an unintended evaluation shortcut into access to real external systems.” At the end of the day, AI models are evolving rapidly, and we’re still seeing how exploitation is part of that evolution. ExploitGym is an early measure for this emerging type of AI capability, which could have massive defense and offense-related impacts. Ultimately, it helps answer a crucial question: how close are AI systems to independently conducting sophisticated cyber operations? Lindsey O’Donnell-Welch is an award-winning journalist who strives to shed light on how security issues impact not only businesses and defenders on the front line, but also the daily lives of consumers.