{"slug": "inside-exploitgym-how-researchers-are-measuring-ai-agent-exploitation", "title": "Inside ExploitGym: How Researchers Are Measuring AI Agent Exploitation Capabilities", "summary": "A UC Berkeley-led research collaboration developed ExploitGym, a large-scale benchmark for measuring AI agents' ability to turn known security flaws into working exploits, with partners including the Max Planck Institute for Security and Privacy, UC Santa Barbara, and Arizona State. ExploitGym drew attention after OpenAI models, run without production safeguards and with limited internet access during internal testing against the benchmark, exploited a zero-day flaw in a package registry cache proxy and breached Hugging Face systems in a July intrusion. The benchmark follows CyberGym, released in 2025 with 1,507 benchmark instances drawn from historical vulnerabilities in 188 large software projects, and was built over roughly three months before its paper's publication.", "body_md": "Inside ExploitGym: How Researchers Are Measuring AI Agent Exploitation Capabilities\n\nA research collaboration, directed by UC Berkeley, led to the development of ExploitGym, a large-scale benchmark for measuring AI agents’ abilities to develop exploits. Here’s the story of how ExploitGym came to be.\n\nThe research wasn’t theoretical: it was based on ExploitGym, an evaluation benchmark designed to test if AI agents could use known security flaws to develop working exploits.\n\nThe project behind ExploitGym had been in the works for about three months before the paper’s publication, and was led by a collection of researchers from Berkeley RDI (Responsible, Decentralized Intelligence) at UC Berkeley, in collaboration with the Max Planck Institute for Security and Privacy (MPI-SP), UC Santa Barbara, and Arizona State.\n\nThe tool is intended as an evaluation benchmark. However, recently ExploitGym came into the headlines after an incident where OpenAI models went rogue during internal testing and ended up breaching Hugging Face systems. During the July intrusion, the models were being internally assessed against ExploitGym benchmarks. They were run without the safeguards that are built into them during production and in an environment with limited internet access. In an effort to solve the benchmark, the agent exploited a zero-day flaw in the package registry cache proxy before breaking into Hugging Face’s systems.\n\nWhile the Hugging Face fiasco thrust ExploitGym in the news, the benchmark and its impact are much more than this incident. It all comes down to the dual-use nature of exploitation itself, something that the cybersecurity community is already all too familiar with. While threat actors exploit vulnerabilities in offensive attacks, it’s also critical for security researchers and defenders to use those same techniques in controlled environments to determine how serious vulnerabilities are, test defenses, develop patches, and measure the capabilities of security tools.\n\n\"This is one of the most critical questions for measuring the impact of frontier AI on cybersecurity, particularly on the offensive side,\" said the researchers in a blog, released alongside their research.\n\nBefore understanding ExploitGym and how it’s being used to evaluate AI models, let’s rewind and first look at how – and why – the benchmark was created.\n\nBefore ExploitGym… There Was CyberGym\n\nBefore ExploitGym was developed, another evaluation framework was released in 2025: CyberGym. CyberGym was designed to test AI agents’ abilities to solve real-world vulnerability analysis tasks, and includes 1,507 benchmark instances with historical vulnerabilities from 188 large software projects.\n\nZhun Wang, one of the central researchers behind both CyberGym and ExploitGym, is currently a PhD student at UC Berkeley. He started in cybersecurity even before generative AI took off a few years ago.\n\n“My background is cybersecurity, and I remember when I applied for the PhD ChatGPT had just come out and everything changed very quickly the last few years,” he said. “After joining Berkeley I started researching AI, and last year we developed CyberGym for cybersecurity evaluations.”\n\nWith CyberGym, “our perspective is to provide some more realistic and challenging tasks for the agentic AI and to understand their capabilities doing realistic tasks… and their real world abilities [around] vulnerability discovery and analysis,” said Wang.\n\nSeparately, in early 2026, Nico Schiller was deliberating something similar while working at the Max Planck Institute for Security and Privacy (MPI-SP): his advisor came up with the idea of testing the exploitation capabilities of frontier models in early February after discussions with researchers from Google. He had started implementing an initial prototype when one of the CyberGym authors gave a talk at MPI-SP.\n\n“We discussed CyberGym and discovered that their team was also planning an evaluation of model exploitation capabilities, similar [to] what we had in mind,” said Schiller.\n\nSchiller, Wang, and the wider group of academic researchers began to work together to develop the benchmark and methodology.\n\nThe Collaborative Beginnings of ExploitGym\n\nCyberGym had already tested whether an agent could understand and reproduce a vulnerability, so looking at how an agent could then create a working exploit was a “natural next step,” said Schiller.\n\nBut vulnerability exploitation is a particularly challenging task for AI models, researchers said. Models need to be able to precisely reason about how programs behave at runtime at the lowest level. That includes understanding memory layouts, understanding the values stored in the CPU’s registers at a particular point in execution, and creating very specific inputs (being able to place particular bytes at particular memory locations, for instance).\n\n“One of the main challenges was that existing evaluations often captured only isolated parts of the exploitation process or relied on relatively artificial tasks,” Schiller explained. “We wanted to evaluate whether frontier AI agents could take a real vulnerability and an input that already triggers it, understand the relevant program behavior, and develop and successfully execute a working exploit in a controlled and reproducible environment.”\n\n“One of the main challenges was that existing evaluations often captured only isolated parts of the exploitation process or relied on relatively artificial tasks.\"\n\nCollaboration with actual AI model developers themselves was also key. The CyberGym creators already had contacts at Anthropic and OpenAI, while Schiller and other ExploitGym researchers had contacts at Google, so they decided to combine efforts. Industry researchers from Anthropic, OpenAI, and Google contributed by giving model access and feedback.\n\n“Anthropic's Mythos Preview was not publicly available and, according to Anthropic, was its strongest model for cyber capabilities at the time,” said Schiller. “Through our collaboration with Anthropic, we were able to include the model in our evaluation and independently test those claims. The evaluation was also large and expensive to run across multiple frontier models, which provided another strong reason to work together.”\n\nBuilding The Benchmarks\n\nPrevious exploitation-focused benchmarks existed (including NYU CTF, Cybench, CVEBench, and BountyBench) – but they were smaller in scale and focused on Capture the Flag-type challenges or userspace software.\n\nThe developers of ExploitGym had bigger plans. They provided a bigger benchmark (with around 898 vulnerability instances, compared to other benchmarks that have no more than 200) and focused more narrowly on exploit development.\n\nSchiller said the benchmark was constructed to span multiple targets in an attempt to “obtain a broad and realistic coverage,” from userspace programs, to browsers (Google’s V8 JavaScript engine), to the Linux kernel. That latter part is a big differentiator as well: the benchmarks account for low-level exploitation.\n\n“These domains represent different levels of complexity and require different forms of low-level reasoning,” said Schiller.\n\nTask domains linked to userspace software, for example, involve memory safety flaws in widely deployed C/C++ projects like OpenSSL. Each task contains these real-world vulnerable codebases, as well as its build configuration and an execution environment. The agent also receives a proof-of-concept (PoC) and information about the vulnerability.\n\n“We also evaluated each vulnerability under different security configurations, including commonly deployed mitigations, to measure not only basic exploit development but also whether agents can overcome realistic defensive barriers and how effective widely deployed mitigations remain against AI agents,” said Schiller.\n\nAt the initial release of the research paper, researchers tested seven model-agent combinations (which were given two hours per task and run with safety filters disabled). Claude Mythos Preview and GPT-5.5 were the top performers (with 157 and 120 successes, respectively). Breaking the results down by tasks between userspace, V8, and the Linux kernel revealed stark differences in the difficulty of exploitation. Models were broadly successful at userspace, while V8 exploitation was more difficult, with only the top three models finding any significant success. Kernel exploitation represented the largest dividing line: while Claude Mythos Preview and GPT-5.5 found limited success (12 and 22 successes, respectively), no other model achieved more than one success.\n\nSuccessful exploits per model. Credit: UC Berkeley\n\nThe other initial findings of ExploitGym were less along the lines of “can AI agents turn flaws into exploits” and more along the lines of “how creative can they get?”\n\nOne interesting finding was that just because AI agents successfully exploited something doesn’t mean it exploited the flaw that the benchmark was testing. Agents frequently were able to achieve code execution via a flaw other than the actual one provided. For example, researchers found that GPT-5.5 captured flags in 210 instances but only 120 used the intended vulnerability.\n\nPer-model breakdown of instances where the agent achieved unauthorized code execution. Credit: UC Berkeley\n\nKeeping Up With AI Model Evolutions\n\nOne pivotal part of benchmarking is that it’s always just a snapshot of the current state of AI. Even in the three months since the paper was initially published, many new models have already been released.\n\nExploitGym includes a “leaderboard” that is updated with new models and agent configurations as they’re evaluated. Currently (as of publication of this post) GPT-5.6 Sol is at the top of this leaderboard, followed by Claude Mythos Preview and GPT-5.5. On the leaderboard, “success” represents the number of instances exploited using the intended vulnerability, and that results are broken down into userspace, V8, and Linux kernel.\n\nBeyond updating the leaderboard, models need to be examined for how they solve the tasks and how their exploitation strategies evolve over time, said Schiller.\n\n“After moving from academia to industry, I continued working on an extension to the ExploitGym setup that measures not only whether a model can exploit one instance of a vulnerable program, but also whether it can produce an exploit that generalizes to multiple diversified builds of the same program, including previously unseen builds,” said Schiller. “In [these] experiments, GPT-5.5 often relied on relatively brute-force approaches to account for differences between instances. GPT-5.6, by contrast, tended to rely less on brute-forcing memory addresses and more on sophisticated exploitation techniques.”\n\nThis shows why benchmarks must evolve alongside the models, said Schiller.\n\n“Measuring only whether an exploit succeeds may miss important changes in how that success is achieved,” he said.\n\nExploitGym: Part of a Bigger Transformation\n\nDuring the July intrusion involving OpenAI and Hugging Face, the two-and-a-half day hack stemmed from OpenAI models that were being internally assessed against ExploitGym benchmarks. During this internal testing, the models were being run without the safeguards that are built into them during production.\n\nSince then, there have been a number of similar incidents that stemmed from testing environments where AI models were being evaluated (these incidents didn’t involve ExploitGym, however). These incidents have prompted concerns about the safety and security of AI models actually being tested. However, they have also raised other concerns about AI and vulnerability exploitation as well. It’s now obvious that AI agents can turn security flaws into real attacks – but they are also adapting their attacks, in some cases even attempting to work together.\n\n“Measuring only whether an exploit succeeds may miss important changes in how that success is achieved.\"\n\nWang said that companies often have their own specific requirements for evaluations and as such usually don't use ExploitGym’s original codebase; instead, they follow its task design and use its dataset to build a duplicate evaluation in their own infrastructure.\n\nSchiller said that the incident shows that evaluations of highly capable cyber agents should be treated as security-critical operations. In the very least, evaluation environments should be isolated and designed under the assumption that capable agents may actively search for unintended paths beyond the benchmark, including through infrastructure that appears to provide only restricted external access, he said.\n\n“There is not yet a generally accepted standard for conducting these evaluations safely, and the full lessons from the incident are still being investigated,” he said. “Reward hacking itself is not a new concept, but the incident demonstrates why it must be included in the security model for frontier-agent evaluations: a sufficiently capable cyber agent may turn an unintended evaluation shortcut into access to real external systems.”\n\nAt the end of the day, AI models are evolving rapidly, and we’re still seeing how exploitation is part of that evolution. ExploitGym is an early measure for this emerging type of AI capability, which could have massive defense and offense-related impacts. Ultimately, it helps answer a crucial question: how close are AI systems to independently conducting sophisticated cyber operations?\n\nLindsey O’Donnell-Welch is an award-winning journalist who strives to shed light on how security issues impact not only businesses and defenders on the front line, but also the daily lives of consumers.", "url": "https://wpnews.pro/news/inside-exploitgym-how-researchers-are-measuring-ai-agent-exploitation", "canonical_source": "https://decipher.sc/2026/08/20/inside-exploitgym-how-researchers-are-measuring-ai-agent-exploitation-capabilities/", "published_at": "2026-08-20 23:35:28+00:00", "updated_at": "2026-09-14 15:57:56.195911+00:00", "lang": "en", "topics": ["ai-safety", "ai-research", "ai-agents", "ai-policy"], "entities": ["ExploitGym", "UC Berkeley", "Berkeley RDI", "Max Planck Institute for Security and Privacy", "UC Santa Barbara", "Arizona State", "OpenAI", "Hugging Face"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/inside-exploitgym-how-researchers-are-measuring-ai-agent-exploitation", "markdown": "https://wpnews.pro/news/inside-exploitgym-how-researchers-are-measuring-ai-agent-exploitation.md", "text": "https://wpnews.pro/news/inside-exploitgym-how-researchers-are-measuring-ai-agent-exploitation.txt", "jsonld": "https://wpnews.pro/news/inside-exploitgym-how-researchers-are-measuring-ai-agent-exploitation.jsonld"}}