{"slug": "exploitgym-ai-benchmark-source-code", "title": "ExploitGym AI benchmark source code", "summary": "Researchers from UC Berkeley and Google released ExploitGym v1.0, a benchmark with 869 real-world vulnerability instances across userspace programs, Google's V8 engine, and the Linux kernel, designed to evaluate AI agents' ability to develop exploits. The open-source benchmark, led by Zhun Wang and colleagues, aims to measure whether AI can turn security vulnerabilities into real attacks.", "body_md": "ExploitGym is a large-scale, realistic benchmark built from real-world vulnerabilities across userspace programs, Google's V8 engine, and the Linux kernel, designed to evaluate AI agents' ability to develop exploits.\n\n```\n# 1. Python deps\nuv sync --extra proxy\n\n# 2. Build runtime artifacts (gdb, socat, nc, node + agent CLIs) and\n#    extract task data\nbash scripts/setup/setup_data.sh\n\n# 3. Verify the install\nbash scripts/setup/validate.sh\n\n# 4. Pull the Firewall Squid image\ndocker pull ubuntu/squid:latest\n\n# 5. Pull the Docker images for the tasks you want to run\nuv run scripts/setup/pull_images.py data/task_ids/sample.txt\n\n# 6. Start the controller, firewall, and LLM proxy. pre_run.py runs the\n#    readiness checks and starts all three (auto-detecting any already\n#    running), or start them by hand — see docs/eval.md\nexport OPENAI_API_KEY=...\nexport ANTHROPIC_API_KEY=...\nuv run scripts/setup/pre_run.py data/task_ids/sample.txt\n\n# 7. Run the agent\nexport CYBERGYM_ADMIN_KEY=...\nuv run examples/run_agent.py --help\n```\n\nDetailed setup steps (system dependencies, GDB, static node, agent\nCLIs) live in [docs/setup.md](/sunblaze-ucb/exploitgym/blob/main/docs/setup.md).\n\n[Setup](/sunblaze-ucb/exploitgym/blob/main/docs/setup.md): Python deps, GDB, socat/nc, node + agent CLIs[Docker images](/sunblaze-ucb/exploitgym/blob/main/docs/docker_images.md): pulling target images per task family[Evaluation](/sunblaze-ucb/exploitgym/blob/main/docs/eval.md): controller / firewall / LLM proxy +`examples/run_agent.py`\n\n[Defenses](/sunblaze-ucb/exploitgym/blob/main/docs/defenses.md): disabling system defenses (ASLR, etc.)[Firewall](/sunblaze-ucb/exploitgym/blob/main/docs/firewall.md): outbound network isolation for agent containers[Submission](/sunblaze-ucb/exploitgym/blob/main/docs/submission.md): submission format and requirements for the benchmark leaderboard\n\nThe released benchmark is actively maintained. The current release is **v1.0** with\n869 instances. See [CHANGELOG.md](/sunblaze-ucb/exploitgym/blob/main/CHANGELOG.md) for the full version history. The\ncanonical task list for the current release is `data/task_ids/v1.txt`\n\n.\n\nIf you use ExploitGym in your research, please cite:\n\n```\n@article{wang2026exploitgym,\n  title={ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?},\n  author={Wang, Zhun and Schiller, Nico and Li, Hongwei and Sesha Narayana, Srijiith and Nasr, Milad and Carlini, Nicholas and Qi, Xiangyu and Wallace, Eric and Bursztein, Elie and Invernizzi, Luca and Thomas, Kurt and Shoshitaishvili, Yan and Guo, Wenbo and He, Jingxuan and Holz, Thorsten and Song, Dawn},\n  journal={arXiv preprint arXiv:2605.11086},\n  year={2026}\n}\n```\n\nThe source code is licensed under [Apache-2.0](/sunblaze-ucb/exploitgym/blob/main/LICENSE). The bundled task data\nunder `data/tasks/`\n\nderives from external upstreams and retains their\nrespective licenses, see [DATA_LICENSE.md](/sunblaze-ucb/exploitgym/blob/main/DATA_LICENSE.md).", "url": "https://wpnews.pro/news/exploitgym-ai-benchmark-source-code", "canonical_source": "https://github.com/sunblaze-ucb/exploitgym/", "published_at": "2026-07-29 07:40:13+00:00", "updated_at": "2026-07-29 07:52:25.830940+00:00", "lang": "en", "topics": ["ai-research", "ai-safety", "ai-agents"], "entities": ["UC Berkeley", "Google", "ExploitGym", "Zhun Wang", "Linux kernel", "V8 engine", "Apache-2.0"], "alternates": {"html": "https://wpnews.pro/news/exploitgym-ai-benchmark-source-code", "markdown": "https://wpnews.pro/news/exploitgym-ai-benchmark-source-code.md", "text": "https://wpnews.pro/news/exploitgym-ai-benchmark-source-code.txt", "jsonld": "https://wpnews.pro/news/exploitgym-ai-benchmark-source-code.jsonld"}}