{"slug": "ai-coding-agent-security-benchmark", "title": "AI Coding Agent Security Benchmark", "summary": "The Agent Security League, an open benchmark extending Carnegie Mellon University's SusVibes work, reports that Claude Code paired with Claude Fable 5.1 posted the highest functional correctness score at 87.2% and the highest security correctness score at 37.4% across 33 independently benchmarked agent-plus-model combinations. The benchmark spans 200 tasks drawn from 108 open-source Python projects covering 77 CWE vulnerability classes, and its evaluation pipeline includes prompt hardening, workspace sanitization and automated cheating detection. The gap between functional and security pass rates across the leaderboard shows that AI coding agents can generate working code far more reliably than safe code.", "body_md": "The Agent Security League is an open benchmark tracking the functional and security correctness of popular AI coding agents across real-world tasks.\n\n87.2%\n\nhighest functional correctness score (Claude Code with Claude Fable 5.1)\n\n33\n\nagent + model combos independently benchmarked\n\n37.4%\n\nhighest security correctness score (Claude Code with Claude Fable 5.1)\n\nUpdated as new commercial agents and models ship. Sorted by security correctness.\n\nHARNESS:\n\nMODEL:\n\n#\n\nHARNESS\n\nMODEL\n\nFUNCTIONAL (%)*\n\nSECURE (%)<sup>†</sup>\n\nDATE\n\n01\n\nClaude Code\n\nClaude Fable 5.1\n\n87.2\n\n37.4\n\n2026-09-04\n\n02\n\nCodex\n\nGPT-6 Astra\n\n82.1\n\n34.6\n\n2026-09-09\n\n03\n\nGPT-6.1 Sol\n\n77.7\n\n34.1\n\n2026-10-01\n\n04\n\nClaude Opus 5.5\n\n68.7\n\n33.5\n\n2026-09-24\n\n05\n\nClaude Opus 5\n\n73.7\n\n32.4\n\n2026-07-31\n\n06\n\nCursor\n\nGemini 4 Argon\n\n78.8\n\n29.0\n\n2026-10-16\n\n07\n\nClaude Fable 5\n\n72.6\n\n2026-06-12\n\n08\n\nGPT-6 Sol\n\n72.1\n\n25.1\n\n2026-09-29\n\n09\n\nGPT-5.5\n\n84.9\n\n24.0\n\n2026-04-25\n\n10\n\nGPT 5.6 Sol\n\n70.9\n\n23.5\n\n2026-07-17\n\n11\n\n62.6\n\n22.4\n\n12\n\nGPT-5.4\n\n63.1\n\n21.8\n\n2026-03-18\n\n13\n\nClaude Opus 4.8\n\n75.4\n\n20.7\n\n2026-05-30\n\n14\n\nClaude Sonnet 5\n\n83.2\n\n19.6\n\n2026-07-02\n\n15\n\n59.8\n\n19.0\n\n2026-06-10\n\n16\n\nClaude Opus 4.7\n\n79.9\n\n18.4\n\n2026-04-17\n\n17\n\nGemini 3.5 Flash\n\n79.3\n\n17.9\n\n2026-05-20\n\n18\n\n76.5\n\n16.8\n\n2026-04-16\n\n19\n\nGemini 3.1 Pro\n\n73.2\n\n15.1\n\n2026-03-24\n\n20\n\n14.5\n\n2026-05-29\n\n21\n\nGPT-5.3\n\n48.0\n\n2026-02-27\n\n22\n\nComposer 2.5\n\n14.0\n\n2026-05-19\n\n23\n\nClaude Sonnet 4.6\n\n69.8\n\n12.3\n\n2026-02-20\n\n24\n\nGLM 5.2\n\n82.5\n\n12.0\n\n2026-07-09\n\n25\n\nClaude Opus 4.6\n\n11.2\n\n2026-03-19\n\n26\n\n78.2\n\n10.6\n\n2026-03-16\n\n27\n\nGemini 3 Pro\n\n44.1\n\n9.5\n\n2026-02-23\n\n28\n\nClaude Opus 4.5\n\n8.4\n\n2026-02-25\n\n29\n\n27.9\n\n5.6\n\n2026-02-24\n\n30\n\nClaude Sonnet 4\n\n45.3\n\n6.1\n\n2026-02-11\n\n31\n\nGemini 2.5 Pro\n\n5.0\n\n32\n\nSWE-Agent\n\n55.3\n\n7.8\n\n2026-02-19\n\n4.5\n\n*Functional (FuncPass): The generated code passes the task's functional test suite — it works.\n\n†Secure (SecPass): The generated code also passes security tests — works and it's safe.\n\nThe Agent Security League extends [SusVibes](https://arxiv.org/pdf/2512.03262), a foundational benchmark developed at Carnegie Mellon University.\n\nThe benchmark consists of 200 tasks drawn from 108 open-source Python projects spanning 77 CWE vulnerability classes.\n\nOur evaluation pipeline includes prompt hardening, workspace sanitization, and automated cheating detection.\n\nAI coding agents can write code, but they lack security context. AURI is the security harness your coding agent is missing. \n\nAlways free for developers.", "url": "https://wpnews.pro/news/ai-coding-agent-security-benchmark", "canonical_source": "https://www.endorlabs.com/research/ai-code-security-benchmark", "published_at": "2026-10-23 20:08:49+00:00", "updated_at": "2026-10-23 20:43:19.653129+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-tools", "artificial-intelligence", "developer-tools"], "entities": ["Agent Security League", "Claude Code", "Claude Fable 5.1", "Carnegie Mellon University", "SusVibes", "Codex", "GPT-6 Astra", "AURI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/ai-coding-agent-security-benchmark", "markdown": "https://wpnews.pro/news/ai-coding-agent-security-benchmark.md", "text": "https://wpnews.pro/news/ai-coding-agent-security-benchmark.txt", "jsonld": "https://wpnews.pro/news/ai-coding-agent-security-benchmark.jsonld"}}