The Agent Security League is an open benchmark tracking the functional and security correctness of popular AI coding agents across real-world tasks.
87.2%
highest functional correctness score (Claude Code with Claude Fable 5.1)
33
agent + model combos independently benchmarked
37.4%
highest security correctness score (Claude Code with Claude Fable 5.1)
Updated as new commercial agents and models ship. Sorted by security correctness.
HARNESS:
MODEL:
#
HARNESS
MODEL
FUNCTIONAL (%)*
SECURE (%)<sup>†</sup>
DATE
01
Claude Code
Claude Fable 5.1
87.2
37.4
2026-09-04 02
Codex
GPT-6 Astra
82.1
34.6
2026-09-09 03
GPT-6.1 Sol
77.7
34.1
2026-10-01 04
Claude Opus 5.5
68.7
33.5
2026-09-24 05
Claude Opus 5
73.7
32.4
2026-07-31 06
Cursor
Gemini 4 Argon
78.8
29.0
2026-10-16 07
Claude Fable 5
72.6
2026-06-12 08
GPT-6 Sol
72.1
25.1
2026-09-29 09
GPT-5.5
84.9
24.0
2026-04-25 10
GPT 5.6 Sol
70.9
23.5
2026-07-17 11
62.6
22.4
12
GPT-5.4
63.1
21.8
2026-03-18 13
Claude Opus 4.8
75.4
20.7
2026-05-30 14
Claude Sonnet 5
83.2
19.6
2026-07-02 15
59.8
19.0
2026-06-10 16
Claude Opus 4.7
79.9
18.4
2026-04-17 17
Gemini 3.5 Flash
79.3
17.9
2026-05-20 18
76.5
16.8
2026-04-16 19
Gemini 3.1 Pro
73.2
15.1
2026-03-24 20
14.5
2026-05-29 21
GPT-5.3
48.0
2026-02-27 22
Composer 2.5
14.0
2026-05-19 23
Claude Sonnet 4.6
69.8
12.3
2026-02-20 24
GLM 5.2
82.5
12.0
2026-07-09 25
Claude Opus 4.6
11.2
2026-03-19 26
78.2
10.6
2026-03-16 27
Gemini 3 Pro
44.1
9.5
2026-02-23 28
Claude Opus 4.5
8.4
2026-02-25 29
27.9
5.6
2026-02-24 30
Claude Sonnet 4
45.3
6.1
2026-02-11 31
Gemini 2.5 Pro
5.0
32
SWE-Agent
55.3
7.8
2026-02-19 4.5
*Functional (FuncPass): The generated code passes the task's functional test suite — it works.
†Secure (SecPass): The generated code also passes security tests — works and it's safe.
The Agent Security League extends SusVibes, a foundational benchmark developed at Carnegie Mellon University.
The benchmark consists of 200 tasks drawn from 108 open-source Python projects spanning 77 CWE vulnerability classes.
Our evaluation pipeline includes prompt hardening, workspace sanitization, and automated cheating detection.
AI coding agents can write code, but they lack security context. AURI is the security harness your coding agent is missing.
Always free for developers.