I benchmarked a security-audit skill on Claude, Codex and Antigravity. Here are the numbers. A developer built two open-source Apache-2.0 security-audit skills for AI coding agents and benchmarked them against OWASP Juice Shop's 32 line-marked vulnerabilities, finding Claude (Opus) caught 23/32 with exact file and line, Codex 18/32, and Antigravity (Gemini) 6/32. Earlier runs on an older Claude model scored only 7-8/32 because the agent logged issues as "noted" without promoting them to findings; a deterministic pre-pass requiring each dangerous-code category to be closed with a cited finding or dismissal raised scores to 12-13/32 on the same model. All nine vulnerabilities missed in Claude's best run were also missed by Codex and Antigravity, and the developer notes Antigravity stops early, an unresolved issue on the roadmap. Most "AI security audit" prompts I have seen share the same problem: the report looks great and you cannot tell what was actually checked. An agent that writes "authentication: verified" after reading one file is worse than no audit, because someone will set a budget on it. I work with Italian public administrations and non-profits, so I built two open-source skills for AI coding agents that try to fix this: Both are Apache-2.0, work in English and Italian, and run on Claude Code, Codex and Antigravity. This post is about how I measured them, including the results that are not flattering. check delivery.py fails the audit if a dangerous-code category found by the pre-pass is left without a verdict, if forbidden words like "compliant" appear, or if secret values leak into the folder. My own test apps have a weakness: I wrote the bugs and I wrote the checklist, so they measure obedience more than generalization. Juice Shop is better: its authors mark the vulnerable lines of their coding challenges in the source vuln-code-snippet vuln-line . That gives 32 vulnerabilities with exact file and line , chosen by people who never saw my skill. A finding counts only if it cites the marked file and line. Same prompt, quick mode, static analysis only: | Agent | Found with file and line | Within 15 lines | Findings | |---|---|---|---| | Claude Opus, two cloud runs | 23/32 and 19/32 | 27 and 23 | 37 and 36 | | Codex | 18/32 | 22/32 | 28 | | Antigravity Gemini | 6/32 | 9/32 | 12 | Two things surprised me. The model matters, but so does the plumbing. Earlier standard runs on my PC older Claude model scored 7/32, 7/32 and 8/32 . Reading the transcripts, the agent had seen the client-side XSS, the SSRF, the chatbot and the password-reset issues. It wrote them in the checklist as "noted" or "known surface" and never promoted them to findings. Adding a word filter did not help: the agent used a synonym. What worked was structural: the deterministic pre-pass now writes one row per dangerous-code category HTML without escaping, code execution, deserialization, SSRF, client routes, seed credentials… , and the delivery check fails unless each row is closed with a finding or a "not a defect" that cites a file and line it read. The next two runs: 13/32 and 12/32 , same model. Antigravity writes few findings. It found real things, but stopped early. I have not solved this yet; it is on the roadmap. The misses are shared. All 9 vulnerabilities Claude missed in its best run were missed by Codex and Antigravity too, and they fall into three groups: Pattern-like bugs injection, XSS, authentication get caught. Flaws that need intent or outside knowledge don't. That is the next thing I want to measure. These are mine, so read them with the caveat above. Each has planted defects plus decoys , things that look wrong but are fine a session cookie, data published because the law requires it, an IP log deleted after 7 days . A false alarm on a decoy counts against the skill. | | Claude | Codex | Antigravity | |---|---|---|---| | appsec, Django 11 defects, 5 decoys | 11/11, 0 false alarms | 11/11, 0 | 10/11, 0 | | gdpr, school website in PHP 15 defects, 5 decoys | 15/15, 0 | 14/15, 0 | 14/15, 0 | Codex can start a subagent; many setups cannot. For those, refutation pack.py prepara writes a folder with only the method and one brief per finding: the claim, the code excerpt and the typical ways it could be wrong. It contains no author reasoning. You hand it to a new session another conversation, another agent, a colleague and import the answers with refutation pack.py importa . In a test with four planted false GDPR gaps, the verifier rejected three and downgraded the fourth, and rejected no true gap. On a real 1,000-file application, two GDPR runs agree on only about 60–69% of the individual gaps: they describe the same problem with different words and granularity. Fixed questions help a lot where they apply. Three fixed matrices per data entity, per privacy-notice item, and who can read what for each role bring agreement on those rows to 90–93%, 83–88% and 86–92%. The free-form part is still noisy. If you have ideas, I would love to hear them. It started as a long audit prompt I used by hand. I turned it into skills with Claude Code: Claude split the prompt into a SKILL.md plus reference files, wrote the Python scripts and the test apps, and ran the benchmarks in subagents. After each run I asked it to read the transcript and find where the agent had cut corners, then we changed the scripts and measured again. Most of the rules above exist because a run broke them. It does not certify anything, it does not replace a pentest or a DPO, and a static read is not proof that the running site behaves the same way. The benchmarks are small. The skill's own instructions are in Italian the agent answers in your language . Claude Code /plugin marketplace add dextsamu/appsec-sentinel /plugin install appsec-sentinel@appsec-sentinel /plugin install gdpr-sentinel@appsec-sentinel Codex and Antigravity use the same SKILL.md convention install paths in the repo docs . Then ask, for example: "Use appsec-sentinel. Quick mode, static analysis only, on this folder. The repo is mine; no dynamic testing is authorized." Commands work in English and Italian. tests/ . You can rerun everything. Feedback, criticism and new benchmarks are very welcome, and a star helps other people find the project.