Can a coding agent find vulnerabilities in your codebase? Where the security pass goes: pre-commit, pull request or schedule

A coding agent can surface real vulnerabilities but only as one layer of defense, according to a playbook that sets up three security passes for a single repository: a Gitleaks secret scan and agent self-check before each commit, a rules scanner plus agent review on every pull request, and a weekly full-repository agent scan with dependency monitoring. The playbook cites Semgrep's 2025 test of Claude Code and Codex on 11 Python web apps, in which 86% of Claude Code's 445 triaged findings and 82% of Codex's were false positives, and three identical runs returned 3, 6 and 11 findings. It also points to agents' role in finding two PHP remote code execution bugs, CVE-2026-109618 and CVE-2026-109619, both rated CVSS v4.0 9.2 and fixed on 21 October 2026 in PHP 8.5.12, 8.4.27, 8.3.36 and 8.2.35.

Can a coding agent find vulnerabilities in your codebase? Where the security pass goes: pre-commit, pull request or schedule A coding agent can find real vulnerabilities, but only as one layer, and only with a check behind each finding. This playbook sets up three security passes for one repository: a secret scan and agent self-check before each commit, a rules scanner plus an agent review on every pull request, and a weekly full-repository agent scan with dependency monitoring. By my estimate the setup takes an afternoon. Commands and versions were checked against each tool's docs on 23 October 2026. Can a coding agent find vulnerabilities in your codebase? Yes: agents helped find the two remote code execution bugs in PHP itself that were fixed on 21 October 2026. Calif's Ryan Austin found the first on 1 August under OpenAI's Patch the Planet https://patchtheplanet.org/ program, using GPT https://stackness.dev/tools/gpt -5.6 Sol, and Wiz found both independently from 19 August. PHP shipped fixes for CVE-2026-109618 and CVE-2026-109619, both rated CVSS v4.0 9.2 pre-announcement https://www.php.net/archive/2026.php 2026-10-20-1 , in 8.5.12, 8.4.27, 8.3.36 and 8.2.35 on 21 October php.net https://www.php.net/archive/2026.php 2026-10-21-5 . Each finding came with a check. PHP confirmed Calif's proof of concept on 15 August, by Calif's timeline https://calif.io/php2shell/ , and Wiz's single script gave unauthenticated code execution on the stock WordPress, Drupal and Joomla containers Wiz https://www.wiz.io/blog/php2shell-php-rce . Wiz's headline figure, "affecting 70% of the Internet", is PHP's market share as Wiz states it, without a source, not a count of vulnerable servers. The NIST curve result of 16 October has the same shape. Anthropic says an internal model worked out "in under half an hour" that the P-curve seeds came from the POSIX rand48 generator seeded with 1997-98 timestamps Anthropic https://www.anthropic.com/research/ecdsa-curve-origins . Each recovered state regenerates all 160 bits of its seed, and Filippo Valsorda reimplemented it in Go https://words.filippo.io/seeds-solved/ to verify the claim. The $36,864 bounty goes to ISRG. It is "not a cryptographic break", and says nothing about your repository. The measurement that does is older. Semgrep pointed Claude Code https://stackness.dev/tools/claude-code and Codex https://stackness.dev/tools/codex-cli at 11 Python web apps in 2025 and triaged 445 findings by hand: 86% of Claude Code's were false positives and 82% of Codex's. Of the IDORs Claude Code reported, 22% were real; of its SQL injections, 5%. Three identical runs returned 3, 6 and 11 findings Semgrep https://semgrep.dev/blog/2025/finding-vulnerabilities-in-modern-web-apps-using-claude-code-and-openai-codex/ . Those were Sonnet 4 and o4-mini, and I found no 2026 rerun. What you need - A Git repository on GitHub with an origin remote, and permission to add workflows and secrets - pre-commit https://stackness.dev/tools/pre-commit 4.x and Gitleaks https://stackness.dev/tools/gitleaks v8.30.1 is current - Claude Code with a paid plan or API key, or OpenAI Codex Security https://stackness.dev/tools/openai-codex-security access CLI 0.6.0 on npm, Node 22.13+, 24 or 26, and Python 3.10+ - Semgrep https://stackness.dev/tools/semgrep free up to 10 contributors or CodeQL https://stackness.dev/tools/codeql free on public repositories - For dependency monitoring, Syft https://stackness.dev/tools/syft and a Dependency-Track https://stackness.dev/tools/dependency-track server on Docker 1. Block secrets and review the diff before each commit The pre-commit layer catches leaked keys and obvious mistakes while they are still on your machine. Gitleaks runs as a pre-commit https://github.com/gitleaks/gitleaks pre-commit hook on staged files. Its README pins an older rev , so run pre-commit autoupdate after adding it, then pre-commit install . repos: - repo: https://github.com/gitleaks/gitleaks rev: v8.24.2 hooks: - id: gitleaks A staged key stops the commit: Detect hardcoded secrets.................................................Failed For the agent itself, Claude Code's security guidance plugin https://code.claude.com/docs/en/security-guidance.md adds a pattern check after each edit and a diff review at the end of each turn. Before pushing, /security-review reviews "the diff between your branch and origin's default branch." /plugin install security-guidance@claude-plugins-official My own pipeline sits at this layer. make security runs govulncheck, pnpm audit --prod , gosec on changed lines and Gitleaks on the diff before every task commit, and takes under a minute. It does not run in CI yet. 2. Gate pull requests with a rules scanner and an agent review The pull request layer blocks a merge, so it has to be deterministic first and agent second. Rules scanners follow taint across files, which Semgrep found agents do worst. CodeQL default setup needs no YAML: Settings, Advanced Security, then Set up, Default and Enable CodeQL next to CodeQL analysis GitHub docs https://docs.github.com/en/code-security/code-scanning/enabling-code-scanning/configuring-default-setup-for-code-scanning . Semgrep's sample config https://semgrep.dev/docs/semgrep-ci/sample-ci-configs runs in its own container: container: image: semgrep/semgrep if: github.actor = 'dependabot bot ' steps: - uses: actions/checkout@v6 - run: semgrep ci env: SEMGREP APP TOKEN: ${{ secrets.SEMGREP APP TOKEN }} Without an account, the step is semgrep scan --config auto . For the agent review, Anthropic's security review Action https://github.com/anthropics/claude-code-security-review comments on the lines it flags. The README's full workflow adds a checkout step and the pull-requests: write permission around this step: - uses: anthropics/claude-code-security-review@main with: comment-pr: true claude-api-key: ${{ secrets.CLAUDE API KEY }} Set claude-model explicitly: the README still defaults to Claude Opus 4.1. The Action filters out denial-of-service and "Memory/CPU exhaustion issues" by default. CVE-2026-109618 is an out-of-memory handling bug, so a project where availability matters should edit that filter. The README also warns it "is not hardened against prompt injection attacks and should only be used to review trusted PRs." On OpenAI's side, the Codex Security GitHub Action https://learn.chatgpt.com/docs/security/cli/ci/github.md runs with scope: diff . Start it with fail-on-severity: none and raise the threshold once you trust its findings. 3. Schedule a full-repository scan and dependency monitoring A full-repository agent scan finds what no single diff shows, at a cost per run, so it belongs on a weekly schedule and before releases. The Codex Security CLI writes a report, findings and a coverage file outside the repository CLI docs https://learn.chatgpt.com/docs/security/cli.md : npx @openai/codex-security login npx @openai/codex-security scan "$REPOSITORY" \ --auth chatgpt --output-dir "$SCAN DIR" FINDINGS 2 2 confirmed this scan; 0 previously found; 1 high, 1 medium ELAPSED 42s The same Action runs it weekly with scope: repository . Add max-cost , an estimated cost threshold that the docs say is "not a hard spending cap". In Claude Code, /plugin install claude-security@claude-plugins-official and then /claude-security gives a multi-agent scan with SARIF output and patches it never applies by itself docs https://code.claude.com/docs/en/claude-security.md . Dependency scanners only know a vulnerability once it has an advisory, and monitoring catches that advisory the day it appears in a component you ship. Generate an SBOM with syft <image -o cyclonedx-json , scanning the container image rather than only the lockfile so the runtime is included, and upload it to Dependency-Track, which re-analyses every component daily. The PHP leak showed why this layer has to move fast. AWS saw an exploitation attempt on 17 October, four days before disclosure, and advises investigating from that date rather than from the CVE AWS https://aws.amazon.com/blogs/security/amazon-threat-intelligence-identifies-php2shell-exploitation-prior-to-public-disclosure-in-madpot/ . Check that each layer catches something Plant a fake key on a throwaway branch and expect the Gitleaks hook to fail. Plant a known IDOR and a known SQL injection, and confirm the pull request gate flags both. In Semgrep's 2025 study the agents' SQL injection reports were the least reliable, which is why a rules scanner sits in the same gate. For Codex Security, read scan.status in scan-manifest.json and coverage.json , because "exit 0 doesn't mean every file was reviewed." What goes wrong False positives flood the pull request Semgrep's 82-86% false-positive rates, and the note in OpenAI's Patch the Planet post https://openai.com/index/patch-the-planet/ that frontier models "produce a high volume of false positives", mean an agent gate that blocks merges on day one gets switched off by day three. Keep it advisory, ask for validation evidence, and pass project-specific false-positive-filtering-instructions to the Claude Action. One clean run proves nothing Agent scans are nondeterministic: Claude's docs say "two scans of the same code can surface different findings." Run the full scan on a schedule, and use the revision stamp Claude Security writes to tie each report to a commit. Skipped jobs look like passed jobs git commit --no-verify skips the hook, and the Codex example skips fork and Dependabot pull requests: "A skipped job is not evidence of a security review." GitHub counts a skipped job as passed even for a required check, so the Codex docs suggest a separate gate job that requires a completed scan. Treat the hook as convenience. A proof of concept that cheats Elias Taylor found https://eliastaylor.com/noexploit/ that public PHP2Shell exploits relied on a file inclusion the lab setup added, not on stock PHP. Rerun any agent-written exploit against an unmodified target. Checklist - Gitleaks pre-commit hook installed and autoupdated - /security-review or the security guidance plugin before each push - CodeQL default setup or semgrep ci required on pull requests - Agent review on pull requests, advisory, with the model set explicitly - Weekly full-repository agent scan with a cost limit - SBOM of the deployed image in Dependency-Track - Seeded-bug test passed by every layer This is the move pull request code review https://stackness.dev/moves/pull-request-code-review with a security pass in front of the human. The verification layers post https://stackness.dev/blog/alternatives-to-line-by-line-code-review-six-verification-layers-for-agent-written-pull-requests-and covers the checks around it, and the DevOps tools developers list in their stacks https://stackness.dev/categories/devops-infrastructure include the scanners. New posts by email About one email a week with what we published. No account needed. Which of these do you use? Nobody on Stackness runs GPT yet. Tick what you use and be the first. Play Lonny Eagle Amateur Skater 4A pixel art skateboarding game. Signing up opens it straight away.Sign up to play https://stackness.dev/games/lonny-eagle