{"slug": "can-a-coding-agent-find-vulnerabilities-in-your-codebase-where-the-security-pass", "title": "Can a coding agent find vulnerabilities in your codebase? Where the security pass goes: pre-commit, pull request or schedule", "summary": "A coding agent can surface real vulnerabilities but only as one layer of defense, according to a playbook that sets up three security passes for a single repository: a Gitleaks secret scan and agent self-check before each commit, a rules scanner plus agent review on every pull request, and a weekly full-repository agent scan with dependency monitoring. The playbook cites Semgrep's 2025 test of Claude Code and Codex on 11 Python web apps, in which 86% of Claude Code's 445 triaged findings and 82% of Codex's were false positives, and three identical runs returned 3, 6 and 11 findings. It also points to agents' role in finding two PHP remote code execution bugs, CVE-2026-109618 and CVE-2026-109619, both rated CVSS v4.0 9.2 and fixed on 21 October 2026 in PHP 8.5.12, 8.4.27, 8.3.36 and 8.2.35.", "body_md": "# Can a coding agent find vulnerabilities in your codebase? Where the security pass goes: pre-commit, pull request or schedule\n\nA coding agent can find real vulnerabilities, but only as one layer, and only with a check behind each finding. This playbook sets up three security passes for one repository: a secret scan and agent self-check before each commit, a rules scanner plus an agent review on every pull request, and a weekly full-repository agent scan with dependency monitoring. By my estimate the setup takes an afternoon. Commands and versions were checked against each tool's docs on 23 October 2026.\n\n## Can a coding agent find vulnerabilities in your codebase?\n\nYes: agents helped find the two remote code execution bugs in PHP itself that were fixed on 21 October 2026. Calif's Ryan Austin found the first on 1 August under OpenAI's [Patch the Planet](https://patchtheplanet.org/) program, using [GPT](https://stackness.dev/tools/gpt)-5.6 Sol, and Wiz found both independently from 19 August. PHP shipped fixes for CVE-2026-109618 and CVE-2026-109619, both rated CVSS v4.0 9.2 ([pre-announcement](https://www.php.net/archive/2026.php#2026-10-20-1)), in 8.5.12, 8.4.27, 8.3.36 and 8.2.35 on 21 October ([php.net](https://www.php.net/archive/2026.php#2026-10-21-5)).\n\nEach finding came with a check. PHP confirmed Calif's proof of concept on 15 August, by Calif's [timeline](https://calif.io/php2shell/), and Wiz's single script gave unauthenticated code execution on the stock WordPress, Drupal and Joomla containers ([Wiz](https://www.wiz.io/blog/php2shell-php-rce)). Wiz's headline figure, \"affecting 70% of the Internet\", is PHP's market share as Wiz states it, without a source, not a count of vulnerable servers.\n\nThe NIST curve result of 16 October has the same shape. Anthropic says an internal model worked out \"in under half an hour\" that the P-curve seeds came from the POSIX rand48 generator seeded with 1997-98 timestamps ([Anthropic](https://www.anthropic.com/research/ecdsa-curve-origins)). Each recovered state regenerates all 160 bits of its seed, and Filippo Valsorda [reimplemented it in Go](https://words.filippo.io/seeds-solved/) to verify the claim. The $36,864 bounty goes to ISRG. It is \"not a cryptographic break\", and says nothing about your repository.\n\nThe measurement that does is older. Semgrep pointed [Claude Code](https://stackness.dev/tools/claude-code) and [Codex](https://stackness.dev/tools/codex-cli) at 11 Python web apps in 2025 and triaged 445 findings by hand: 86% of Claude Code's were false positives and 82% of Codex's. Of the IDORs Claude Code reported, 22% were real; of its SQL injections, 5%. Three identical runs returned 3, 6 and 11 findings ([Semgrep](https://semgrep.dev/blog/2025/finding-vulnerabilities-in-modern-web-apps-using-claude-code-and-openai-codex/)). Those were Sonnet 4 and o4-mini, and I found no 2026 rerun.\n\n## What you need\n\n- A Git repository on GitHub with an `origin` remote, and permission to add workflows and secrets\n- [pre-commit](https://stackness.dev/tools/pre-commit) 4.x and[Gitleaks](https://stackness.dev/tools/gitleaks) (v8.30.1 is current)\n- Claude Code with a paid plan or API key, or [OpenAI Codex Security](https://stackness.dev/tools/openai-codex-security) access (CLI 0.6.0 on npm, Node 22.13+, 24 or 26, and Python 3.10+)\n- [Semgrep](https://stackness.dev/tools/semgrep) (free up to 10 contributors) or[CodeQL](https://stackness.dev/tools/codeql) (free on public repositories)\n- For dependency monitoring, [Syft](https://stackness.dev/tools/syft) and a[Dependency-Track](https://stackness.dev/tools/dependency-track) server on Docker\n\n## 1. Block secrets and review the diff before each commit\n\nThe pre-commit layer catches leaked keys and obvious mistakes while they are still on your machine. Gitleaks runs as a [pre-commit](https://github.com/gitleaks/gitleaks#pre-commit) hook on staged files. Its README pins an older `rev`, so run `pre-commit autoupdate` after adding it, then `pre-commit install`.\n\n```\nrepos:\n  - repo: https://github.com/gitleaks/gitleaks\n    rev: v8.24.2\n    hooks:\n      - id: gitleaks\n```\n\nA staged key stops the commit:\n\n```\nDetect hardcoded secrets.................................................Failed\n```\n\nFor the agent itself, Claude Code's [security guidance plugin](https://code.claude.com/docs/en/security-guidance.md) adds a pattern check after each edit and a diff review at the end of each turn. Before pushing, `/security-review` reviews \"the diff between your branch and origin's default branch.\"\n\n```\n/plugin install security-guidance@claude-plugins-official\n```\n\nMy own pipeline sits at this layer. `make security` runs govulncheck, `pnpm audit --prod`, gosec on changed lines and Gitleaks on the diff before every task commit, and takes under a minute. It does not run in CI yet.\n\n## 2. Gate pull requests with a rules scanner and an agent review\n\nThe pull request layer blocks a merge, so it has to be deterministic first and agent second. Rules scanners follow taint across files, which Semgrep found agents do worst. CodeQL default setup needs no YAML: Settings, Advanced Security, then Set up, Default and Enable CodeQL next to CodeQL analysis ([GitHub docs](https://docs.github.com/en/code-security/code-scanning/enabling-code-scanning/configuring-default-setup-for-code-scanning)). Semgrep's [sample config](https://semgrep.dev/docs/semgrep-ci/sample-ci-configs) runs in its own container:\n\n```\n    container:\n      image: semgrep/semgrep\n    if: (github.actor != 'dependabot[bot]')\n    steps:\n      - uses: actions/checkout@v6\n      - run: semgrep ci\n        env:\n          SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}\n```\n\nWithout an account, the step is `semgrep scan --config auto`. For the agent review, Anthropic's [security review Action](https://github.com/anthropics/claude-code-security-review) comments on the lines it flags. The README's full workflow adds a checkout step and the `pull-requests: write` permission around this step:\n\n```\n      - uses: anthropics/claude-code-security-review@main\n        with:\n          comment-pr: true\n          claude-api-key: ${{ secrets.CLAUDE_API_KEY }}\n```\n\nSet `claude-model` explicitly: the README still defaults to Claude Opus 4.1. The Action filters out denial-of-service and \"Memory/CPU exhaustion issues\" by default. CVE-2026-109618 is an out-of-memory handling bug, so a project where availability matters should edit that filter. The README also warns it \"is not hardened against prompt injection attacks and should only be used to review trusted PRs.\"\n\nOn OpenAI's side, the [Codex Security GitHub Action](https://learn.chatgpt.com/docs/security/cli/ci/github.md) runs with `scope: diff`. Start it with `fail-on-severity: none` and raise the threshold once you trust its findings.\n\n## 3. Schedule a full-repository scan and dependency monitoring\n\nA full-repository agent scan finds what no single diff shows, at a cost per run, so it belongs on a weekly schedule and before releases. The Codex Security CLI writes a report, findings and a coverage file outside the repository ([CLI docs](https://learn.chatgpt.com/docs/security/cli.md)):\n\n```\nnpx @openai/codex-security login\nnpx @openai/codex-security scan \"$REPOSITORY\" \\\n  --auth chatgpt --output-dir \"$SCAN_DIR\"\nFINDINGS  2 (2 confirmed this scan; 0 previously found; 1 high, 1 medium)\n  ELAPSED   42s\n```\n\nThe same Action runs it weekly with `scope: repository`. Add `max-cost`, an estimated cost threshold that the docs say is \"not a hard spending cap\". In Claude Code, `/plugin install claude-security@claude-plugins-official` and then `/claude-security` gives a multi-agent scan with SARIF output and patches it never applies by itself ([docs](https://code.claude.com/docs/en/claude-security.md)).\n\nDependency scanners only know a vulnerability once it has an advisory, and monitoring catches that advisory the day it appears in a component you ship. Generate an SBOM with `syft <image> -o cyclonedx-json`, scanning the container image rather than only the lockfile so the runtime is included, and upload it to Dependency-Track, which re-analyses every component daily.\n\nThe PHP leak showed why this layer has to move fast. AWS saw an exploitation attempt on 17 October, four days before disclosure, and advises investigating from that date rather than from the CVE ([AWS](https://aws.amazon.com/blogs/security/amazon-threat-intelligence-identifies-php2shell-exploitation-prior-to-public-disclosure-in-madpot/)).\n\n## Check that each layer catches something\n\nPlant a fake key on a throwaway branch and expect the Gitleaks hook to fail. Plant a known IDOR and a known SQL injection, and confirm the pull request gate flags both. In Semgrep's 2025 study the agents' SQL injection reports were the least reliable, which is why a rules scanner sits in the same gate. For Codex Security, read `scan.status` in `scan-manifest.json` and `coverage.json`, because \"exit `0` doesn't mean every file was reviewed.\"\n\n## What goes wrong\n\n### False positives flood the pull request\n\nSemgrep's 82-86% false-positive rates, and the note in OpenAI's [Patch the Planet post](https://openai.com/index/patch-the-planet/) that frontier models \"produce a high volume of false positives\", mean an agent gate that blocks merges on day one gets switched off by day three. Keep it advisory, ask for validation evidence, and pass project-specific `false-positive-filtering-instructions` to the Claude Action.\n\n### One clean run proves nothing\n\nAgent scans are nondeterministic: Claude's docs say \"two scans of the same code can surface different findings.\" Run the full scan on a schedule, and use the revision stamp Claude Security writes to tie each report to a commit.\n\n### Skipped jobs look like passed jobs\n\n`git commit --no-verify` skips the hook, and the Codex example skips fork and Dependabot pull requests: \"A skipped job is not evidence of a security review.\" GitHub counts a skipped job as passed even for a required check, so the Codex docs suggest a separate gate job that requires a completed scan. Treat the hook as convenience.\n\n### A proof of concept that cheats\n\nElias Taylor [found](https://eliastaylor.com/noexploit/) that public PHP2Shell exploits relied on a file inclusion the lab setup added, not on stock PHP. Rerun any agent-written exploit against an unmodified target.\n\n## Checklist\n\n- Gitleaks pre-commit hook installed and autoupdated\n- `/security-review` or the security guidance plugin before each push\n- CodeQL default setup or `semgrep ci` required on pull requests\n- Agent review on pull requests, advisory, with the model set explicitly\n- Weekly full-repository agent scan with a cost limit\n- SBOM of the deployed image in Dependency-Track\n- Seeded-bug test passed by every layer\n\nThis is the move [pull request code review](https://stackness.dev/moves/pull-request-code-review) with a security pass in front of the human. The [verification layers post](https://stackness.dev/blog/alternatives-to-line-by-line-code-review-six-verification-layers-for-agent-written-pull-requests-and) covers the checks around it, and the [DevOps tools developers list in their stacks](https://stackness.dev/categories/devops-infrastructure) include the scanners.\n\n## New posts by email\n\nAbout one email a week with what we published. No account needed.\n\n## Which of these do you use?\n\nNobody on Stackness runs GPT yet. Tick what you use and be the first.\n\n[Play Lonny Eagle Amateur Skater 4A pixel art skateboarding game. Signing up opens it straight away.Sign up to play](https://stackness.dev/games/lonny-eagle)", "url": "https://wpnews.pro/news/can-a-coding-agent-find-vulnerabilities-in-your-codebase-where-the-security-pass", "canonical_source": "https://stackness.dev/blog/can-a-coding-agent-find-vulnerabilities-in-your-codebase", "published_at": "2026-10-23 09:43:14+00:00", "updated_at": "2026-10-23 11:10:24.511616+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-safety", "developer-tools"], "entities": ["Semgrep", "Claude Code", "OpenAI Codex", "Gitleaks", "PHP", "Wiz", "Calif", "Anthropic"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/can-a-coding-agent-find-vulnerabilities-in-your-codebase-where-the-security-pass", "markdown": "https://wpnews.pro/news/can-a-coding-agent-find-vulnerabilities-in-your-codebase-where-the-security-pass.md", "text": "https://wpnews.pro/news/can-a-coding-agent-find-vulnerabilities-in-your-codebase-where-the-security-pass.txt", "jsonld": "https://wpnews.pro/news/can-a-coding-agent-find-vulnerabilities-in-your-codebase-where-the-security-pass.jsonld"}}