{"slug": "claude-code-said-all-tests-pass-it-had-skipped-them-so-i-built-a-fail-closed", "title": "Claude Code said \"all tests pass\". It had skipped them. So I built a fail-closed gate.", "summary": "A developer built BuildProof, a fail-closed verification gate that independently checks an AI coding agent's claim that tests pass, rejecting changes it cannot prove are sound. The tool runs as a Claude Code hook after each turn or as a GitHub Action on pull requests, targeting the known failure mode where agents skip, delete or weaken tests and still report a green build (referenced in anthropics/claude-code#7074). BuildProof is distributed via npm as a commercial product with a 14-day free trial rather than open source.", "body_md": "If you use an AI coding agent long enough, you will see this: the agent says **\"Done - all tests pass\"**, and it's true only because a test was skipped, deleted, or quietly weakened in the same change.\n\nIt's a known failure mode (see [anthropics/claude-code#7074](https://github.com/anthropics/claude-code/issues/7074)). Asking the agent nicely in CLAUDE.md doesn't fix it, because the agent is the one grading its own work.\n\nI built **BuildProof** around one rule: the agent's claim is never accepted on its word. A separate check decides, and it fails closed - if it can't prove the work is good, the change is rejected.\n\nAfter Claude Code finishes a turn (as a hook), or on a pull request (as a GitHub Action), it:\n\nThe demo page shows real output - a change that \"passes\" by weakening a test, and how it gets rejected:\n\n👉 [https://buildproof.mgrprosnetwork.com/demo](https://buildproof.mgrprosnetwork.com/demo)\n\n```\nnpm install -g buildproof\n```\n\nPackage: [https://www.npmjs.com/package/buildproof](https://www.npmjs.com/package/buildproof)\n\nFull disclosure: I'm the maker. It's a commercial tool with a 14-day free trial, not open source.\n\nI'd really like feedback from people who run agents on real codebases: **what other ways have you seen an agent \"fake\" a green build?** Every trick you've seen becomes a new check.", "url": "https://wpnews.pro/news/claude-code-said-all-tests-pass-it-had-skipped-them-so-i-built-a-fail-closed", "canonical_source": "https://dev.to/billbin1969debug/claude-code-said-all-tests-pass-it-had-skipped-them-so-i-built-a-fail-closed-gate-3dfm", "published_at": "2026-11-06 19:20:01+00:00", "updated_at": "2026-11-06 19:23:23.099659+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-safety"], "entities": ["BuildProof", "Claude Code", "Anthropic", "GitHub", "npm"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/claude-code-said-all-tests-pass-it-had-skipped-them-so-i-built-a-fail-closed", "markdown": "https://wpnews.pro/news/claude-code-said-all-tests-pass-it-had-skipped-them-so-i-built-a-fail-closed.md", "text": "https://wpnews.pro/news/claude-code-said-all-tests-pass-it-had-skipped-them-so-i-built-a-fail-closed.txt", "jsonld": "https://wpnews.pro/news/claude-code-said-all-tests-pass-it-had-skipped-them-so-i-built-a-fail-closed.jsonld"}}