{"slug": "can-your-verify-gate-actually-fail", "title": "Can your verify gate actually fail?", "summary": "Andréas, a full-stack developer and CTO, discovered that his automated verify gate, designed to lint, typecheck, build, and run tests before merging AI-generated code, was failing to catch broken code in three out of four repositories. The gate returned a green status without running any tests due to a configuration fallback that treated missing gate steps as success. He fixed the structural flaw by making empty gate configurations fail explicitly and added missing build steps to the affected repos.", "body_md": "I let Claude Code commit directly to my repositories. I don't review the diffs. I didn't think I needed to, because I built a deterministic verify gate — a script that lints, typechecks, builds, and runs tests. If the gate goes green, the PR merges automatically.\n\nI trusted that gate implicitly. Until I actually sat down and asked the one question that matters: **If an agent pushes completely broken code right now, will this gate actually go red?**\n\nNot \"is the script configured?\" Not \"does the file exist?\"\n\n*Will it actually fail?*\n\nTurns out, for three out of my four repos, the answer was an emphatic **no**. The gate was returning `GREEN`\n\nwithout running a single line of code.\n\nMy gate script is repo-agnostic. It looks up the repository name in a policy JSON file (`gates.<repoName>`\n\n) and falls back to a default if it doesn't find one.\n\nHere is the PowerShell logic driving the whole operation:\n\n``` php\n$steps = $policy.gates.$RepoName\nif ($null -eq $steps) { $steps = $policy.gates.default }\nif (-not $steps -or $steps.Count -eq 0) {\n    Write-Log \"GREEN (no gate steps configured for '$RepoName')\"\n    exit 0                       # <-- Unconditional green\n}\n```\n\nAnd here is my config file:\n\n```\n\"gates\": {\n  \"app\": [\n    \"npm run lint\",\n    \"npm run test\",\n    \"<design-gate script>\"\n  ],\n  \"default\": []\n}\n```\n\nLook at that exit code. If a repo doesn't have an explicit entry in the JSON file, it falls through to `default: []`\n\n. The script sees zero steps, prints `GREEN`\n\n, and exits with `0`\n\n.\n\nThe runner sees exit code `0`\n\nand instantly merges the code. I built a system where **having no tests configured is structurally identical to passing all tests.**\n\nWhen I audited all four repos the runner is allowed to touch, the reality was pretty grim:\n\n`app`\n\n(The Runner itself):`npm run lint`\n\nruns, but `tsconfig`\n\nexplicitly excludes the `runner/`\n\ndirectory — meaning `build`\n\nisn't in the gate at all, so client/server boundary breaks ship freely. Vitest runs 28 test files, but the glob pattern completely misses `*.test.tsx`\n\nfiles.`product`\n\n(Next.js app):`tsc --noEmit`\n\n, `vitest`\n\n(10 suites), and `next build`\n\nready to go in `package.json`\n\n. None of them were wired into the gate config. It had 0% coverage purely because of lazy config debt.`notes`\n\n(Markdown vault):`portfolio`\n\n(Vite site):The real bug here isn't missing npm scripts. The structural flaw is that **an empty gate defaults to success**.\n\nTo fix this properly:\n\n`$steps.Count -eq 0`\n\n, the script must exit non-zero or return an explicit `UNGATED`\n\nstatus. A missing key should break the build, not bypass it.`product`\n\nrepo was fixed with a single line in `policy.json`\n\n. `npm run build`\n\nto the gate costs execution time, but it's the only way to catch real deployment breakers.`tsconfig`\n\nand fix the Vitest globs before someone writes a `.tsx`\n\ntest that never actually gets executed.If you run AI agents against your codebase without reading the diffs, go break a file on purpose and run your gate. If it doesn't yell at you, you don't have a safety gate — you just have a script that automatically approves bad code.\n\n*I'm Andréas — full-stack dev, CTO at a B2B SaaS, building my own agent tooling. Portfolio: https://andreas-bodin.vercel.app*", "url": "https://wpnews.pro/news/can-your-verify-gate-actually-fail", "canonical_source": "https://dev.to/arti0/can-your-verify-gate-actually-fail-3ib4", "published_at": "2026-08-15 14:20:43+00:00", "updated_at": "2026-08-15 14:42:52.713607+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "mlops"], "entities": ["Andréas", "Claude Code", "Next.js", "Vite", "Vitest"], "alternates": {"html": "https://wpnews.pro/news/can-your-verify-gate-actually-fail", "markdown": "https://wpnews.pro/news/can-your-verify-gate-actually-fail.md", "text": "https://wpnews.pro/news/can-your-verify-gate-actually-fail.txt", "jsonld": "https://wpnews.pro/news/can-your-verify-gate-actually-fail.jsonld"}}