{"slug": "i-measured-whether-ai-writes-hollower-tests-than-humans-it-doesn-t", "title": "I measured whether AI writes hollower tests than humans. It doesn't", "summary": "A new open-source tool called voidguard scans code repositories for 'void guards' — checks that are present, plausible, but verify nothing. The tool, which can be installed via pip and run as a GitHub Action, identifies tests that never run, type gates that check nothing, settings silently discarded, and CI conditions that cannot fire. Its creator developed voidguard after finding seven such void guards in a single repository in one week.", "body_md": "# Does your green actually check anything?\n\n`voidguard`\n\nis a static scanner for *void guards* —\nchecks that are present, plausible, and verify nothing. It asks one question per\nguard: *could this, as configured, ever be observed to fail?* — and shows its\nevidence for every answer.\n\n```\npip install voidguard\nvoidguard scan .\nname: voidguard\non: [pull_request]\npermissions:\n  contents: read\n  pull-requests: write\njobs:\n  scan:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: driivai/voidguard@v0\n```\n\nReport-only by default — it comments findings on the PR and never fails\nyour build unless you opt in with `fail-on`\n\n. Requires Python 3.11+\nfor the CLI; the Action brings its own.\n\nA guard that **fails** is doing its job.\n\nA guard that's **absent** is at least honest.\n\nThe dangerous one runs, reports **green**, and verifies nothing.\n\n## What it catches\n\n**1 · Tests that never run** Skips gated on an environment flag that is set nowhere your repo actually runs; markers every CI invocation deselects; go, rust and js best-effort.\n\n**2 · Type gates that check nothing** A mypy that cannot see your own types and passes vacuously; imports skipped; check targets matching no files; a weak tsconfig behind an advertised typecheck.\n\n**3 · Settings silently discarded** PYTHON* variables handed to\n\n`python -I`\n\n/`-E`\n\n, which\ndrop them; workflow env set and never read; a Docker ARG consumed after FROM\nwithout re-declaration.**4 · CI conditions that cannot fire** An\n\n`if:`\n\nrequiring an event the workflow's triggers never\ndeliver; schedules with no run on the record; golden-file assertions whose path\nmatches nothing.Every verdict — VOID, WARN, or an honest UNKNOWN — carries its enumerated search set: what was searched, what was found, absent conventional locations named as absent. A tool about unverified claims does not get to make any.\n\n## What it cannot catch\n\nThe taxonomy this tool comes from has **seven** instance-types. voidguard v0\ndetects the shapes of **four**. It would not have caught the other three:\n\n**Semantic voids**— a verdict typed nullable so “nothing” can be mistaken for a value, or a field the code carries but nothing ever persists. These need type-flow and data-flow analysis, not file-shape analysis.**Process voids**— a human approval gate that a merge routed around while every check was green. No scanner catches a decision that nobody waited for.**Anything requiring execution**— voidguard never runs your code. A guard that runs and is*wrong*is outside its question; it only asks whether a guard could ever be observed to fail at all.\n\nWhere static analysis cannot decide, the verdict is UNKNOWN with the reason — because a scanner that overclaims void guards is itself a void guard.\n\n## Where it comes from\n\nIn one week, one repository turned up **seven guards that were present,\nplausible, and void** — a core integrity test that had silently skipped in CI\nsince inception, a type gate that passed while checking nothing, an approval step\na merge walked straight past. Every one of them was green. voidguard is the\ngeneralization of the sweep that found them.", "url": "https://wpnews.pro/news/i-measured-whether-ai-writes-hollower-tests-than-humans-it-doesn-t", "canonical_source": "https://driivai.github.io/voidguard/", "published_at": "2026-07-17 04:48:44+00:00", "updated_at": "2026-07-17 05:21:22.226541+00:00", "lang": "en", "topics": ["developer-tools"], "entities": ["voidguard", "driivai"], "alternates": {"html": "https://wpnews.pro/news/i-measured-whether-ai-writes-hollower-tests-than-humans-it-doesn-t", "markdown": "https://wpnews.pro/news/i-measured-whether-ai-writes-hollower-tests-than-humans-it-doesn-t.md", "text": "https://wpnews.pro/news/i-measured-whether-ai-writes-hollower-tests-than-humans-it-doesn-t.txt", "jsonld": "https://wpnews.pro/news/i-measured-whether-ai-writes-hollower-tests-than-humans-it-doesn-t.jsonld"}}