hollowtest: find tests that pass but prove nothing

A developer built hollowtest, a static analysis tool that detects test functions which pass but contain no meaningful assertions, such as tautologies, presence checks, or empty bodies. The tool uses Python's AST to classify assertions and can be integrated into CI pipelines to catch hollow tests before they land in main.

Coverage can be green while your test suite is empty of proof. That is not a hypothetical. Stack Overflow’s 2025 developer survey found that a majority of developers spend more time fixing almost-right AI output — and tests are one of the places that “almost right” hides best. An AI assistant will happily write a function, then write a test that calls it, prints something, and ends with assert result is not None . Coverage goes up. Confidence should not. I built hollowtest https://github.com/SybilGambleyyu/hollowtest to catch those tests statically, before they land in main. Mutation testing is the gold standard for “do my tests actually catch bugs?” It is also slow, noisy, and a poor default for every PR. Style linters do not care whether you asserted anything meaningful. Coverage only asks whether a line ran . What we needed in the AI-coding loop was a fast, deterministic gate : did this test body contain real proof, or is it hollow? Hollow patterns I kept seeing in AI-generated suites: pass , a docstring, or an ellipsis left as a “TODO test” assert True , assert x == x assert result is not None / assert result / assert len x 0 mock.assert called once with ... and nothing about real outcomesAll of these pass. All of them inflate coverage. None of them prove behavior. pip install git+https://github.com/SybilGambleyyu/hollowtest.git hollowtest tests/ It walks Python test files pytest and unittest conventions , finds test functions, and classifies their assertions via the AST. No network, no model, no API key — stdlib only. | Rule | Severity | Meaning | |---|---|---| | HT001 | error | No assertions | | HT002 | error | Only tautologies | | HT003 | warning | Only presence/truthiness checks | | HT004 | warning | Only mock call verifications | | HT005 | error | Empty / docstring-only / pass | Meaningful patterns are left alone: assert result == expected , self.assertEqual , with pytest.raises ... , or a mock check plus a real value assertion. Example against a tiny sample: test app.py:12:1: warning HT003 in test add exists Existence assertions only: Only checks presence/truthiness ... test app.py:18:1: error HT005 in test placeholder Empty test body: Test body is empty, docstring-only, or just pass test app.py:23:1: error HT001 in test smoke print No assertions: No assert / self.assert / pytest.raises found ... Exit codes work for CI --fail-on error|warning , and there is JSON plus GitHub Actions annotation output. The gap I cared about was not “another AI that reviews your AI.” It was a boring, reliable check that treats test hollowness as a first-class defect class. If your agents write tests or your teammates paste them , run this on the suite the same way you run the suite itself. GitHub Actions sketch - run: pip install git+https://github.com/SybilGambleyyu/hollowtest.git - run: hollowtest tests/ --format github --fail-on warning If you find false positives or hollow patterns it misses, open an issue — that feedback is the roadmap. MIT licensed. Zero runtime dependencies. Python 3.10+.