The Test Guarding My List of Deliberate Bugs Made Every Bug Look Caught for a Month A developer running a daily autonomous Claude Code agent on a Windows PC found that a self-check test guarding the agent's list of deliberately injected bugs had been masking real gaps for a month. Because the list-integrity test lived in the same suite the injected bugs were run against, every injected break removed the text that test checked for, causing it to fail and the tool to count each break as caught regardless of whether a real test noticed it. After excluding the list test from mutation runs and adding a pre-run baseline check, a full rerun of 116 entries found 114 caught and 2 genuinely survived, both of which received new tests. A field note from the autonomous Claude Code agent I run every day on one Windows PC. The numbers come from its own ledgers, not from memory. Free to read first: a PDF sample of the book behind this setup — its first 3 chapters: https://dbsoul.gumroad.com/l/autonomous-ai-agents-claude-code-free-sample https://dbsoul.gumroad.com/l/autonomous-ai-agents-claude-code-free-sample My agent checks its own test suite by breaking its code on purpose. A list holds each break: a file, an exact piece of text from that file, and what to put in its place. The piece is often one line, sometimes part of a line or several lines. For each entry, a tool swaps the replacement in and runs the suite, stopping at the first failure. If something fails, the break counts as caught. If everything passes, the break survived, and the guard in that spot has no test behind it. On August 25, an audit found entries in that list pointing at code I had already changed. A break whose text no longer exists can't be put in, so it checks nothing. I added a test for the list itself: for every entry, the exact text must still be in its file. If it isn't, the test fails and names the entry. That test lived in the same suite the breaks were run against. Putting a break in removes its own text from the source. So with any break in place, the list test failed, and the tool counted the break as caught. Whether a real test noticed it no longer mattered. It went unnoticed for a month. On September 25, an audit of five new breaks found two that only the list test had caught. The tests written for those two passed with the break in place. Since then, the tool leaves the list test out of its runs. The list test still runs in the normal suite, where its job is to catch stale entries. The tool also runs the suite once before any break goes in and stops if anything already fails. A suite that is red before the break makes every break look caught for the same reason. I added two tests for the two uncaught breaks, and all five were caught with the list test left out. That still left every entry already on the list. Early the next morning, a scheduled run went through the whole list without the list test: 114 caught, 2 survived, 0 targets missing. One survivor removed a filter that kept a product from recommending itself. The other removed the setting that turns off headers and footers when the agent prints a PDF. Both entries had been added after the list test went in. Each got a test the same morning. So the month of false catches hid two real gaps among 116 entries. I only knew that after rerunning everything. Once it stopped judging breaks, the list test went back to the work it was written for. Between September 26 and November 15, eleven work sessions in my log recorded it failing on an entry I had left pointing at old code. Two of those were on November 15. In each, code I had edited still had an entry naming its old text. Where this comes from. Every post here comes from one setup I run daily: a CLAUDE.md, memory files the agent reads before it touches anything, and a separate auditor agent that returns PASS or FAIL. The first 3 chapters of the book that walks through it are free as a PDF: https://dbsoul.gumroad.com/l/autonomous-ai-agents-claude-code-free-sample https://dbsoul.gumroad.com/l/autonomous-ai-agents-claude-code-free-sample The full edition is 11 chapters plus 4 ready-to-use templates CLAUDE.md starter, memory files, auditor checklist, measurement guide and a hands-on section for every chapter, $19 as a PDF: https://dbsoul.gumroad.com/l/autonomous-ai-agents-claude-code https://dbsoul.gumroad.com/l/autonomous-ai-agents-claude-code Prefer Google Play? The same book is on Google Play Books: https://play.google.com/store/books/details?id=hesIEgAAQBAJ https://play.google.com/store/books/details?id=hesIEgAAQBAJ Questions about the setup are welcome in the comments — I'll answer with what actually happened, not theory.