Rich, Click and qs passed my fix checker. My checker didn't.

A developer behind the open-source bug checker Bite ran its free check against 40 recent changes each in Rich, Click and qs, finding that every fix the tool could verify had a test that caught its removal, with only two small untested gaps: Rich's Markdown `<kbd>` support and Click's `help` text on custom setting types. The exercise also surfaced two tooling flaws: 21 of Bite's 22 initial flags on Rich were noise, prompting new rules that skip type-hint-only, cross-version and speed-up commits, and an editable-install trap in Click caused all 40 checks to falsely report failing tests until Bite was changed to isolate each copy's `src/` folder.

In my last post https://dev.to/connabyss/i-pointed-my-bug-checker-at-itself-33j5 I pointed Bite https://github.com/connabyss/bite at its own code. This time I wanted the opposite: projects I trust, to see whether Bite says anything useful about good code, or just makes noise. I had Claude run Bite's free check on three well-known projects, 40 recent changes each: | Project | Changes checked | Noticed by a test | Real gaps | |---|---|---|---| | Rich | 40 | 15 | 1 small: <kbd keys in Markdown | | Click | 40 | 30 | 1 small: help on custom setting types | | qs | 40 | 40 | none | In all three projects, every bug fix Bite could check had a test that noticed when the fix was taken out. qs was the cleanest: all 40 changes had one. The two gaps are small. Rich's Markdown support for <kbd tags it draws <kbd Ctrl</kbd in bold yellow has no test. In Click, a change let every kind of parameter take a help text, but its tests only try the two kinds that could already do that. Neither is a bug, just something nothing would warn about if it broke. Every unnoticed change was read by hand. Apart from those two, they were things a test can't be expected to catch: type hints, speed-ups, Windows-only code, unused code being removed, and rewrites that work the same. Rich: too many false alarms. On the first run, 21 of Bite's 22 flags on Rich were noise. That's a different problem from last time. Those flags weren't wrong about the code; they just weren't worth anyone's time. So Bite now also skips commits that only touch type hints, changes for another Python version, and speed-ups, since tests check results, not speed. On Rich that removes 11 of the 21, and the <kbd gap is still flagged. On Click the new rule fired once, correctly. Click: my old bug, back in a new shape. In the last post, Bite was fooled by an installed copy of its own code. Click hit a cousin of that. Click keeps its code in a src/ folder and is installed with pip install -e . , like many Python projects. So when Bite checked an old commit, Python quietly loaded today's Click instead of the old one, and all 40 checks came back "tests already fail". I'd fixed the installed-copy trap for one case and missed this one. Bite now makes each copy use its own src/ folder. The full table, with links to each gap, is in Bite's README https://github.com/connabyss/bite famous-repos-bite-has-checked . Disclosure: I'm not a coder. Claude ran these checks, wrote Bite's fixes and helped me draft this post. Every flag was read by hand, and the numbers match the README table.