In my last post I pointed Bite at its own code. This time I wanted the opposite: projects I trust, to see whether Bite says anything useful about good code, or just makes noise.
I had Claude run Bite's free check on three well-known projects, 40 recent changes each:
| Project | Changes checked | Noticed by a test | Real gaps |
|---|---|---|---|
| Rich | 40 | 15 | 1 small: <kbd> keys in Markdown |
| Click | 40 | 30 | 1 small: help on custom setting types |
| qs | 40 | 40 | none |
In all three projects, every bug fix Bite could check had a test that noticed when the fix was taken out. qs was the cleanest: all 40 changes had one.
The two gaps are small. Rich's Markdown support for <kbd> tags (it draws <kbd>Ctrl</kbd> in bold yellow) has no test. In Click, a change let every kind of parameter take a help text, but its tests only try the two kinds that could already do that. Neither is a bug, just something nothing would warn about if it broke.
Every unnoticed change was read by hand. Apart from those two, they were things a test can't be expected to catch: type hints, speed-ups, Windows-only code, unused code being removed, and rewrites that work the same.
Rich: too many false alarms. On the first run, 21 of Bite's 22 flags on Rich were noise. That's a different problem from last time. Those flags weren't wrong about the code; they just weren't worth anyone's time. So Bite now also skips commits that only touch type hints, changes for another Python version, and speed-ups, since tests check results, not speed. On Rich that removes 11 of the 21, and the <kbd> gap is still flagged. On Click the new rule fired once, correctly.
Click: my old bug, back in a new shape. In the last post, Bite was fooled by an installed copy of its own code. Click hit a cousin of that. Click keeps its code in a src/ folder and is installed with pip install -e ., like many Python projects. So when Bite checked an old commit, Python quietly loaded today's Click instead of the old one, and all 40 checks came back "tests already fail". I'd fixed the installed-copy trap for one case and missed this one. Bite now makes each copy use its own src/ folder.
The full table, with links to each gap, is in Bite's README. Disclosure: I'm not a coder. Claude ran these checks, wrote Bite's fixes and helped me draft this post. Every flag was read by hand, and the numbers match the README table.