In my last post, Bite checked some well-known projects and found a few small gaps: changes that no test would notice if they broke. Two of them:
<kbd>Ctrl</kbd> is drawn in bold yellow, and no test checks it.
Finding a gap is the easy half. Someone still has to write the test. So this week Bite got a new option, --write-tests, that tries to write it.
For each change no test noticed, a fresh AI worker writes one small test on a copy of the project. Then Bite doesn't take its word for it. It runs the test itself:
If either part goes wrong, Bite throws the test away and says so. It uses your own Anthropic API key, and --budget caps what a run spends.
| Gap | Bite's test | With the change | Without it | Cost |
|---|---|---|---|---|
Rich <kbd> |
prints <kbd>Ctrl</kbd> , checks Ctrl is bold yellow |
passes, twice | fails: Ctrl not bold yellow | $0.14 |
| chi XML | sends XML through chi's compressor, checks it comes back gzipped | passes, twice | fails: XML not gzipped | $0.10 |
Total: $0.34. That's more than the two rows add up to, because of the next part.
The first chi try got thrown away, and that was Bite's fault. Go runs tests by folder, not by file, and Bite asked Go to run one file alongside the whole project. Go refused before running anything. Bite's rule caught it: a test that fails before it even runs doesn't count. So it threw the test out instead of showing a fake result. The fix is in PR #67, and the second try worked.
Before Bite could do this, Claude had already written a test for each gap the slow way, to confirm the gaps were real. So there was an answer key.
Here's the Rich one, exactly as Bite wrote it:
from rich.console import Console
from rich.markdown import Markdown
def test_kbd_inline_html_is_styled():
console = Console(
file=__import__("io").StringIO(),
force_terminal=True,
color_system="truecolor",
width=40,
)
console.print(Markdown("Press <kbd>Ctrl</kbd> now"))
output = console.file.getvalue()
assert "\x1b[1;93mCtrl\x1b[0m" in output
On my own copy of Rich, Bite ran as a GitHub Action on a pull request. It found the <kbd> gap, wrote the test, and opened a small pull request with the test in it. I merged that pull request, and Bite's check on the original pull request turned green. Gap found, test written, test merged, without anyone typing a test.
It's in v0.7.0 (pipx upgrade bite-check). On your own computer: bite --write-tests. In GitHub Actions, the setup is in Bite's README. Since this trial, Bite also asks a second fresh check whether each test is fair before showing it.
Disclosure: I'm not a coder. Claude ran these trials, wrote Bite's code and helped me draft this post. The numbers come from the run logs.