cd /news/ai-tools/your-ai-says-fixed-would-a-test-noti… · home › topics › ai-tools › article
[ARTICLE · art-164363] src=dev.to ↗ pub= topic=ai-tools verified=true sentiment=↑ positive

Your AI says "fixed". Would a test notice if it wasn't?

A developer released Bite, a free open-source tool that reverts each recent fix in a codebase and reruns the test suite to verify whether any test actually guards against the bug returning. On public projects it flagged a memory leak in prosemirror-py that a commit had claimed to fix, and a fix has since been merged upstream. The tool runs on copies without editing source code and is installable via pip, pipx, or uvx.

read1 min views2 publishedNov 5, 2026

When an AI coding tool says "Fixed, and I added a test", how do you know the test would catch the bug coming back?

Bite is a small free tool that checks. It takes each recent fix back out of your code and runs your tests. If they still pass, nothing guards that fix, and Bite names it. On real public projects, not a staged demo, it found a memory leak in prosemirror-py that a commit said it had fixed. The fix for it has since been merged there.

Try it, no key needed. It works on copies and never edits your code. With plain Python, two lines in any terminal:

`python -m pip install bite-check`

`python -m bite_check --free`

It shows a demo on a made-up project first, then tells you how to check your own (drag its folder into the window). Have pipx or uv? pipx run bite-check --free or uvx bite-check --free.

Doing MLH Hack Week (Nov 6-12)? Bite has good first issues: small Python fixes, each with "where to look" and "done when". They get picked up fast, so the label always shows what's still open. Comment on one to claim it.

https://github.com/connabyss/bite/labels/good%20first%20issue

So: which of your AI's "fixed" commits would you bet on? Run it and find out.

The Bite cat says hi.

── more in #ai-tools 4 stories · sorted by recency
── more on @bite 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/your-ai-says-fixed-w…] indexed:0 read:1min 2026-11-05 · —