{"slug": "i-tested-my-ai-bug-hunter-on-44-real-bugs-from-express-eslint-hexo-and-others-it", "title": "I tested my AI bug hunter on 44 real bugs from Express, ESLint, Hexo and others. It caught 32.", "summary": "A developer built BuildProof, an AI bug-hunting tool that attempts to prove a bug exists in a code change rather than guess. Tested against the BugsJS dataset of real bugs from projects including Express, ESLint, Hexo, Karma, Shields, Bower and hessian.js, it caught 32 of 44 re-introduced bugs with no hints, and reported no findings on 4 open pull requests in lodash, koa and axios. The tool runs code in a locked, network-free container that is deleted after each check, and offers a free mode using a local Ollama model.", "body_md": "AI writes more and more of our code, and code review can't keep up. So I built **BuildProof**: a tool that looks at a code change and tries to **prove** a bug, not just guess one.\n\nBugsJS is a public dataset of real bugs from real projects (Express, ESLint, Hexo, Karma, Shields, Bower, hessian.js). I re-introduced each bug and let the hunter look at the change, with no hints.\n\nIt also knows when to stay quiet: I ran it on 4 open pull requests in lodash, koa and axios. It found nothing, and it was right not to. It doesn't invent bugs to look busy.\n\nYour code runs in a locked container with no network and is deleted when the check ends. There's also a free mode that uses your own local model (Ollama).\n\n30-second demo, no project needed:\n\n```\nnpx buildproof try\n```\n\nOn your own git project (3 free cloud checks):\n\n```\nnpx buildproof hunt --base main --cloud\n```\n\nI'd really like feedback, especially on the misses. Which kinds of bugs should it catch that it doesn't?", "url": "https://wpnews.pro/news/i-tested-my-ai-bug-hunter-on-44-real-bugs-from-express-eslint-hexo-and-others-it", "canonical_source": "https://dev.to/billbin1969debug/i-tested-my-ai-bug-hunter-on-44-real-bugs-from-express-eslint-hexo-and-others-it-caught-32-4fn4", "published_at": "2026-11-09 12:43:25+00:00", "updated_at": "2026-11-09 12:55:13.273013+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-agents", "artificial-intelligence"], "entities": ["BuildProof", "BugsJS", "Express", "ESLint", "Hexo", "Karma", "Shields", "Ollama"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-tested-my-ai-bug-hunter-on-44-real-bugs-from-express-eslint-hexo-and-others-it", "markdown": "https://wpnews.pro/news/i-tested-my-ai-bug-hunter-on-44-real-bugs-from-express-eslint-hexo-and-others-it.md", "text": "https://wpnews.pro/news/i-tested-my-ai-bug-hunter-on-44-real-bugs-from-express-eslint-hexo-and-others-it.txt", "jsonld": "https://wpnews.pro/news/i-tested-my-ai-bug-hunter-on-44-real-bugs-from-express-eslint-hexo-and-others-it.jsonld"}}