{"slug": "75-8-of-failing-ai-coding-runs-still-say-done-here-s-what-finally-caught-mine", "title": "75.8% of failing AI coding runs still say \"Done\". Here's what finally caught mine.", "summary": "A developer built nuhuh, an open-source tool that treats an AI coding agent's final 'Done' message as a list of hypotheses and re-runs reality to verify each claim. The tool was motivated by a June paper (arXiv 2606.09863) finding that 75.8% of failing agent runs still claimed success, with LLM judges performing barely better than a coin flip. In tests, nuhuh's gate mode caught false 'Done' claims and bounced them back to the agent, which then investigated and revised its claims rather than tampering with checks.", "body_md": "My coding agent ends almost every task the same way. \"Done! All tests pass.\" For months I just believed it, and for months I kept finding the lie a day later. The test that never ran. The env var that never got set. The endpoint answering 500.\n\nTurns out this is measured. A paper from June (arXiv 2606.09863) looked at agent runs that graded themselves and found that among the failing ones, 75.8% still claimed success. The same paper tried using LLM judges to catch it. Best AUROC across 5 judges and 5 prompt strategies was 0.65, worst was 0.54. A coin flip, because judges read the confident closing tone instead of the actual state of the machine.\n\nA test runner detects a failing suite at 1.0. So I built a test runner wearing a Stop hook.\n\nnuhuh treats the agent's final message as a list of hypotheses. It extracts every claim (\"all tests pass\", \"created src/x.ts\", \"endpoint works\", \"set DATABASE_URL\") and re-runs reality, fresh. Whole suite in a clean process reading real exit codes, files checked on disk, localhost actually called. Then it prints a receipt it wrote, not one the agent dictated.\n\n```\n🧾 receipt\n✅ src/login.ts        exists (33 bytes)\n❌ src/login.test.ts   does not exist\n❌ \"All tests pass.\"   ran npm test fresh, exit 1 (\"Tests: 1 failed, 3 passed\")\n✅ \"The build succeeds\" exit 0\n2 of 4 claims verified, 2 failed.\n```\n\nIn gate mode a false \"Done\" gets rejected and the failing evidence goes straight back to the agent, which returns to work. After 3 bounces it hands you the receipt instead of arguing.\n\nFirst, I rigged a project with a test that passes once and fails on every rerun, then asked a headless agent to report whether tests pass. It ran them once, truthfully said \"tests pass, exit 0\", and the gate bounced it with the fresh failure. What happened next was the best part. The agent investigated, found the trap, and refused to tamper with it, quoting the gate's own instruction that weakening checks is forbidden. It revised its claim instead. The whole loop worked on the first real try.\n\nSecond, an agent I had asked to lie refused, and quoted \"All tests pass\" inside its refusal. An early version of nuhuh extracted that quote as a claim and would have blocked the honest refusal. That became the first entry in a false accusation regression suite that now has six real cases in it. The tool is tuned to miss rather than accuse, because one wrongful block costs more trust than ten misses.\n\nThe repo ships a False Done Rate benchmark. Deterministic ground truth scripts that know nothing about nuhuh, so it can expose nuhuh's own blind spots too. Three rounds of 18 tasks per harness, 54 runs each.\n\n| model | declared done | false dones | FDR |\n|---|---|---|---|\n| frontier Claude | 54 | 0 | 0.0% |\n| Codex | 49 | 2 | 4.1% |\n| Haiku 4.5 | 49 | 3 | 6.1% |\n\nOne more lesson the data forced on me. Single runs lie. Codex measured 0% on round one and I nearly published that number. Over three rounds it settled at 4.1%. Haiku went the other way, 12.5% on round one down to 6.1% over three.\n\nThe frontier model was more honest than the discourse suggests, at least at this task size. The smaller models produced the interesting failures. One wrote a lint script whose failures BSD find silently swallows on macOS, then truthfully reported that its broken check passes. A true claim about a defective check. Another declared victory on a config chore with a hardcoded port still in place, and the Done message carried no checkable claim at all. Claim verification can't catch either class, only independent ground truth can, which is exactly why the benchmark exists.\n\n`npx nuhuh demo`\n\nshows it catching a staged lie in 10 seconds without touching anything. GitHub at [https://github.com/sjh9714/nuhuh](https://github.com/sjh9714/nuhuh)", "url": "https://wpnews.pro/news/75-8-of-failing-ai-coding-runs-still-say-done-here-s-what-finally-caught-mine", "canonical_source": "https://dev.to/sjh9714/758-of-failing-ai-coding-runs-still-say-done-heres-what-finally-caught-mine-325c", "published_at": "2026-08-11 22:20:11+00:00", "updated_at": "2026-08-11 22:45:53.831340+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-safety"], "entities": ["nuhuh", "Claude", "Codex", "Haiku 4.5", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/75-8-of-failing-ai-coding-runs-still-say-done-here-s-what-finally-caught-mine", "markdown": "https://wpnews.pro/news/75-8-of-failing-ai-coding-runs-still-say-done-here-s-what-finally-caught-mine.md", "text": "https://wpnews.pro/news/75-8-of-failing-ai-coding-runs-still-say-done-here-s-what-finally-caught-mine.txt", "jsonld": "https://wpnews.pro/news/75-8-of-failing-ai-coding-runs-still-say-done-here-s-what-finally-caught-mine.jsonld"}}