Show HN: Linejudge – an independent verification harness for coding agents Linejudge, an independent verification harness for coding agents, found that 8 out of 8 agent-authored patches passed machine checks but only 6 out of 8 actually fixed the target issues in sqlite-utils, with two patches failing to address the bugs. The harness, built by Phillip Mex, runs agents on goals and independently verifies results using real commands, diffs, and exit codes, never trusting the agent's own success claims. The independent line judge for coding agents. Never trust the player's call. Coding agents routinely claim success. linejudge runs your agent on a goal, then independently verifies the result — real commands, real diffs, real exit codes — and renders its own verdict. The agent's opinion of its own work is never consulted. Eight agent-authored patches for eight real sqlite-utils https://github.com/simonw/sqlite-utils issues, put through every gate the harness has: | evidence | result | |---|---| | Runs succeeded | 7/8 | | Independently verified pass — judged by verifiers, not by the claim | 8/8 | | Regression test proven — fails with the fix reverted | 8/8 | | Diff reviewed against the issue it claims to close | 6/8 | Three machine gates, 8/8 on every one. Reading the diffs found two that don't fix the bug: 439 — the patch counts bytes as utf-8-sig . Fixes utf-16, regresses the default: a 708-byte plain-utf-8 CSV reports 1011. The added test covers utf-16-le only. 762 — the constraint parser masks string literals but not SQL comments, so a commented-out -- CHECK id 0 is re-emitted as an active constraint. A table that accepted id = -1 starts rejecting it. Neither is reachable by a test suite that doesn't already know about the bug. Machine verification is necessary and it is not sufficient — so the scoreboard reports those levels separately instead of as one number. Full evidence trail: PROOF.md /phillipmex/linejudge/blob/main/PROOF.md . The long version, with both rejects walked through in detail: Eight patches passed every check I had. Two of them were wrong. https://phillipmex.github.io/linejudge/ bash $ python proofs/demo.py --root demo 3 tasks, one agent LIES 20260721T…-issue-101-config-loader…: SUCCESS 20260721T…-issue-102-add-json-flag…: FAILED ← claimed success, wrote nothing 20260721T…-issue-103-docs-quickstart…: SUCCESS The failed run's agent reported Status: SUCCESS . The files exist verifier checked the filesystem and disagreed. The verdict, not the claim, decides the run status. No tokens spent — see the full sample PROOF.md /phillipmex/linejudge/blob/main/docs/PROOF-sample.md . Every agent loop on the market grades its own homework: the model says "done" and the harness believes it. linejudge splits the roles. The agent plays the point; the harness calls the lines: Independent verification — a declarative verifier spec command , files exist , diff constraints , http check executed by the harness, outside the agent session. Full spec /phillipmex/linejudge/blob/main/docs/verifier-spec.md . Blast-radius guarding — read-only directories are snapshotted git status before and after every run; any unexpected mutation fails the run with diagnostics. Verified-diff-only writes — write access goes through a git worktree on an unmerged linejudge/