{"slug": "proofrun-a-local-verification-receipt-for-ai-coding-agents", "title": "ProofRun – a local verification receipt for AI coding agents", "summary": "ProofRun, a new open-source tool released at version 0.2.0, provides cryptographic verification receipts for AI coding agents by binding test results to the exact code state, with statuses of PASS, FAIL, STALE, or NOT RUN. The tool, written by Claude Code under human direction and adversarially reviewed, runs real subprocesses without any LLM calls, operates fully offline, and compares argument arrays rather than strings. It addresses the problem of AI agents claiming tests pass without actually running them, and its own command comparison was found and fixed after a security review.", "body_md": "ProofRun doesn't judge whether your code is correct. It proves — cryptographically, not by asking nicely — which checks actually ran against the exact code you have right now.\n\nAn AI coding agent says \"all tests pass.\" Is that true?\n\nMaybe. It was true the last time the agent actually ran the tests. But that might have been three edits ago. The agent might not even remember running them — it might just be inferring \"the change looks right, tests probably still pass.\" From the words alone, you have no way to tell \"I ran it and it passed\" apart from \"I'm pretty sure it would pass.\"\n\nProofRun closes that gap. Not by making the agent more honest — by making the claim itself checkable.\n\n``` bash\n$ proofrun run test -- pytest\n...\ntest: pass (exit 0, 1841ms)\n\n$ proofrun status\ntest                 PASS    (exit 0, 1841ms)\n\n# code changes after this point — agent or human, doesn't matter\n\n$ proofrun status\ntest                 STALE   (last run: pass, exit 0 — code changed since)\n```\n\nEvery check result is bound to a fingerprint of your exact code state: the git commit, plus a hash of everything uncommitted — staged or not, tracked or not. Change a single byte, and the result flips to `STALE`\n\nautomatically. Nobody has to remember to ask \"does this PASS still count?\"\n\n```\ncurl -L https://github.com/yebiguo/proofrun/releases/download/v0.2.0/proofrun_linux_amd64.tar.gz | tar xz\n# other platforms: https://github.com/yebiguo/proofrun/releases\n```\n\nOr build from source:\n\n```\ngo install github.com/yebiguo/proofrun/cmd/proofrun@latest\nproofrun init                      # writes .proofrun.yml\nproofrun run test -- pytest        # runs pytest for real, binds the result\nproofrun status --strict           # non-zero exit if anything isn't PASS\n```\n\n**No LLM calls, anywhere.** ProofRun doesn't use AI to verify AI. It starts a real subprocess and reads its real exit code — that's the entire mechanism.**Four statuses, never a guess.**`PASS`\n\n,`FAIL`\n\n,`STALE`\n\n,`NOT RUN`\n\n— each one comes from an observed execution, or the documented absence of one. There's no fifth \"probably fine.\"**Fully offline.** Zero network calls, zero telemetry, zero accounts.**Argv-exact, not string-matched.** A check declared as`pytest -k \"foo bar\"`\n\ncan't be satisfied by a command that merely looks similar once flattened to text — ProofRun compares real argument arrays, not strings.\n\nIt does not parse test output, does not judge code quality, and does not auto-fix anything. See [AGENTS.md](/yebiguo/ProofRun/blob/main/AGENTS.md) for the complete boundary.\n\nProofRun was written by an AI coding agent (Claude Code) under human direction, then went through several rounds of independent, read-only adversarial review before the first release. That review found that ProofRun's own command comparison could be tricked: a misquoted shell argument made a check silently run zero tests and still report `PASS`\n\n. Full repro, the exact fix, and why a simple patch wasn't enough → [docs/case-study.md](/yebiguo/ProofRun/blob/main/docs/case-study.md).\n\nEvery fix was verified against a real reproduction before being accepted — not just reviewed for plausibility. A tool built to hold AI agents accountable has no business existing if it can't survive that same scrutiny applied to itself.\n\n```\nproofrun init                      # generate .proofrun.yml\nproofrun run <check-name> -- <cmd> # run <cmd> for real, bind exit code + duration to current git state\nproofrun run-all [--only <name>]   # run every declared check, saving a result after each one\nproofrun status [--strict]         # PASS / FAIL / STALE / NOT RUN per check; --strict exits non-zero if a required check isn't PASS\nproofrun report [--json]           # full report, human- or machine-readable\nchecks:\n  test:\n    command: [pytest]\n    required: true\n  build:\n    command: [npm, run, build]\n    required: true\n  lint:\n    command: [ruff, check, .]\n    required: false\n```\n\n`command`\n\nis an argv list, not a shell string — ProofRun never goes through a shell, and comparing what actually ran against what's declared has to be exact, element for element. `required: true`\n\nis what makes a check block `status --strict`\n\n, which is what you'd wire into a pre-commit hook or CI gate.\n\nEvery result is bound to your current git `HEAD`\n\nplus a SHA-256 hash of `git diff HEAD`\n\ncombined with the contents of any untracked, non-ignored files. `proofrun status`\n\nrecomputes that fingerprint every time and compares it against what's stored locally — any mismatch, down to a single changed space or one new file, reports `STALE`\n\n.\n\n```\non: pull_request\npermissions:\n  contents: read\njobs:\n  verify:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: yebiguo/proofrun@v1\n```\n\nThis does its own checkout of the exact PR head commit — it never trusts whatever the calling workflow already checked out, so a `pull_request`\n\ntrigger can't silently hand it GitHub's synthetic merge-preview commit instead. It then clears out any `receipt.json`\n\nthat came in on the PR branch, downloads a checksum-verified `proofrun`\n\nbinary, and runs `proofrun run-all`\n\nfor real before gating on `proofrun status --strict`\n\n. Nothing about a receipt checked into the PR branch is ever trusted — every result the gate sees was produced by this run.\n\n**Known limitation:** this does not protect `.proofrun.yml`\n\nitself from being weakened by the same PR that changes the code — a PR could loosen or remove a check's command and the Action would faithfully re-run the weaker version. It warns (via a build annotation) when `.proofrun.yml`\n\ndiffers from the PR's base branch, but it does not block on that; review that diff the same way you'd review any other part of the change.\n\n**v0.3**— structured output support for common test runners (pytest, Jest, JUnit)- Signed, tamper-evident receipts are on the radar, not yet designed\n- Protecting\n`.proofrun.yml`\n\nitself from being weakened within the same PR that changes the code (currently only warned about, not blocked — see \"Known limitation\" above)\n\nIssues and PRs welcome. This is a young, pre-1.0 project with a narrow, deliberate scope — see [AGENTS.md](/yebiguo/ProofRun/blob/main/AGENTS.md) before proposing anything that touches STALE detection or the receipt schema; those are the parts this project can least afford to get wrong.\n\nMIT", "url": "https://wpnews.pro/news/proofrun-a-local-verification-receipt-for-ai-coding-agents", "canonical_source": "https://github.com/yebiguo/ProofRun", "published_at": "2026-08-16 03:22:15+00:00", "updated_at": "2026-08-16 03:40:38.096520+00:00", "lang": "en", "topics": ["ai-tools", "ai-safety", "developer-tools"], "entities": ["ProofRun", "Claude Code", "yebiguo"], "alternates": {"html": "https://wpnews.pro/news/proofrun-a-local-verification-receipt-for-ai-coding-agents", "markdown": "https://wpnews.pro/news/proofrun-a-local-verification-receipt-for-ai-coding-agents.md", "text": "https://wpnews.pro/news/proofrun-a-local-verification-receipt-for-ai-coding-agents.txt", "jsonld": "https://wpnews.pro/news/proofrun-a-local-verification-receipt-for-ai-coding-agents.jsonld"}}