Show HN: DOS – a referee between AI agents that doesn't believe their "done"

A developer released DOS, an open-source tool that verifies AI agents' completion claims by checking git history and other artifacts instead of trusting the agents' own statements. The tool, available as a Python package, can detect false "done" claims, prevent agents from overwriting each other's files, and flag stalled runs. It aims to bring accountability to multi-agent coding workflows.

📊 See it run on real repos:thescores 15 popular AI-built repos roborev, open-interpreter, crewAI, autogen, … — how much agents wrote, which ones, and whether each commit's claim is backed by its own diff. Score yours: scoreboard dos commit-audit --sweep --workspace . BASE..HEAD . https://raw.githubusercontent.com/anthony-chaudhary/dos-kernel/master/docs/assets/caught-lie-cast.svg The whole pitch in one recording: the agent claims two features shipped; git backs one. dos verify answers from the commits, the lie exits 1, and a gate on that exit code refuses the false "done". Every line is the real CLI's verbatim output — scripts/build caught lie cast.py https://github.com/anthony-chaudhary/dos-kernel/blob/master/scripts/build caught lie cast.py re-records it whenever the output changes. https://raw.githubusercontent.com/anthony-chaudhary/dos-kernel/master/docs/assets/loop-hero.svg Run a fleet of agents on one repo. The left loop just feels like progress; the right one you can steer. The only difference is a verdict DOS reads from the real world — here, git — never the agent's word. An AI agent will tell you it finished. DOS checks the real world instead of taking its word — and the nearest piece of the real world is your git history. An agent says it shipped the login endpoint; did it? Run one command, dos verify , and it answers from the artifacts the work left behind, not from what the agent typed: a commit backs the claim → SHIPPED , exit 0 ; nothing landed → NOT SHIPPED , exit 1 . The agent's story never enters into it. Git is just the first witness DOS reads; the file tree, the clock, a CI status, a test environment's own state are others — anything the agent didn't author. dos verify AUTH AUTH1 → SHIPPED AUTH AUTH1 e62f74d exit 0 dos verify AUTH AUTH2 → NOT SHIPPED AUTH AUTH2 exit 1 That's the smallest version. It scales up, too: point a dozen agents at one repo — in CI, in a fleet, racing on the same files — and DOS also tells you which ones are stepping on each other, which one is spinning in circles, and which claim of "done" is real. Every answer comes from the artifacts git, the file tree, the clock , never the narration. It works on a plain git repo with zero config and gets smarter the more you tell it, and the only thing you ever install is one small Python package. ⚡ Just add it — two commands, zero decisions.From the repo where your agent works: pip install dos-kernel dos init --hooks auto finds the agent runtime s you already use, wires in the checks From then on: your agent can't tell you "done"unless the work actually landed, two agents can't silently overwrite each other's files, and a run that stalls gets flagged instead of quietly spinning. Nothing about your workflow changes, and you don't need to learn any of the vocabulary below to be covered. It prints the one config file it wrote; deleting the dos hook entries there undoes it. No runtime detected? It says so and lists the names to pick from — it never guesses. v0.28.0 · 5,600+ tests · CI: Python 3.11–3.13 on Linux + a Windows 3.13 smoke run · the only runtime dependency is PyYAML · MIT . 🧭 Where to go next:the why & evidence plain-words story, the 20-lines-of-bash answer, what's proven , wire it into your stack MCP · hooks · install , the syscall + CLI reference , or,reading this as an AI agent?, AGENTS.md — build/test/check in three lines. The full map is the router just below. 🔤 Five words the rest of this page leans on.Aplanis a named goal AUTH ; aphaseis one shippable step of it AUTH1 ; alaneis the slice of the file tree one agent may touch; theoracleis the part of DOS that reads the evidence and rules; astampis the mark a shipped phase leaves in a commit subject AUTH1: … — the thing the oracle greps for. That's the whole vocabulary. A coding agent does work, then tells you how it went. Usually the story is true; sometimes it's the cheerful "all work completed " from a worker that shipped nothing. With one agent you catch that yourself by re-reading its output — a real tax you already pay. Run twenty at once and that tax stops being payable: nobody reads everything, each worker grades its own homework, and the unchecked problems pile up quietly until the codebase sorta works and nobody can safely change it. DOS is the referee that never reads the story — it reads what happened the commit, the file, the clock and hands you a verdict no narration can move. It costs about an afternoon, has one runtime dependency, and stays in its lane: it tells you what happened , never whether the code is good — quality stays with your tests and reviews. The full plain-words version https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/guide/why-a-referee.md the-plain-words-version . Every number here is scored against a fact the agent can't fake a test environment's DB state, git history . A DOS gate caught 15 "I shipped it" lies in 258 tasks across two models with zero false alarms ; the same referee stopped 6 of 8 silent collisions on one shared record; quitting doomed runs at the right moment saved ~11% of fleet compute with 0 of 1,634 winners wrongly killed ; and the reward-set admission label lifted acceptance precision 60% → 100% by purging poison a self-graded collector keeps. The methodology, the two money-moment figures, and the projected-vs-bet honesty gradient are in what's proven and what's still a bet . This page keeps the hook, the demo, and the failure it fixes. Everything deeper lives on a focused page — find the question you arrived with and jump: | You're asking… | Go to | |---|---| "What is this in plain words, and why should my team care? Is it real?" | | "Show me it working, fast." Try it in 60 seconds try-it-in-60-seconds , just below — one command "I already run agents — how do I wire the verdict into my stack?" Wire it in https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/guide/wire-it-in.md — MCP, runtime hooks, the exit-code tier, fleet frameworks, and the install matrix "What's the full command / syscall surface?" The syscall ABI & CLI reference https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/guide/cli-reference.md — every verb, the three live screens, the verdict journal "I run a fleet every day — how do I watch it, triage it, debug it?" Operating a fleet https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/guide/operating-a-fleet.md + Debug a stuck fleet https://github.com/anthony-chaudhary/dos-kernel/blob/master/examples/playbooks/06 debug-a-stuck-fleet.md "How do I bend it to my org without forking it?" Extending it https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/guide/extending.md — the seven axes, the docs index, the playbooks "What is actually proven, and can I re-run it?" For researchers https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/guide/for-researchers.md — claims → invariants → reproduction "I'm an AI agent orienting in this repo." — what DOS is in three lines, build/test/check, the ~5 files worth reading AGENTS.md https://github.com/anthony-chaudhary/dos-kernel/blob/master/AGENTS.md "What surfaces are stable and what's the deprecation window?" — the compatibility promise, what the version number means, and what will never break docs/STABILITY.md https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/STABILITY.md Got a terminal? This runs the whole thing in a throwaway repo — one command scaffolds it, makes a real commit, verifies it, and cleans up after itself: pip install dos-kernel PyYAML is the only runtime dep dos quickstart → SHIPPED AUTH AUTH1 … then NOT SHIPPED AUTH AUTH2 One SHIPPED , one NOT SHIPPED : the first is a claim git can back, the second is a claim nothing landed for. That contrast is the product. The demo closes with a router to wherever you already run agents — a Claude Code / Cursor tab dos init --hooks , an MCP host, a CI step, or a fleet — so your next move is one line, not a docs dig. Add --keep ./demo to keep the repo and poke at it. Don't even want the install? uvx --from dos-kernel dos quickstart runs the same demo ephemerally — nothing left behind. The same thing by hand, in five lines, is docs/QUICKSTART.md . https://raw.githubusercontent.com/anthony-chaudhary/dos-kernel/master/examples/demo/verify-moment.svg Two equally confident claims, one verdict each — SHIPPED for the one git can back, NOT SHIPPED for the one nothing landed for. Every string is verbatim output of examples/demo/verify demo.sh https://github.com/anthony-chaudhary/dos-kernel/blob/master/examples/demo/verify demo.sh . Step through it locally https://github.com/anthony-chaudhary/dos-kernel/blob/master/examples/demo/verify visual.html for the click-through version it's an HTML file — clone the repo and open it in a browser; GitHub shows its source, not the running page . The smallest real win: in a CI step or dispatch loop, replace the line that trusts an agent's "done" with dos verify PLAN PHASE and branch on its exit code 0 shipped / 1 not . No parsing, no plan, no config — the CI integration cookbook https://github.com/anthony-chaudhary/dos-kernel/blob/master/examples/playbooks/cookbook-ci-integration.md walks it end-to-end. To run it on a repo shaped like yours, start with Onboard a repo in 10 minutes https://github.com/anthony-chaudhary/dos-kernel/blob/master/examples/playbooks/01 onboard-a-repo.md . Point the same witness at a review queue when commits pile up faster than anyone can read them. Residual review https://github.com/anthony-chaudhary/dos-kernel/blob/master/examples/residual review/ folds commit-audit 's per-commit verdict into three bands — CLEARED the diff witnessed the claim, so spend ~0 attention re-asking "did it do what it said" , RESIDUAL a claim git couldn't back — the human's 100% , and the no-claim rest. On this repo's own last 200 commits it cleared 170 of 171 checkable claims: that's the re-review you skip, proven by git rather than a model's confidence score. CLEARED means the change's shape matched its claim — not that the code is correct; correctness review still applies to every commit. The band can only ever ask for more eyes, never fewer. Next level up — wire the verdict into your own stack: Wire it in. Run a pile of agents at once with nobody refereeing, and here's how it goes: each worker reports its own success, and you believe the reports, because what else is there to go on? The unchecked problems pile up quietly — a lie here, two agents clobbering the same file there, a little scope creep, one worker spinning in circles — until the codebase sorta works and nobody can safely change it. The trouble is you launched the agents and then let them grade their own homework. DOS gives you the missing signal — a verdict from ground truth — so the loop closes. Here is the same fleet under both regimes: The two regimes as a flowchart — NO REFEREE: you believe the narration; DOS ADJUDICATES: you steer on a verdict flowchart LR subgraph OPEN "NO REFEREE — you believe the narration" direction TB A1 "agent: 'done '" -- B1 "believed" A2 "agent: 'done '" -- B1 A3 "agent: 'done '" -- B1 B1 -- C1 "silent corruption piles up<br/ lies · collisions · spin " C1 -- D1 "'sorta works' — can't be changed" end subgraph CLOSED "DOS ADJUDICATES — you steer on a verdict" direction TB A4 "agent: 'done '" -- V{{"dos verify<br/ reads git"}} V -- |in git ancestry| S "SHIPPED exit 0 " V -- |found nowhere| N "NOT SHIPPED exit 1 " S -- L "land it" N -- R "re-dispatch / flag — caught" R -.verdict steers the loop.- A4 end Here are the failures a fleet actually produces, each next to the ground truth that quietly contradicts the worker's story — and the verdict DOS hands back: | A worker… | …but the ground truth is | DOS verdict | |---|---|---| | says it shipped a unit of work | no commit ever landed | verify → caught lie | | tried, but the commit silently failed | no commit ever landed | verify the flake — indistinguishable from a lie without git | | edits files another worker owns | two agents, one shared file | arbitrate → refuse the second | | overruns the file region it claimed | footprint reaches beyond the declared tree | scope-gate → REFUSE before the write lands | | reports "making progress" | 0 commits, only a fresh heartbeat | liveness → SPINNING | The first row is the most common one. The classic tell is a cheerful one-liner, "all work completed " , from a worker that did little or nothing. DOS never reads that line; it reads the ground truth, so the claim collapses the instant no artifact backs it more in docs/108 https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/108 the-cheap-lie-and-the-narration-taxonomy.md . That's also what makes it cheap to adopt: verify needs no plan, no registry, no config, and the exit code is the verdict — any shell or CI step can branch on it without parsing a word. Prefer to watch it move? The two loops are also a self-contained animation you step through one frame at a time — clone the repo and open docs/assets/loop visual.html https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/assets/loop visual.html in a browser. It's an HTML file, so GitHub shows its source rather than running it — open it locally. Lease scope — single filesystem today. The verification half verify , commit-audit , liveness travels across machines freely because it reads git history. The admission half arbitrate , lane leases is local-filesystem only: the WAL lives on one disk, and workers on separate machines share no serialization point. A fleet that runs all its workers on one machine or in one shared filesystem is fully covered; a fleet spanning multiple hosts should treat dos arbitrate as advisory not a hard mutex until a remote-lease driver ships. See docs/366 https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/366 single-filesystem-lease-boundary.md for the design. It works on a plain git init with zero config, and gets smarter the more you tell it. You don't adopt a framework and pick a tier; you start at the shallow end and it keeps paying off as you wade deeper — the same kernel the whole way: Zero config. Point dos verify PLAN PHASE at a plain git repo — no plan, no registry, no dos.toml . It answers from commit history alone via grep-subject / via none . This is the whole of QUICKSTART https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/QUICKSTART.md and the day-one CI win above. Tell it your structure. dos init writes a dos.toml lanes, paths, ship grammar as data ; add a plan doc and dos plan lays each phase's claim beside the oracle's verdict. Here's exactly what a plan file looks like https://github.com/anthony-chaudhary/dos-kernel/blob/master/examples/plans/example-plan.md copyable, round-trips with the built-in reader , and four worked example workspaces https://github.com/anthony-chaudhary/dos-kernel/tree/master/examples/workspaces . Teach it your own types. Declare your own block reasons, gate verdicts, output renderers, admission predicates, a model-backed judge, a custom plan dialect, or a whole host driver — all as workspace policy, never a fork. The map is seven extension axes + the copy-me docs/HACKING.md https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/HACKING.md . examples/dos ext/ That slope is how deep your config goes. The other axis is how you call the referee at all — and you adopt through whichever surface matches how you already work, not by restructuring your stack. The same kernel verdicts are reachable through every row here, lowest-friction first: | Surface | Adopt it when… | The move | |---|---|---| MCP server | you drive an agent through an MCP host Claude Desktop, Cursor, Cline, an Agent-SDK app | add one line to the host config { "command": "dos-mcp" } and ask the agent to dos verify its own last claim — zero code. The advisory path the agent asks . See | Runtime hooks | you run an agent loop Claude Code, Cursor, Codex CLI, Gemini CLI and want the verdict to act, not just be available | dos init --hooks <runtime wires the verdict into that host's own hook config — a refused call is denied before it runs, a false "done" is refused. The enforcement path the host denies . One command, no hand-edited YAML. See | CLI exit-code | you have any command-running environment — a CI step, a pre-push hook, or an agentic CLI like aider whose lint/test-cmd trusts a "done" | branch on a dos verb's exit code dos verify : 0 shipped / 1 not; dos commit-audit : 0 clean / 1 over-claim — the verdict , no hook adapter and no MCP client. The honest tier for hook-less hosts Windsurf, Warp, Zed . The is the exit code | Python API | your dispatcher/orchestrator is already Python | import dos and call the pure syscalls dos.oracle.is shipped , dos.arbiter.arbitrate , … — state-in / verdict-out, no subprocess. The | Fleet framework | your fleet already runs on LangGraph, CrewAI, AutoGen, or the OpenAI/Claude Agents SDK | bolt the referee onto the framework's own seam — a referee node, a termination condition only git can satisfy, an output guardrail with a git tripwire. One function, no rewrite; every seam executed against the real framework. The | Swarm runtime Hermes, OpenClaw , or a SwarmClaw-style autonomous swarm — privileged tools, shared memory docs / task boards, and no lock manager for either guard action refuses an arbitrary-exec command before it runs , and acquire lease / release lease bracket each shared-state write so the lost update never lands. No import dos — it shells the CLI; Hermes' pre tool call hook also speaks DOS natively dos hook pretool --dialect hermes . The runnable, A/B-measured Hermes / OpenClaw worked example https://github.com/anthony-chaudhary/dos-kernel/tree/master/examples/hermes integration + docs/278 https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/278 integrating-dos-with-hermes-and-openclaw-the-missing-lock-manager-for-agent-swarms.md . Skill pack dos init --skills drops editable SKILL.md screenplays that wire the syscalls into a snapshot → audit → gate → take-a-lane loop. See QUICKSTART §2 https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/QUICKSTART.md . Driver computed , or you add a provider-backed judge dos/drivers/<host .py a LaneTaxonomy + a config factory , loaded by name, never imported by the kernel. The map is HACKING.md https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/HACKING.md .The two axes are independent: a zero-config repo can adopt through any surface, and a deeply-configured one still answers over the same CLI and MCP tools. Start at the top row — it's the one that costs nothing to try. The first two rows also compose: MCP advises the agent checks its own work , hooks enforce the host stops a bad action — wire both for the full loop. Those surfaces are the upstream half of the value chain — who calls the referee. The same verdicts also flow downstream, to the systems that act on them: every adjudication lands in a verdict journal that dos export drains to your observability stack Datadog / Honeycomb / Grafana — docs/266 https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/266 the-verdict-exporter-shipping-the-journal-to-where-dashboards-live.md , dos notify pushes what-needs-a-human to Slack, dos reward gates what a fine-tune may train on, and dos attest mints a signed receipt a skeptic can check without loop access docs/246 https://github.com/anthony-chaudhary/dos-kernel/blob/master/docs/246 dos-attest-the-portable-signed-receipt.md . One kernel, one verdict vocabulary, from the agent's tool call to your dashboard. Next level up — run it every day: Operating a fleet. The ideas here are written up in a paper — "Verification Is All You Need — But Not Where You Think" — on the out-of-loop referee for agent fleets. A built PDF lives at paper/releases/ https://github.com/anthony-chaudhary/dos-kernel/tree/master/paper/releases ; the arXiv preprint is in preparation. Until the arXiv ID lands, cite the repository: @misc{dos kernel, title = {Verification Is All You Need --- But Not Where You Think}, author = {Chaudhary, Anthony}, howpublished = {\url{https://github.com/anthony-chaudhary/dos-kernel}}, note = {DOS --- the Dispatch Operating System; arXiv preprint in preparation}, year = {2026} } MIT — see LICENSE https://github.com/anthony-chaudhary/dos-kernel/blob/master/LICENSE .