{"slug": "bugs-are-innocent-until-reproduced-building-verdict-an-evidence-first-agent", "title": "Bugs Are Innocent Until Reproduced: Building Verdict, an Evidence-First Agent Harness", "summary": "A developer built Verdict, an evidence-first agent harness that turns GitHub issues into bounded investigations, requiring bugs to be reproduced before being fixed. The tool uses three subagents—Hunter, Surgeon, and Insurance—to find triggers, localize changes, and create regression plans, with all observations recorded in an evidence ledger. Verdict successfully reproduced TrueForge issue #417, where snapshot registration could hang indefinitely, achieving a deterministic verdict with 10 of 10 stalled runs and 0 of 10 responsive controls.", "body_md": "Most flaky bug reports end in one of two places: \"cannot reproduce\" or a patch nobody can prove fixed the problem.\n\nI built [Verdict](https://github.com/himanshu748/verdict) around a stricter idea:\n\nBugs are innocent until reproduced.\n\nVerdict turns a GitHub issue into a bounded investigation. It runs an approved command repeatedly under approved conditions, keeps every observation and refuses to claim a reproduction unless the evidence crosses a deterministic threshold.\n\nThis is not an autonomous patch generator. It is an evidence-producing agent harness for the difficult step that comes before a patch.\n\nAn LLM can read a stack trace and propose a plausible explanation quickly. Plausible is not the same as reproduced.\n\nFor an intermittent failure, the questions that matter are concrete:\n\nVerdict treats those as an experiment rather than a conversation.\n\nVerdict uses three bounded subagents:\n\n```\nGitHub issue\n    |\n    v\nHunter: find the trigger\n    |\n    v\nSurgeon: localize the change\n    |\n    v\nInsurance: keep it fixed\n    |\n    v\nMaintainer review\n```\n\nHunter searches only the condition matrix and command budget approved by the maintainer. Successful, failed, partial and unresolved runs all stay in the evidence ledger. An inconvenient result cannot disappear just because it weakens the story.\n\nSurgeon narrows the reproduced condition to the smallest suspect range the records support. Static inspection stays visibly different from a proven execution boundary. Surgeon does not author a patch.\n\nInsurance converts the reproduction into a regression plan: the test name, fixture, failing assertion and publication manifest. A draft pull request can only be created through a workflow the maintainer explicitly approves.\n\nEach act is allowed to claim less than the act before it. None can talk the deterministic reducer into a stronger verdict.\n\nVerdict reproduced [TrueForge issue #417](https://github.com/truefoundry/trueforge/issues/417), where snapshot registration can wait indefinitely when an upstream request never resolves.\n\nThe pinned runtime used `@truefoundry/trueforge-core@0.1.4#DaytonaSandboxProvider`\n\nand ran two conditions:\n\n| Condition | Result |\n|---|---|\n`daytona-stalled-endpoint` |\n10 of 10 runs matched, `REPRODUCTION_PINNED`\n|\n`daytona-responsive-endpoint` |\n0 of 10 matched, `NOT_REPRODUCED`\n|\n\nThe control is the important half. A condition that fails every time next to one that never fails is stronger evidence than twenty failures with no contrast.\n\nAnyone can recompute the record:\n\n```\npnpm --filter @verdict/agent verify:runtime-evidence\n```\n\nThe verifier returns:\n\n```\n{\n  \"verdict\": \"REPRODUCED\",\n  \"stalledRuns\": 10,\n  \"responsiveControls\": 10,\n  \"provider\": \"@truefoundry/trueforge-core@0.1.4#DaytonaSandboxProvider\",\n  \"canonicalSha256\": \"a8bb5dd22e083782bd7782fccb0a1343b59fc77ea8525b6358fecc9b5b8baffa\"\n}\n```\n\nThe evidence binds the observations to the TrueForge session, Hunter thread, repository commit, npm provenance commit and shared source blob.\n\nThe model gathers candidate observations. It does not decide what those observations prove.\n\nVerdict's evidence contract is simple:\n\nThe same records always produce the same verdict. That is the boundary between an agent exploring a problem and a system making a claim.\n\nThe [recorded case](https://verdict-steel.vercel.app/case/trueforge-417) renders the executed artifact. It includes both conditions, all twenty runs and the recomputable hash.\n\nThe [interactive workspace](https://verdict-steel.vercel.app/case/demo) is a conceptual fixture. Every generated value is labelled. It is not quietly presented as live runtime evidence.\n\nThat distinction matters for a product whose entire argument is that a claim needs a record.\n\nVerdict also exercised its publication boundary against real GitHub. After explicit approval, a nonce-bound workflow ran, verified the external reproduction reference and created a draft pull request in Verdict's repository. The upstream TrueForge repository remained read-only.\n\nThe workflow proof says `runtimeReproducedByThisWorkflow: false`\n\n. That is deliberate. The provider run reproduced the bug. GitHub Actions verified the harness and published the independently checkable proof. Combining those into one vague \"verified\" flag would erase the boundary.\n\nEvery substantive change went through a pull request reviewed by Qodo before merge.\n\nThe most useful findings were not dramatic crashes. They were mismatches between the implementation and what the project claimed:\n\n`main`\n\nand pull requests.Those reviews fit the product philosophy perfectly: do not ship a stronger claim than the evidence supports.\n\nThe repository runs the same gate locally and in CI:\n\n```\npnpm lint\npnpm typecheck\npnpm test\npnpm build\npnpm --filter @verdict/agent verify:runtime-evidence\n```\n\nThe current suite contains 220 tests across the agent, protocol and web packages.\n\nVerdict is open source under the MIT licence and was built for the WeMakeDevs x TrueFoundry Agent Harness Hackathon.\n\nThe goal is not to make an agent sound certain. The goal is to make certainty inspectable.", "url": "https://wpnews.pro/news/bugs-are-innocent-until-reproduced-building-verdict-an-evidence-first-agent", "canonical_source": "https://dev.to/himanshu_748/bugs-are-innocent-until-reproduced-building-verdict-an-evidence-first-agent-harness-50lf", "published_at": "2026-08-30 04:43:06+00:00", "updated_at": "2026-08-30 05:22:37.684669+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "artificial-intelligence"], "entities": ["Verdict", "TrueForge", "GitHub", "Hunter", "Surgeon", "Insurance"], "alternates": {"html": "https://wpnews.pro/news/bugs-are-innocent-until-reproduced-building-verdict-an-evidence-first-agent", "markdown": "https://wpnews.pro/news/bugs-are-innocent-until-reproduced-building-verdict-an-evidence-first-agent.md", "text": "https://wpnews.pro/news/bugs-are-innocent-until-reproduced-building-verdict-an-evidence-first-agent.txt", "jsonld": "https://wpnews.pro/news/bugs-are-innocent-until-reproduced-building-verdict-an-evidence-first-agent.jsonld"}}