Verdict: Evidence-First Agent Harness for Reproducible Bug Fixes

Verdict, an evidence-first agent harness for reproducible bug fixes, has been introduced. It enforces a strict contract where bugs are considered innocent until reproduced, requiring agents to produce verifiable evidence before any patch is accepted. The harness uses three bounded subagents—Hunter, Surgeon, and Insurance—to find triggers, localize changes, and create regression tests, with all evidence stored in a versioned ledger.

Most flaky bug reports end in one of two places: "cannot reproduce" or a patch nobody can prove fixed the problem. Verdict enforces a stricter contract: bugs are innocent until reproduced. No patch, no claim of success, no merge until the agent produces verifiable evidence that the failure exists and can be triggered on demand. This is not an autonomous patch generator. It is an evidence-producing agent harness for the difficult step that comes before a patch: proving the bug is real, isolating the trigger, and building a regression test that prevents recurrence. Verdict treats bug investigation as a bounded experiment, not a conversation. An LLM can read a stack trace and propose a plausible explanation quickly. Plausible is not the same as reproduced. For an intermittent failure, the questions that matter are concrete: The harness refuses to claim a reproduction unless the evidence crosses a deterministic threshold. Every observation stays in the evidence ledger. An inconvenient result cannot disappear just because it weakens the story. Verdict uses three bounded subagents that run sequentially: GitHub issue | v Hunter: find the trigger | v Surgeon: localize the change | v Insurance: keep it fixed | v Maintainer review Each agent has a specific scope and cannot proceed without satisfying its evidence contract. Hunter searches only the condition matrix and command budget approved by the maintainer. It runs an approved command repeatedly under approved conditions. Successful, failed, partial, and unresolved runs all stay in the evidence ledger. The agent does not get to cherry-pick results. If a condition fails 3 times out of 10, that ratio is part of the evidence. If a different condition fails 10 times out of 10, that difference is part of the evidence. Hunter outputs: Surgeon narrows the reproduced condition to the smallest suspect range the records support. It uses the trigger condition from Hunter to bisect the repository history or module boundary. Static inspection stays visibly different from a proven execution boundary. Surgeon does not author a patch. It localizes the change that introduced the failure and hands that range to the maintainer. Surgeon outputs: Insurance converts the reproduction into a regression plan: the test name, fixture, failing assertion, and expected pass condition. This is not a patch. It is a test case that will fail until someone fixes the underlying issue. Insurance outputs: The maintainer reviews the test case, merges it still failing , and then works on a patch. The patch is only considered successful if the test case passes. Verdict runs as a GitHub Action or standalone CLI. The maintainer configures the harness with: The harness enforces these boundaries at runtime. An agent cannot escalate privileges or exceed the budget. | Component | Scope | Evidence Contract | |---|---|---| | Hunter | Condition search | Trigger + failure rate + control | | Surgeon | Range bisection | Suspect boundary + execution proof | | Insurance | Regression plan | Test case + fixture + assertion | | Harness | Orchestration | Budget enforcement + artifact storage | Every run produces: These artifacts are stored in a structured ledger, not a flat log. The ledger is versioned alongside the repository. An agent cannot rewrite history or hide a failed run. The harness uses a content-addressed store for artifacts. Identical outputs same stdout, same exit code get deduplicated. This keeps the ledger compact even when an agent runs the same command 100 times. Verdict does not trust the agent. The harness enforces: The maintainer reviews the evidence ledger before merging any test case. The agent never gets write access to the main branch. The harness exposes: Common failure modes: Verdict runs as: The harness does not require a persistent server. It is stateless except for the evidence ledger, which is versioned alongside the repository. Use Verdict when: Avoid Verdict when: The core insight is the reproduction-proof boundary. An agent cannot claim success without evidence. An agent cannot hide a failed run. An agent cannot escalate privileges or exceed the budget. This turns a flaky bug report into a verifiable experiment.