{"slug": "verdict-evidence-first-agent-harness-for-reproducible-bug-fixes", "title": "Verdict: Evidence-First Agent Harness for Reproducible Bug Fixes", "summary": "Verdict, an evidence-first agent harness for reproducible bug fixes, has been introduced. It enforces a strict contract where bugs are considered innocent until reproduced, requiring agents to produce verifiable evidence before any patch is accepted. The harness uses three bounded subagents—Hunter, Surgeon, and Insurance—to find triggers, localize changes, and create regression tests, with all evidence stored in a versioned ledger.", "body_md": "Most flaky bug reports end in one of two places: \"cannot reproduce\" or a patch nobody can prove fixed the problem. Verdict enforces a stricter contract: bugs are innocent until reproduced. No patch, no claim of success, no merge until the agent produces verifiable evidence that the failure exists and can be triggered on demand.\n\nThis is not an autonomous patch generator. It is an evidence-producing agent harness for the difficult step that comes before a patch: proving the bug is real, isolating the trigger, and building a regression test that prevents recurrence.\n\nVerdict treats bug investigation as a bounded experiment, not a conversation. An LLM can read a stack trace and propose a plausible explanation quickly. Plausible is not the same as reproduced. For an intermittent failure, the questions that matter are concrete:\n\nThe harness refuses to claim a reproduction unless the evidence crosses a deterministic threshold. Every observation stays in the evidence ledger. An inconvenient result cannot disappear just because it weakens the story.\n\nVerdict uses three bounded subagents that run sequentially:\n\n```\nGitHub issue\n    |\n    v\nHunter: find the trigger\n    |\n    v\nSurgeon: localize the change\n    |\n    v\nInsurance: keep it fixed\n    |\n    v\nMaintainer review\n```\n\nEach agent has a specific scope and cannot proceed without satisfying its evidence contract.\n\nHunter searches only the condition matrix and command budget approved by the maintainer. It runs an approved command repeatedly under approved conditions. Successful, failed, partial, and unresolved runs all stay in the evidence ledger.\n\nThe agent does not get to cherry-pick results. If a condition fails 3 times out of 10, that ratio is part of the evidence. If a different condition fails 10 times out of 10, that difference is part of the evidence.\n\nHunter outputs:\n\nSurgeon narrows the reproduced condition to the smallest suspect range the records support. It uses the trigger condition from Hunter to bisect the repository history or module boundary.\n\nStatic inspection stays visibly different from a proven execution boundary. Surgeon does not author a patch. It localizes the change that introduced the failure and hands that range to the maintainer.\n\nSurgeon outputs:\n\nInsurance converts the reproduction into a regression plan: the test name, fixture, failing assertion, and expected pass condition. This is not a patch. It is a test case that will fail until someone fixes the underlying issue.\n\nInsurance outputs:\n\nThe maintainer reviews the test case, merges it (still failing), and then works on a patch. The patch is only considered successful if the test case passes.\n\nVerdict runs as a GitHub Action or standalone CLI. The maintainer configures the harness with:\n\nThe harness enforces these boundaries at runtime. An agent cannot escalate privileges or exceed the budget.\n\n| Component | Scope | Evidence Contract |\n|---|---|---|\n| Hunter | Condition search | Trigger + failure rate + control |\n| Surgeon | Range bisection | Suspect boundary + execution proof |\n| Insurance | Regression plan | Test case + fixture + assertion |\n| Harness | Orchestration | Budget enforcement + artifact storage |\n\nEvery run produces:\n\nThese artifacts are stored in a structured ledger, not a flat log. The ledger is versioned alongside the repository. An agent cannot rewrite history or hide a failed run.\n\nThe harness uses a content-addressed store for artifacts. Identical outputs (same stdout, same exit code) get deduplicated. This keeps the ledger compact even when an agent runs the same command 100 times.\n\nVerdict does not trust the agent. The harness enforces:\n\nThe maintainer reviews the evidence ledger before merging any test case. The agent never gets write access to the main branch.\n\nThe harness exposes:\n\nCommon failure modes:\n\nVerdict runs as:\n\nThe harness does not require a persistent server. It is stateless except for the evidence ledger, which is versioned alongside the repository.\n\n**Use Verdict when:**\n\n**Avoid Verdict when:**\n\nThe core insight is the reproduction-proof boundary. An agent cannot claim success without evidence. An agent cannot hide a failed run. An agent cannot escalate privileges or exceed the budget. This turns a flaky bug report into a verifiable experiment.", "url": "https://wpnews.pro/news/verdict-evidence-first-agent-harness-for-reproducible-bug-fixes", "canonical_source": "https://dev.to/mech_app_ai/verdict-evidence-first-agent-harness-for-reproducible-bug-fixes-flk", "published_at": "2026-08-30 10:05:19+00:00", "updated_at": "2026-08-30 10:23:20.216917+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "mlops"], "entities": ["Verdict"], "alternates": {"html": "https://wpnews.pro/news/verdict-evidence-first-agent-harness-for-reproducible-bug-fixes", "markdown": "https://wpnews.pro/news/verdict-evidence-first-agent-harness-for-reproducible-bug-fixes.md", "text": "https://wpnews.pro/news/verdict-evidence-first-agent-harness-for-reproducible-bug-fixes.txt", "jsonld": "https://wpnews.pro/news/verdict-evidence-first-agent-harness-for-reproducible-bug-fixes.jsonld"}}