{"slug": "evidence-gates-for-ai-coding-agents-in-ci-recoverable-merge-over-mean-time-to", "title": "Evidence Gates for AI Coding Agents in CI — Recoverable Merge over Mean Time to Green", "summary": "Angela (Lo An), a practitioner in CI and testing, argues that coding agents' ability to quickly fix failing CI logs and open pull requests can be dangerous if teams treat a green pipeline as proof of safety. She emphasizes that recoverable merge and evidence packs matter more than mean time to green, and provides a checklist for platform engineers, SDEs, and testers to ensure agent speed without silent quality loss. The article outlines risk tiers, evidence requirements, and the principle that 'agent proposes, humans dispose' to maintain accountability.", "body_md": "**Recoverable merge matters more than mean time to green.**\n\nCoding agents can read failing CI logs and open pull requests in minutes. That is useful for mechanical failures. It becomes dangerous when teams treat a green pipeline as proof that an agent-authored change is safe to land.\n\nThis article is a practical checklist for platform engineers, SDEs, and testers who want agent speed without silent quality loss. It is not a product pitch.\n\nMost “AI fixes CI” demos optimize one metric: time until the pipeline is green.\n\nThat metric ignores three questions:\n\nIf those questions are unanswered, the agent is not repairing engineering—it is optimizing a dashboard.\n\nMinimum bar before an agent may open a mergeable PR:\n\n```\n## Rollback\n- Previous green SHA on target: ________\n- Revert plan (one line): ________\n\n## Trigger\n- Failing CI run URL/id: ________\n```\n\nIf the team cannot name the previous green SHA, they are not ready for agent auto-merge.\n\n“I ran the tests” is not evidence. Merge criteria should require an **evidence pack**:\n\n| Field | Example |\n|---|---|\n| Commands | `npm run test:unit && npm run test:api` |\n| Results | unit 412 pass; api 38 pass |\n| Uncovered scope | e2e nightlies not run |\n| Tests edited? | YES — `login.spec.ts` (tightened assertion) |\n\nTreat test diffs as higher risk than production diffs of similar size. Review for:\n\nA green suite that no longer guards the bug is a **false green**.\n\n| Risk tier | Examples | Agent may | Merge requires |\n|---|---|---|---|\n| Low | format, comment typos | open PR | optional policy auto-merge |\n| Medium | test logic, business code | open PR | human owner |\n| High | auth, payments, migrations, release flags | draft only | human (+ often second reviewer) |\n\nRaise automation by classifying Low risk accurately—not by expanding silent merges.\n\nScore these separately from happy-path repair rate:\n\nAsk: did gates **block merge**, even if the model attempted the unsafe action?\n\n```\n## Evidence Pack (agent-assisted change)\n- [ ] Failing run id linked\n- [ ] Branch-only change (no direct default push)\n- [ ] File list attached\n- [ ] Tests edited? yes/no + paths\n- [ ] Exact commands re-run\n- [ ] Uncovered scope stated\n- [ ] Rollback one-liner + previous green SHA\n- [ ] Risk tier: Low / Medium / High\n```\n\nPaste it into your PR template. Block merge when empty—the same way you block PRs without reproduction steps.\n\nDurable pattern: **agent proposes, humans dispose**—especially on Medium/High tiers.\n\nSpeed is real. Accountability is still human.\n\nUntil rollback, reviewable evidence, and merge authority are explicit, automatic landing should not be the default—no matter how impressive the mean time to green looks in a demo.\n\n*Author: Angela (Lo An). Practitioner notes for CI + testing teams. Comments welcome on what your team already requires in PR templates.*", "url": "https://wpnews.pro/news/evidence-gates-for-ai-coding-agents-in-ci-recoverable-merge-over-mean-time-to", "canonical_source": "https://dev.to/lo_an_e746e473b842ff53cf9/evidence-gates-for-ai-coding-agents-in-ci-recoverable-merge-over-mean-time-to-green-2a8h", "published_at": "2026-08-02 07:02:18+00:00", "updated_at": "2026-08-02 07:11:56.777232+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "mlops", "ai-safety"], "entities": ["Angela (Lo An)"], "alternates": {"html": "https://wpnews.pro/news/evidence-gates-for-ai-coding-agents-in-ci-recoverable-merge-over-mean-time-to", "markdown": "https://wpnews.pro/news/evidence-gates-for-ai-coding-agents-in-ci-recoverable-merge-over-mean-time-to.md", "text": "https://wpnews.pro/news/evidence-gates-for-ai-coding-agents-in-ci-recoverable-merge-over-mean-time-to.txt", "jsonld": "https://wpnews.pro/news/evidence-gates-for-ai-coding-agents-in-ci-recoverable-merge-over-mean-time-to.jsonld"}}