Recoverable merge matters more than mean time to green.
Coding agents can read failing CI logs and open pull requests in minutes. That is useful for mechanical failures. It becomes dangerous when teams treat a green pipeline as proof that an agent-authored change is safe to land.
This article is a practical checklist for platform engineers, SDEs, and testers who want agent speed without silent quality loss. It is not a product pitch.
Most “AI fixes CI” demos optimize one metric: time until the pipeline is green.
That metric ignores three questions:
If those questions are unanswered, the agent is not repairing engineering—it is optimizing a dashboard.
Minimum bar before an agent may open a mergeable PR:
## Rollback
- Previous green SHA on target: ________
- Revert plan (one line): ________
## Trigger
- Failing CI run URL/id: ________
If the team cannot name the previous green SHA, they are not ready for agent auto-merge.
“I ran the tests” is not evidence. Merge criteria should require an evidence pack:
| Field | Example |
|---|---|
| Commands | npm run test:unit && npm run test:api |
| Results | unit 412 pass; api 38 pass |
| Uncovered scope | e2e nightlies not run |
| Tests edited? | YES — login.spec.ts (tightened assertion) |
Treat test diffs as higher risk than production diffs of similar size. Review for:
A green suite that no longer guards the bug is a false green.
| Risk tier | Examples | Agent may | Merge requires |
|---|---|---|---|
| Low | format, comment typos | open PR | optional policy auto-merge |
| Medium | test logic, business code | open PR | human owner |
| High | auth, payments, migrations, release flags | draft only | human (+ often second reviewer) |
Raise automation by classifying Low risk accurately—not by expanding silent merges.
Score these separately from happy-path repair rate:
Ask: did gates block merge, even if the model attempted the unsafe action?
## Evidence Pack (agent-assisted change)
- [ ] Failing run id linked
- [ ] Branch-only change (no direct default push)
- [ ] File list attached
- [ ] Tests edited? yes/no + paths
- [ ] Exact commands re-run
- [ ] Uncovered scope stated
- [ ] Rollback one-liner + previous green SHA
- [ ] Risk tier: Low / Medium / High
Paste it into your PR template. Block merge when empty—the same way you block PRs without reproduction steps.
Durable pattern: agent proposes, humans dispose—especially on Medium/High tiers.
Speed is real. Accountability is still human.
Until rollback, reviewable evidence, and merge authority are explicit, automatic landing should not be the default—no matter how impressive the mean time to green looks in a demo.
Author: Angela (Lo An). Practitioner notes for CI + testing teams. Comments welcome on what your team already requires in PR templates.