Here's a small thing that causes real incidents.
An AI agent writes a change. The test suite runs and passes. Someone notices a naming issue and tweaks the change by hand, or the agent is asked for one more small fix. The pull request still shows a green check, so it merges.
But the green check belongs to the version before the edit. Nobody tested what actually shipped.
With people, edits after review tend to be few and visible. With AI agents, a change can go through several quick rounds: generate, review, fix, adjust. Each round is a new version. If your process treats "tests passed at some point" as "tests passed", the faster the iteration, the wider the gap between what was tested and what merges.
It's a quiet failure. Nothing looks wrong. The check is green, the approval is there, and the diff looks reasonable.
The fix is a simple rule. A test result counts only for the version it ran on.
In AI CodeOps, that rule is built into the release gate:
Notice the second point. The old result doesn't count as a failure either. It simply stops counting, because it's evidence about a different change.
The gate your CI can call follows it too. It looks at every session that touched the pull request or commit. You can also ask it to deny while a session is still running (blockOnInFlight), so a merge can't sneak in between a new version and its test run.
You don't need a special tool to apply the principle. Ask three questions about any green check before merging:
If you can't answer all three from your tooling, your process has the gap. With AI writing more of the code, it's worth closing.
AI CodeOps is free during the open beta: [https://aicodeops.dev](https://aicodeops.dev)
*Drafted with AI assistance and edited by the AI CodeOps team. Originally published on [Medium](https://medium.com/@aicodeops/your-tests-passed-on-which-version-9da9a9bffd4d).*