Your Tests Passed. On Which Version?

The AI CodeOps team outlined a release-gate rule for AI-assisted development: a test result counts only for the exact code version it ran on, so stale green checks from earlier agent iterations can't authorize a merge. The gate inspects every session that touched a pull request or commit and can block merges while a session is still in flight via a blockOnInFlight option. The tool is free during its open beta.

Here's a small thing that causes real incidents. An AI agent writes a change. The test suite runs and passes. Someone notices a naming issue and tweaks the change by hand, or the agent is asked for one more small fix. The pull request still shows a green check, so it merges. But the green check belongs to the version before the edit. Nobody tested what actually shipped. With people, edits after review tend to be few and visible. With AI agents, a change can go through several quick rounds: generate, review, fix, adjust. Each round is a new version. If your process treats "tests passed at some point" as "tests passed", the faster the iteration, the wider the gap between what was tested and what merges. It's a quiet failure. Nothing looks wrong. The check is green, the approval is there, and the diff looks reasonable. The fix is a simple rule. A test result counts only for the version it ran on. In AI CodeOps, that rule is built into the release gate: Notice the second point. The old result doesn't count as a failure either. It simply stops counting, because it's evidence about a different change. The gate your CI can call follows it too. It looks at every session that touched the pull request or commit. You can also ask it to deny while a session is still running blockOnInFlight , so a merge can't sneak in between a new version and its test run. You don't need a special tool to apply the principle. Ask three questions about any green check before merging: If you can't answer all three from your tooling, your process has the gap. With AI writing more of the code, it's worth closing. AI CodeOps is free during the open beta: https://aicodeops.dev https://aicodeops.dev Drafted with AI assistance and edited by the AI CodeOps team. Originally published on Medium https://medium.com/@aicodeops/your-tests-passed-on-which-version-9da9a9bffd4d .