{"slug": "your-tests-passed-on-which-version", "title": "Your Tests Passed. On Which Version?", "summary": "The AI CodeOps team outlined a release-gate rule for AI-assisted development: a test result counts only for the exact code version it ran on, so stale green checks from earlier agent iterations can't authorize a merge. The gate inspects every session that touched a pull request or commit and can block merges while a session is still in flight via a blockOnInFlight option. The tool is free during its open beta.", "body_md": "Here's a small thing that causes real incidents.\n\nAn AI agent writes a change. The test suite runs and passes. Someone notices a naming issue and tweaks the change by hand, or the agent is asked for one more small fix. The pull request still shows a green check, so it merges.\n\nBut the green check belongs to the version before the edit. Nobody tested what actually shipped.\n\nWith people, edits after review tend to be few and visible. With AI agents, a change can go through several quick rounds: generate, review, fix, adjust. Each round is a new version. If your process treats \"tests passed at some point\" as \"tests passed\", the faster the iteration, the wider the gap between what was tested and what merges.\n\nIt's a quiet failure. Nothing looks wrong. The check is green, the approval is there, and the diff looks reasonable.\n\nThe fix is a simple rule. A test result counts only for the version it ran on.\n\nIn AI CodeOps, that rule is built into the release gate:\n\nNotice the second point. The old result doesn't count as a failure either. It simply stops counting, because it's evidence about a different change.\n\nThe gate your CI can call follows it too. It looks at every session that touched the pull request or commit. You can also ask it to deny while a session is still running (`blockOnInFlight`), so a merge can't sneak in between a new version and its test run.\n\nYou don't need a special tool to apply the principle. Ask three questions about any green check before merging:\n\nIf you can't answer all three from your tooling, your process has the gap. With AI writing more of the code, it's worth closing.\n\nAI CodeOps is free during the open beta: [https://aicodeops.dev](https://aicodeops.dev)\n\n*Drafted with AI assistance and edited by the AI CodeOps team. Originally published on [Medium](https://medium.com/@aicodeops/your-tests-passed-on-which-version-9da9a9bffd4d).*", "url": "https://wpnews.pro/news/your-tests-passed-on-which-version", "canonical_source": "https://dev.to/aicodeops_dev/your-tests-passed-on-which-version-4ia1", "published_at": "2026-11-02 01:39:08+00:00", "updated_at": "2026-11-02 01:52:27.759520+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "mlops"], "entities": ["AI CodeOps", "Medium"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/your-tests-passed-on-which-version", "markdown": "https://wpnews.pro/news/your-tests-passed-on-which-version.md", "text": "https://wpnews.pro/news/your-tests-passed-on-which-version.txt", "jsonld": "https://wpnews.pro/news/your-tests-passed-on-which-version.jsonld"}}