{"slug": "building-codeguard-ai-in-public-the-ai-fix-that-made-a-failing-test-pass", "title": "Building CodeGuard AI in Public: The AI Fix That Made a Failing Test Pass", "summary": "A developer building CodeGuard AI, an AI-powered GitHub pull-request reviewer, documented a failure mode in which the tool detected a failing test but proposed a fix that rewrote the assertion to match the actual output, turning assertEquals(2, 1) into assertEquals(1, 1) and making the test pass without addressing the underlying bug. The developer logged the behavior as a bug and said it will be fixed after the current testing batches, noting that an AI reviewer must verify that a suggested fix preserves the test's original intent.", "body_md": "I’m testing CodeGuard AI, my AI-powered GitHub PR reviewer, and today I caught an interesting failure.\n\nHere’s the test scenario.\n\n```\nassertEquals(2, 1, \"Discount calculation is incorrect\");\n```\n\nThe test expects `2`, but the actual result is `1`. So it fails, as intended.\n\nGood news: CodeGuard correctly identified that the test was failing.\n\nBut then came the problem.\n\nThe AI suggested changing the assertion to:\n\n```\nassertEquals(1, 1, \"Discount calculation is incorrect\");\n```\n\nNow the test passes.\n\nBut did we fix the underlying problem?\n\n**No. We changed the expectation to match the actual result.**\n\nThe test no longer checks whether the code produces the intended result. It simply confirms that `1` equals `1`.\n\nDetecting a failing test is not enough. An AI reviewer must also ensure that its suggested fix preserves the test’s original intent.\n\nI’ve logged this as a bug and will address it after completing the current testing batches. For now, I’m continuing to test the other features and record the failures before fixing them together.\n\nThat’s the reality of building in public: sometimes the most useful update is what your own product gets wrong.\n\n**What would you want CodeGuard to validate before allowing an AI-suggested test fix?**\n\nI'm building CodeGuard AI for developers who want more trustworthy AI-assisted code reviews.\n\nIf you'd like to explore the source code, you can find it here: [CodeGuard AI on Gumroad](https://javacoder716.gumroad.com/l/codeguard-ai)\n\nI'm continuing to test it, document the failures, and improve it based on what I learn.", "url": "https://wpnews.pro/news/building-codeguard-ai-in-public-the-ai-fix-that-made-a-failing-test-pass", "canonical_source": "https://dev.to/sweety717/building-codeguard-ai-in-public-the-ai-fix-that-made-a-failing-test-pass-eg7", "published_at": "2026-10-27 09:39:12+00:00", "updated_at": "2026-10-27 09:42:49.328006+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-agents", "ai-safety"], "entities": ["CodeGuard AI", "GitHub", "Gumroad"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/building-codeguard-ai-in-public-the-ai-fix-that-made-a-failing-test-pass", "markdown": "https://wpnews.pro/news/building-codeguard-ai-in-public-the-ai-fix-that-made-a-failing-test-pass.md", "text": "https://wpnews.pro/news/building-codeguard-ai-in-public-the-ai-fix-that-made-a-failing-test-pass.txt", "jsonld": "https://wpnews.pro/news/building-codeguard-ai-in-public-the-ai-fix-that-made-a-failing-test-pass.jsonld"}}