{"slug": "5-mistakes-ai-coding-agents-make-and-how-to-catch-them-before-merge", "title": "5 mistakes AI coding agents make, and how to catch them before merge", "summary": "A developer published a pre-merge checklist for reviewing AI-generated code, cataloguing five recurring failure modes: calls to nonexistent functions or packages, swallowed errors, assertion-free or mock-only tests, unverified test-run claims, and refactors that quietly change behavior. The guidance recommends searching for every unfamiliar call, requiring real assertions on observable behavior, rerunning test commands personally, and splitting behavior changes into separate commits. A 12-point filter and a Claude Code code-reviewer subagent are offered free on GitHub.", "body_md": "AI coding agents write code fast. They also make the same handful of mistakes over and over. None of them are exotic, which is exactly why they slip through review: the diff looks plausible, the summary sounds confident, and the reviewer is busy.\n\nHere's what to check every AI-assisted diff for before it merges.\n\n**How to spot it:** calls to functions, methods, flags or packages that don't exist in the versions you actually depend on. The name is usually reasonable, which is the problem.\n\n**What to do:** search the codebase and the library's docs for every unfamiliar call. Delete or replace anything you can't find. If the tests \"pass,\" check that they actually execute that line.\n\n**How to spot it:** empty `catch` blocks, `except: pass`, or an error that's logged and then ignored so the function carries on with bad state.\n\n**What to do:** handle the error, rethrow it, or write a comment explaining why ignoring it is safe. \"It made the test pass\" isn't a reason.\n\n**How to spot it:** tests with no assertions, tests that only check a mock was called, or tests that mirror the implementation line by line so they can never fail.\n\n**What to do:** rewrite them to assert behavior someone outside the function would notice: the return value, the saved record, the HTTP response.\n\n**How to spot it:** \"All tests pass\" or \"Fixed the bug,\" with no command output shown.\n\n**What to do:** run the command yourself and read the output. Every time. An agent's summary of a test run isn't a test run.\n\n**How to spot it:** a diff labeled as a refactor that also changes defaults, error messages, log lines or the order things happen in.\n\n**What to do:** move behavior changes into their own commit, with a test that shows the new behavior on purpose.\n\nBefore merging, ask:\n\nWould I be comfortable defending this diff in review if I had written every line myself?\n\nIf not, it isn't ready, no matter who or what wrote it.\n\nA checklist only helps if it's used on every diff, not just the ones that look suspicious. Two things that help:\n\nThe full 12-point slop filter, the pre-merge checklist and a ready-to-use code-reviewer subagent for Claude Code are free on GitHub: [lean-team-products/ai-code-review-gates](https://github.com/lean-team-products/ai-code-review-gates). Copy them into your repo and you're set up in two minutes.\n\n*Originally published on the [Lean Team blog](https://www.leanteamproducts.com/blog/ai-code-review-mistakes). Free download: the [Codebase Antipattern Checklist](https://www.leanteamproducts.com/?checklist=1), 48 signs a codebase is getting harder to change.*", "url": "https://wpnews.pro/news/5-mistakes-ai-coding-agents-make-and-how-to-catch-them-before-merge", "canonical_source": "https://dev.to/lean-team-products/5-mistakes-ai-coding-agents-make-and-how-to-catch-them-before-merge-3092", "published_at": "2026-11-21 20:57:59+00:00", "updated_at": "2026-11-21 21:12:13.405608+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "artificial-intelligence"], "entities": ["Claude Code", "GitHub", "Lean Team"], "also_reported_by": [], "takeaway": "Review every AI-assisted diff for hallucinated calls, swallowed exceptions, assertion-free tests, unverified test-run claims and hidden behavior changes before merging.", "recommendation": {"advice": "Rerun the test command yourself and read the output on every AI-written diff, and rewrite any test that only checks a mock was called.", "playbook": "coding-agents", "playbook_url": "https://wpnews.pro/playbooks/coding-agents"}, "alternates": {"html": "https://wpnews.pro/news/5-mistakes-ai-coding-agents-make-and-how-to-catch-them-before-merge", "markdown": "https://wpnews.pro/news/5-mistakes-ai-coding-agents-make-and-how-to-catch-them-before-merge.md", "text": "https://wpnews.pro/news/5-mistakes-ai-coding-agents-make-and-how-to-catch-them-before-merge.txt", "jsonld": "https://wpnews.pro/news/5-mistakes-ai-coding-agents-make-and-how-to-catch-them-before-merge.jsonld"}}