# How to Stop Catching the Same AI Mistakes in Every Code Review > Source: > Published: 2026-10-26 08:54:50+00:00 # How to Stop Catching the Same AI Mistakes in Every Code Review Adding comments such as “please do not write to the `x` table directly” is a PITA most of the time. If it happens occasionally, fine. But recurring comments? You’d better prepare another liter of coffee. And when this happens a few times a month, you’ve basically got yourself a very expensive cron job. Here’s how to stop it. A comment left more than twice is an unwritten policy. To fix it, mine your code review history for recurring feedback, then route each pattern to the cheapest layer that can enforce it. Formatting and simple syntax patterns belong in formatters and linters. Rules that require understanding what a change actually *does* belong in a semantic check, such as an invariant in [Aviator Verify](https://verify.aviator.co). Judgment calls are left to humans. **TL;DR:** AI slop is often code that ignores your team’s conventions, and senior engineers keep catching it by hand. Turn each recurring review comment into a rule: filter for rules that apply across authors, send syntax issues to linters and semantic rules to invariants, write each rule as a single verifiable sentence, test it in draft mode, then promote it. ## Why THAT Same Old Review Comment, Again? **Coding agents start every session from zero.** Cursor, Claude Code, Copilot and their pals start new sessions without the feedback your team has accumulated across hundreds of review threads. So the agent repeats the same mistake, and since agents write code fast, those mistakes pile up at a much higher volume. Yikes. **Review knowledge is stored in a reviewer’s memory.** The senior engineer who remembers last quarter’s partial-write bug only catches it on the PRs she reviews. Her comment fixes one PR, but the underlying rule never becomes part of the codebase’s checks. You could try stuffing every rule into the prompt. [Aviator’s docs on verification layers](https://docs.aviator.co/verify/concepts/verification-layers) explain why that is the wrong approach. The agent would need to remember the team’s full ruleset on every submission. One bad submission could slip through if a rule gets overlooked, and every ruleset change would mean re-instructing the agent. Files like `CONTRIBUTING.md` do help, but they are just guidance an agent may or may not follow. They don’t block a merge. ## How Do You Turn a Review Thread Into an Invariant? First, a quick definition. In Verify, an **invariant** is a team-defined rule applied to every matching change. Acceptance criteria describe what *this* change should do; invariants describe what *every* change should respect ([Aviator docs](https://docs.aviator.co/verify/concepts/invariants)). Think of your invariant catalog as an anti-slop registry that contains mistakes your team has already caught and corrected. ### 1. Mine the Code Review History Scan the last quarter of PRs for recurring comments. Aviator’s docs note that many useful invariants start as review comments left more than twice. If you’d rather not grep through threads by hand, Verify can do the mining for you. The **AI from PR comments** source analyzes your PR history and drafts invariants based on recurring feedback. According to the docs, it’s often the highest-yield option. **AI from docs** works similarly by extracting rules from `CONTRIBUTING.md`, `LLM.md`, or similar files. ### 2. Keep Only the Rules That Apply Across Authors Filter hard. Aviator’s recurrence test is a good one: if you’d write the same criterion on the next ten submissions, it should be an invariant. Personal preferences, like “I would have named this differently,” belong in human review. ### 3. Route Each Pattern to the Cheapest Layer Not every recurring comment needs an LLM-backed check. Send each one to the cheapest layer that can reliably enforce it: | What the recurring comment looks like… | Cheapest layer | Why | |---|---|---| | “Fix the indentation” / “Sort these imports” | Formatter | Fully mechanical, auto-fixable | | “Use camelCase here” / “No `console.log` “ | Linter | Simple pattern, deterministic and fast | | “This should not be `any` “ | Type checker | The compiler already knows | | “Go through the repository, not raw SQL” / “Handlers must authenticate first” | Verify invariant | Needs an understanding of what the change does | | “Is this the right abstraction?” | Human reviewer | Judgment, not a rule | ### 4. Rewrite Each One as a Checkable Statement An invariant has to stand on its own, without the PR context. *Aviator Verify’s Invariants in action*Aviator’s [guidance on writing invariants](https://docs.aviator.co/verify/concepts/invariants)comes down to a few rules: - **Be specific about the assertion and vague about the implementation.** Name the rule, not the file path or function, so that it survives renames. - **Make it verifiable in isolation.** Stick to rules that can be checked from the diff or a single runtime probe. - **Do not write the fix.** The verifier explains what’s wrong, and the author decides how to fix it. - **One rule per invariant.** Put scope in conditions. - **Allow for exceptions.** An invariant can be waived, too. | Original review comment | Checkable invariant | |---|---| | “Do not write to `users` directly, use`UserRepository.UpdateProfile` .” | Writes to the users table must go through UserRepository. Direct INSERT, UPDATE, or DELETE statements against the users table are not allowed outside the repository package. | | “Migrations should be reversible.” | All migrations must declare a `down` block. | | “Error messages should not be in first person.” | User-facing error messages must not use first-person pronouns. | ## Where Do Linters Belong? Wherever they already are. 👍 Formatting, naming conventions, and simple code patterns belong in your formatter and linter (ESLint, Ruff, golangci-lint, and friends). They are deterministic, run in milliseconds on a developer’s laptop, and can often auto-fix issues. | | Formatter / Linter | Verify invariant | Human reviewer | |---|---|---|---| | **Best for** | Formatting, naming, banned patterns | Cross-cutting semantic rules (layering, auth, data access) | Design, trade-offs, “should we build this at all?” | | **Deterministic** | Yes | Partly (code analysis where possible, labeled LLM fallback otherwise) | No | | **Knows the change’s intent** | No | Yes, from the submitted intent and criteria | Yes | | **Where it runs** | Editor, pre-commit, CI | On the PR’s verification run | Whenever someone has time | | **Limitations** | Cannot express “what the code means” | Needs a well-written body; selector can skip it | Does not scale with agent output | ## Next Steps Here’s the loop, one last time: mine the comments, keep the rules that apply across authors, route syntax to linters and semantics to invariants, write each rule as one checkable sentence, then trial, promote, and prune. You can do most of that on a whiteboard and still end up with a better `CONTRIBUTING.md`. But if you’d rather skip the manual mining and have those rules checked on every PR (including the ones your agents open at 2 a.m.), [Aviator Verify](https://verify.aviator.co) is built for exactly that. The [setup docs](https://docs.aviator.co/verify) can get you to your first invariant in about ten minutes. (｡•̀ᴗ-)✧ ## FAQ ### What is AI slop in code? AI slop is AI-generated code that works but ignores how your team builds things, such as skipping a data-access layer or an authentication step. It usually passes tests and looks fine in a diff, which is why it tends to get caught only by experienced reviewers. ### What’s an invariant in code review? An invariant is a team-defined rule that every matching change must respect, such as “HTTP handlers must call authentication middleware before business logic”. In Aviator Verify, invariants are stored in an account-level catalog and are applied automatically to every eligible PR. ### Should I use a linter or an invariant? Use a linter when the rule concerns syntax or a simple code pattern (such as formatting, naming, banned calls). Use an invariant when checking the rule requires understanding what the change does, like data-access layering or authentication order. ### Can Aviator Verify find recurring review comments for me? Yes. The **AI from PR comments** source analyzes your PR history and drafts invariants from recurring feedback. Drafts do not produce verdicts until an admin promotes them.