Doubling PR output is neat. Doubling reviewer count? Not so much. The math just isn’t mathing there. But is it even possible to keep up with volume without increasing the reviewer head count? Good news, it is! The formula is simple, there are no crazy setups, and humans are still kept in the loop (yay, I won’t be replaced, yet).
Let’s see how you can keep the AI-generated code quality high without giving your reviewers a migraine. 😉
TL;DR Keeping AI-generated code up to standard without increasing reviewer count generally comes down to:
- Constraining PRs’ size and letting the agent split them
- Capturing intent as acceptance criteria and making the agent explain its choices
- Codifying repeated review comments into reusable checks
- Verifying behavior against the running code and diffing the tests
- Keeping humans in charge of the decisions while prioritizing reviews by blast radius
- Measuring the review burden
Why Are More Reviewers Not the Solution? #
More code just means more reviewers, right? Well, if reviewers have nothing better to do than read diffs all day, then maybe. But back in the real world, it doesn’t quite work that way.
First of all, AI makes PRs huge and frequent. As a result, every additional reviewer who spends hours on ever-larger diffs eats into the efficiency AI was supposed to bring.
Second, catching AI mistakes requires reconstructing what the code is actually supposed to do. That’s a context problem, which has nothing to do with reviewer capacity. A reviewer who isn’t familiar with a codebase can flag glaring issues but may miss mistakes involving intent or business logic. And that’s usually what AI-generated code gets wrong.
And no, you can’t just hand the overflow to juniors. Junior developers often lack the business context (as they should) and have no awareness of the undocumented shenanigans or deep secrets that senior developers tend to take to their graves.
Meaning of Quality in AI-Made Changes #
Before we can protect code quality, we need a definition of said quality that a machine can consistently check. We can break it down into three questions:
- Intent : Does this change do what was agreed (and nothing outside of that scope)?
- Rules : Does it follow already established rules enforced by your team on each PR (logging, data access, dependencies, auth)?
- Behavior : Does the running code actually behave as expected, including in edge cases?
| Failure mode | Example | Which question catches it |
|---|---|---|
| Plausible but wrong | Happy path works; edge case misread | Intent + behavior |
| Over-engineered | 200-line abstraction for a 15-line need | Intent (scope) |
| Convention-blind | Ignores your structured logger | Rules |
| Confidently hallucinated | Calls an API that doesn’t exist | Behavior |
| Cargo-cult patterns | Retries where it makes no sense | Rules + intent |
What Does an Operating Model With Control Points Look Like? #
Instead of leaving one reviewer at the end to press Approve, you should add smaller control points before a PR reaches the human.
1. Constrain PR Size
Tale as old as time: small PRs get real reviews, large PRs get an LGTM. Ask your agent to ship the work as a stack of small, dependent PRs. Aviator’s free, open-source Stacked PRs CLI (av) handles creating, syncing, and merging stacks.
⚠️ Trade-off: Stacking adds overhead, and some changes (a coherent refactor, for example) really do belong in one PR.
2. Capture Intent as Acceptance Criteria
Write a short scope and a handful of concrete criteria that can be checked. “Endpoint is secure” is too vague, whereas “Returns 401 for a missing or invalid token” is checkable. Aviator’s guide to writing effective acceptance criteria also recommends negative requirements (“Doesn’t modify existing user records”), because they catch bugs that positive ones can miss.
Then make the agent explain itself. Require a PR description that offers reasoning, including what it did not do. No “why” = no review. Send it back.
⚠️ Trade-off: This takes time. In Aviator’s spec-driven experiment, the spec took two days of team time for a medium-sized, full-stack feature.
3. Codify Repeated Review Comments
Pull your team’s last 100 review comments and sort them into three categories: deterministic (a rule can check it), execution-testable (running the code catches it), and genuine judgment. Ankit Jain reports teams generally land at roughly 45% / 30% / 25%, which means about three-quarters of review feedback can be automated.
⚠️ Trade-off: Rules go stale. A rule written for last year’s architecture might start throwing false positives, so someone has to maintain the catalog.
4. Verify Behavior, and Diff the Tests
Run the code against a preview environment instead of trying to deduce how it behaves from the diff alone. Then read the tests with a healthy dose of suspicion. Agents are helpful to a fault. Ask one to make the build pass, and it might change the tests to match the code instead of the other way around.
Removed assertions, skipped tests, broadened mocks, and changed expected values all deserve a second look (I go deeper into this in the audit-ready checklist). Green ≠ correct.
5. Keep Humans on the Decisions
Accountability isn’t something you should automate. Humans should approve the intent, resolve failures, and decide what ships. Divide your attention based on risk. Auth, payments, data handling, concurrency, and anything that’s hard to undo is a priority. A copy tweak or a new log line, on the other hand, can be simply skimmed.
| Control point | What it catches | Cost | Best for |
|---|---|---|---|
| Small, stacked PRs | Reviewer fatigue, hidden changes | Low | Every team |
| Acceptance criteria | Intent drift, scope creep | Medium (writing time) | Changes affecting behavior, security, or money |
| Codified rules | Convention and rule violations | Medium upfront, low ongoing | Teams with repeated review comments |
| Behavioral verification | Hallucinated APIs, broken error paths, weakened tests | Medium to high (preview setup) | User-facing and API changes |
| Human decision | Wrong intent, judgment calls | High per PR, but a smaller surface to review | Final sign-off |
When Does Line-by-Line Review Still Make Sense?
Reading the diff is still mandatory for security-critical code, novel architecture, and onboarding.
How Do You Verify AI-Generated Code Without Reading Every Line? #
Five control points on every PR, forever, is a lot. Some of the judgment should stay yours, but the rest is mechanical checking a machine should be doing for you. (I have tried doing it all by hand. My attention span had opinions. ¯_(ツ)_/¯)
That mechanical half is what Aviator Verify is built for. Instead of pointing another model at the diff and asking it to guess, Verify checks the running code against acceptance criteria a human has approved:
- Build with your agent (Cursor, Claude, Copilot, or whatever you already use).
- Submit the intent. The agent captures the intent and acceptance criteria and submits them to Aviator in a single call from inside the session, through the Aviator Model Context Protocol (MCP) server or command-line interface (CLI).
- Verify routes each criterion to the best-suited method:
- Code-scan for structural facts (the endpoint exists, no new dependency snuck in), checked against the abstract syntax tree (AST);
- Scenarios for behavior, exercised end to end on a preview deployment, with screenshots, request and response bodies, and traces as evidence;
- Invariants for your team’s recurring review comments, encoded once and applied to every matching change.
- A human reviews the intent, a verdict per criterion, and the evidence. They approve, waive with a recorded reason, or ask for another scenario.
In Aviator’s own experiment, an agent checked 65 acceptance criteria against roughly 6,000 lines of AI-generated code in 6 minutes: 60 passed, 4 failed, 1 partial.
But, before we get carried away, it’s important to note that there are some limits:
- Not every criterion resolves deterministically. When code-scan, scenarios, and invariants cannot decide, Verify falls back to a large language model (LLM) check with a confidence threshold, and it keeps that distinction visible in the evidence.
- It only checks what was submitted. A missing criterion goes unchecked, so the criteria list itself still needs a human touch.
- It doesn’t replace****your linters, scanners, tests, or reviewers. Verification and review cover different blind spots, which is why you should run them together.
Where Do You Go From Here? #
Love it or hate it, AI-written code is already in your repo. You don’t need more reviewers, though. You need them to spend their time on the 25% of feedback that requires genuine judgment. Start with small PRs and a codified rule list this week. Both are useful, and they work without any tooling.
When the mechanical stuff gets old, take a look at Aviator Verify and the Verify docs. Wire it up! 🚀
FAQ #
Can AI replace human code review?
No. AI can check rules and behavior, but it cannot be accountable for business context or the decision to ship. A human should still approve intent and resolve failures.
Why not just add more reviewers for AI-generated code?
AI makes PRs larger, so every added reviewer spends more time on bigger diffs. Headcount also doesn’t fix the context problem. Catching AI errors means reconstructing intent, which only reviewers with solid context awareness can do.
What is the difference between AI code review and AI code verification?
AI code review reads the diff, infers intent, and can produce different comments on each run. Verification checks the running code against acceptance criteria agreed upon beforehand and keeps the evidence.
Are AI-written tests enough to verify AI-generated code?
No. If the agent has misunderstood the task, its tests will confirm the misunderstanding, and an agent asked to make the build pass may weaken the tests. Tests catch regressions, and verification against agreed criteria catches intent drift.
Does every PR need acceptance criteria?
No. Reserve them for changes that affect behavior, security, or money. Typos, docs, and trivial config rarely justify the overhead.