At 2:47 AM, a payment service starts retrying charges.
Not because someone wrote a bug. Because someone wrote a retry loop, and the retry loop was correct in every way a code reviewer checks: it had backoff, it had a max attempt count, it had a context timeout, it had a nice comment explaining the exponential backoff strategy. It had test coverage. Two humans approved it in under four minutes with the industry-standard incantation:
LGTM 🚀
By 6 AM, 1,100 customers had been charged twice.
The AI didn't write a bug. The AI wrote correct-looking code for the wrong world — one where HTTP timeouts mean "nothing happened" instead of "I have no idea what happened." And nobody caught it, because catching it required knowing something that wasn't in the diff.
That's the whole essay, really. But let's do it properly.
The version of this argument you hear on Twitter is: "nobody writes code anymore, it's all AI, zero human involvement."
That's not true, and I want to be honest about the numbers instead of scaring you with a made-up one.
Here's what's actually on the record:
| Claim | Source | When |
|---|---|---|
| 20–30% of code in Microsoft's repos is "written by software" | Satya Nadella, LlamaCon | Apr 2025 |
| >25% of new code at Google is AI-generated, then engineer-reviewed | Sundar Pichai, earnings call | Late 2024 |
| ~75% of new code at Google is AI-generated and engineer-approved | Reported figure | Apr 2026 |
| 84% of devs use or plan to use AI tools; ~51% use them daily | Stack Overflow Dev Survey | 2025 |
| 95% of all code AI-generated | Kevin Scott's prediction | by 2030 |
Notice the load-bearing words in the Google framing: "and approved by engineers." Humans are still in the loop.
Which is worse, actually. Because "human in the loop" has quietly stopped meaning "a human understood this" and started meaning "a human was present when this happened."
Being present is not the same as reviewing. I have been present at many meetings.
Here is the thing that broke:
Generating code got roughly 10x cheaper. Reviewing code got 0x cheaper.
Reading code is bounded by one resource that has not improved since 1970: a human paying attention for a bounded number of minutes per day. You cannot 10x that. You cannot parallelize it. You cannot hire it away, because the bottleneck isn't headcount, it's comprehension per engineer per hour.
So what happens when you 10x the input to a fixed-throughput system? The queue grows, and then the queue gets managed — not by working faster, but by lowering the standard until the queue fits.
This isn't a hypothesis. The data has been pointing at it for two years:
catch {}
, ?? fallback
, except: pass
) up Read that last bullet again. Refactoring didn't slow down a bit. It functionally stopped. From one line in five to one line in twenty-six.
That's the signature of a codebase nobody is reading. You cannot refactor what you have not read. You can only append to it.
And duplicated blocks aren't a style complaint — cloned code correlates with 15–50% more defects, because every clone is a landmine that only detonates when someone fixes one copy and misses four.
So: the code is being written. The code is being merged. The code is not being understood. The review step still exists as a UI element.
This is the obvious move, and it's the one the industry made. CodeRabbit, Bugbot, Copilot review, claude review
, agent-on-agent PR bots. The pitch writes itself: AI generates faster than humans can review, so use AI to review.
And I want to be fair here, because AI review is genuinely good at a real class of problems:
snake_case
here and camelCase
there)If your baseline is "no review," AI review is a massive upgrade. Use it. Genuinely.
But it fails at exactly the class of problem that causes incidents, and it fails for four structural reasons.
The reviewer and the author were trained on overlapping data, share a similar prior about what "good code" looks like, and often are literally the same model family. When your author and your reviewer draw from the same distribution, the reviewer doesn't catch the author's mistakes — it agrees with them, confidently, in well-formatted markdown.
Two humans disagreeing is a feature of code review. Two instances of the same model agreeing is not review. It's an echo with a checkmark.
Logic behind the fix: if you're going to use AI review, use a different model than the one that wrote it. Decorrelate the errors. It's the same reason you don't have the author approve their own PR.
A reviewer bot sees a diff. Incidents live in the space between files: an invariant held in a different service, a lock ordering established in a module nobody touched, a queue that's at-least-once, a downstream system that doesn't do idempotency.
The 2:47 AM retry loop was flawless within the diff. The bug lived in a sentence in a payment provider's docs.
Models are tuned to be helpful and pleasant. Ask one "is this code good?" and you are, statistically, asking it to say yes with reasons. Ask it "find three ways this breaks in production" and you get a different, much better answer — same model, same diff.
You are not querying a truth oracle. You are sampling from a distribution conditioned on your prompt. Prompt for the failure, not for the verdict.
This is the deep one. A probabilistic generator feeding a probabilistic reviewer is a pipeline with no deterministic quality gate anywhere in it. Stacking a second stochastic layer on a stochastic layer doesn't give you certainty; it gives you two opinions and a false sense of process.
The only things in your pipeline that cannot be sweet-talked are: the type checker, the test suite, the linter, the race detector, the migration checker, the fuzzer, and production. Those are your ground truth. Everything else is a vibe.
Enough theory. Here's what this looks like in a PR. Every one of these is a pattern I've seen ship, in a shape an agent will happily produce, and every one gets approved by a reviewer — human or bot — who is reading the diff instead of the system.
func (c *Client) ChargeCard(ctx context.Context, req ChargeRequest) (*Charge, error) {
var lastErr error
for attempt := 0; attempt < 3; attempt++ {
charge, err := c.post(ctx, "/v1/charges", req)
if err != nil {
lastErr = err
time.Sleep(backoff(attempt))
continue
}
return charge, nil
}
return nil, fmt.Errorf("charge failed after 3 attempts: %w", lastErr)
}
Why it looks fine: bounded attempts, exponential backoff, error wrapping, context threaded through. This is textbook. It would pass a Go style review at most companies.
Why it's a landmine: a network timeout is not a "no." It's a "don't know." The request may have reached the provider, created the charge, and had the response die on the way back. Retrying is not retrying — it's charging again.
Why AI review misses it: the correctness depends on a property of the remote system (does /v1/charges
deduplicate?) which appears nowhere in the diff, nowhere in the repo, and possibly nowhere except a paragraph in a vendor doc.
The fix:
// Idempotency key is derived from the logical charge, NOT the attempt —
// all 3 attempts must carry the same key so the provider dedupes them.
req.IdempotencyKey = idempotencyKeyFor(order.ID, order.Version)
for attempt := 0; attempt < 3; attempt++ {
charge, err := c.post(ctx, "/v1/charges", req)
if err == nil {
return charge, nil
}
// Only retry errors we KNOW left no side effect.
if !errors.Is(err, ErrConnRefused) && !errors.Is(err, ErrDNS) {
return nil, err // timeouts included: outcome unknown, do not retry blind
}
lastErr = err
time.Sleep(backoff(attempt))
}
What actually catches it: a checklist rule — "any retry on a mutating call requires an idempotency key or an explicit comment justifying why not." Deterministic. Greppable. Not a judgment call.
// Before
const orders = await db.order.findMany({
where: { buyerId },
include: { lines: true },
});
// After — "cleaner separation of concerns", said the agent
const orders = await db.order.findMany({ where: { buyerId } });
const ordersWithLines = await Promise.all(
orders.map(async (order) => ({
...order,
lines: await db.orderLine.findMany({ where: { orderId: order.id } }),
}))
);
Why it looks fine: it's more explicit, it's not doing magic ORM joins, and Promise.all
makes it look concurrent and fast. Genuinely reads nicer.
Why it's a landmine: one query became 1 + N. On your seed data, N is 5. On your biggest buyer, N is 4,000, and Promise.all
fires all 4,000 at once, drains the connection pool, and takes down every other request on the box. The refactor is now a self-inflicted DDoS with good variable names.
Why AI review misses it: N+1 is invisible without knowing the cardinality of your data. The diff contains no numbers. The model has no idea if orders
has 3 rows or 3 million.
What actually catches it: a test that asserts query count, not a reviewer's intuition.
test("buyer order list stays O(1) in queries", async () => {
await seedOrders({ buyerId: "b1", count: 50 });
const { queryCount } = await withQueryCounter(() => getBuyerOrders("b1"));
expect(queryCount).toBeLessThanOrEqual(2); // fails loudly at 51
});
Note what happened: I converted "a senior engineer must notice this" into "CI is red." That's the whole game.
Future<void> _loadDashboard() async {
final data = await api.fetchDashboard();
setState(() => _data = data);
}
Why it looks fine: four lines. What could possibly be wrong with four lines.
Why it's a landmine: the user taps back during the 800ms fetch. The widget is disposed. setState
fires on a dead element and throws. In debug you see a red screen; in production you see a Crashlytics graph that nobody can reproduce, because it only happens on slow networks — which is to say, on your actual users' phones, not yours.
The fix is one line, and it's a line agents omit constantly:
Future<void> _loadDashboard() async {
final data = await api.fetchDashboard();
if (!mounted) return; // the widget may be gone; every await is a trapdoor
setState(() => _data = data);
}
Why AI review misses it: it's not a logic error, it's a lifecycle error. The rule "state may not exist after an await" is a property of the framework's runtime, not of the syntax on screen. Ironically, flutter analyze
with use_build_context_synchronously
catches the BuildContext
variant of this instantly. A linter beat the reviewer. That's the theme.
ALTER TABLE orders ADD COLUMN status text NOT NULL DEFAULT 'pending';
CREATE INDEX idx_orders_status ON orders (status);
ALTER TABLE orders ALTER COLUMN legacy_ref SET NOT NULL;
Why it looks fine: three lines of ordinary DDL. It runs in 40ms on your dev database with 200 rows. Tests green.
Why it's a landmine on a 40M-row table:
CREATE INDEX
takes a SHARE
lock and orders
SET NOT NULL
takes ACCESS EXCLUSIVE
and full-scans the table. Your checkout is down The fix:
-- Must run OUTSIDE a transaction block:
CREATE INDEX CONCURRENTLY idx_orders_status ON orders (status);
-- Two-step, neither of which takes a long exclusive lock:
ALTER TABLE orders
ADD CONSTRAINT legacy_ref_not_null CHECK (legacy_ref IS NOT NULL) NOT VALID;
ALTER TABLE orders VALIDATE CONSTRAINT legacy_ref_not_null;
Why AI review misses it: the danger is entirely a function of table size and traffic — runtime facts. The SQL is syntactically perfect. This is my favourite example because it shows the failure mode so cleanly: AI reviews the artifact, incidents come from the environment.
What catches it: a migration linter in CI (squawk
, eugene
, Strong Migrations) that fails the build on unsafe DDL. Deterministic. Again.
[Authorize]
[HttpGet("invoices/{id}")]
public async Task<ActionResult<InvoiceDto>> Get(Guid id)
{
var invoice = await _db.Invoices.FindAsync(id);
if (invoice is null) return NotFound();
return Ok(invoice.ToDto());
}
Why it looks fine: there's an [Authorize]
attribute right there. It handles 404. It returns a DTO instead of the entity. Someone was clearly being careful.
Why it's a landmine: [Authorize]
proves the caller is someone. It doesn't prove they're the right someone. Any logged-in user who can produce an invoice ID can read any invoice in the system. This is IDOR, it's been in the OWASP top 10 forever, and it is the single most common vulnerability I see in agent-written CRUD.
The fix — at the boundary, not the endpoint:
// Every query is tenant-scoped by construction, so no endpoint can forget.
modelBuilder.Entity<Invoice>()
.HasQueryFilter(i => i.TenantId == _tenantContext.TenantId);
Why AI review misses it: the model pattern-matches "has [Authorize]
→ auth handled." The missing check is an absence, and absences are invisible in a diff. You cannot see a line that isn't there.
What catches it: a cross-tenant test that exists once and runs forever.
[Fact]
public async Task User_from_tenant_A_cannot_read_tenant_B_invoice()
{
var invoice = await SeedInvoice(tenant: "B");
var response = await ClientAs(tenant: "A").GetAsync($"/invoices/{invoice.Id}");
Assert.Equal(HttpStatusCode.NotFound, response.StatusCode);
}
Go back and look at what actually caught each one:
| Bug | Caught by |
|---|---|
| Retry double-charge | A written rule about mutating retries |
| N+1 explosion | A query-count assertion |
| Flutter async gap | A linter rule |
| Locking migration | A migration linter in CI |
| IDOR | A cross-tenant test |
Zero of them were caught by a reviewer being smart. All five were caught by something deterministic that a tired human — or an agreeable model — cannot talk their way past at 5:40 PM on a Friday.
That's the actual answer to "who reviews the AI code." Not another AI. Not a heroically attentive senior. Machines that can't be persuaded.
Here's how I think about allocating review effort now. The logic is simple: push everything down the pyramid as far as it will go, because lower layers are cheaper, faster and un-bribable, and the top layer is the one resource that doesn't scale.
┌─────────────────────────┐
│ HUMAN (scarce) │ ← invariants, blast radius,
│ ~20% of review time │ "should this exist at all"
├─────────────────────────┤
│ AI REVIEW (cheap) │ ← consistency, missed callers,
│ catches the boring │ dropped errors, docs drift
├─────────────────────────┤
│ DETERMINISTIC (free) │ ← types, tests, linters,
│ catches the deadly │ migration checks, race detector
└─────────────────────────┘
Most teams have this upside down: humans doing linter work (nit: extra blank line
) while nobody checks blast radius.
Every time a human catches something, ask: "could a check have caught this?" If yes, write the check and never spend a human on it again. Your review capacity is a budget. Stop spending it on whitespace.
None of the above is an argument against agents. I use them daily and I'm not giving them up. It's an argument about where you put the rigor. Here's what actually works.
The single highest-leverage habit. Not "build user notifications" — that's a wish, and you'll spend three rounds discovering what you meant.
## Notification delivery
GOAL: deliver in-app notifications, at-least-once, ordered per user.
INVARIANTS (violating any = bug, no exceptions):
- A notification is never delivered to a user who has muted its channel.
- Delivery is idempotent on (user_id, event_id). Duplicate events = one row.
- No query in this path may be unbounded — every list is paginated, max 100.
- Nothing in this path may block the HTTP response.
OUT OF SCOPE: push/APNs, email digest, i18n.
DONE WHEN: the four invariant tests in notifications_test.go pass,
and p99 of POST /events stays under 80ms at 500 rps.
The logic: an invariant is a thing you can test and a thing you can review against. "Build notifications" gives a reviewer nothing to check. The list above turns review from "does this look right?" (unanswerable) into "does this violate line 3?" (answerable). It's also the only reliable way to hand the same task to a review agent later — you have a contract to check against.
Set a number — mine is ~400 lines — and when an agent blows past it, don't read it more carefully. Throw it away and re-prompt with a smaller scope.
The logic: review quality doesn't degrade linearly with diff size, it falls off a cliff. Past a few hundred lines humans switch from reading to scanning, and scanning catches typos, not race conditions. A 2,000-line agent PR isn't 5x a 400-line PR; it's an unreviewed PR with extra steps. The constraint that used to be enforced by typing speed now has to be enforced on purpose.
Let the agent write the implementation. Let it write test scaffolding. But the assertions — the actual expect(...)
lines encoding what must be true — you write, or you at least read every single one with real attention.
The logic: if the same process writes the code and defines correctness, your test suite doesn't verify behavior, it photographs it. Tests generated from an implementation pass by construction, including on the bugs. I have watched an agent "fix" a failing test by changing the expected value. It was very pleased with itself.
Different session, different context, ideally a different model. Claude Code wrote it? Have Codex review it, or vice versa. And prompt adversarially:
You are reviewing a diff written by another AI agent. Assume it compiles and
the tests pass — that tells you nothing. Your job is not to praise it.
Spec and invariants: [paste them]
Output exactly these sections:
1. INVARIANTS — for each invariant above, state whether this diff can break
it, and cite the exact line. "Cannot break it" requires a reason.
2. FAILURE MODES — three concrete production scenarios where this behaves
wrong. Assume 500 rps, 40M rows, 300ms p99 network, concurrent writers,
and one node dying mid-request.
3. SILENT FAILURES — every place this swallows an error, catches broadly,
falls back to a default, or logs-and-continues.
4. BLAST RADIUS — what data can this delete, overwrite, or expose to the
wrong user?
5. If you find nothing real, output "NO FINDINGS" and state what evidence
would have changed your mind.
Do not comment on style. Do not summarize the diff back to me.
The logic: every clause is doing work. "Assume tests pass" removes the easiest false comfort. "Assume 500 rps / 40M rows" injects the environment the diff doesn't contain — that's Examples 2 and 4 solved by prompt. "Cite the exact line" makes hallucinated findings obvious. "NO FINDINGS + what would change your mind" gives it a dignified exit so it doesn't invent problems to please you. And "do not summarize the diff" stops it burning its output budget describing code you can already read.
CLAUDE.md
, AGENTS.md
, whatever your tool reads — this is where you stop re-explaining yourself.
## Non-negotiables
- Every DB query is tenant-scoped. No exceptions. See TenantContext.
- No `catch {}` or `except: pass`. Errors propagate or are handled explicitly.
- Migrations: CREATE INDEX CONCURRENTLY only. Never SET NOT NULL directly.
- Any retry on a mutating call needs an idempotency key.
- Money is int64 minor units. Never float. Never decimal-as-string.
## Known traps in this repo
- `LegacyOrderService` is NOT thread-safe. Don't call it from goroutines.
- `users.deleted_at` is soft-delete. Filtering it out is not optional.
- The reporting replica lags ~30s. Never read-after-write against it.
## Before you finish
Run `make verify`. If it's not green, you're not done.
The logic: every line here is a bug you're now permanently immune to, and every bug you've had to explain twice belongs in this file. It also doubles as onboarding docs for humans, which is the first honestly-good side effect AI has had on documentation.
verify:
go vet ./... && golangci-lint run
go test -race ./...
squawk migrations/*.sql # unsafe DDL fails the build
npx tsc --noEmit
flutter analyze
semgrep --config=auto --error
One command. The agent runs it. CI runs it. Nothing merges without it.
The logic: this is the only layer in the entire pipeline that is not probabilistic. An agent can argue with your code review. It cannot argue with exit 1
. Every rule you move from "I'll catch it in review" to "make verify
catches it" is a permanent, compounding upgrade — and it's the only kind of quality that survives you being tired.
Stop asking "explain this code." It will explain it beautifully and you will learn nothing, because the explanation is generated from the same wrong premise as the code.
Ask instead:
That last one is unreasonably effective. It reframes the model from "defend this code" to "predict its obituary," and models are much better at the second task.
Not top to bottom, not alphabetically by filename. Start at the data model, then the boundary (API/auth), then business logic, then presentation.
The logic: wrong data model poisons everything above it, and you'll never spot it while you're deep in a widget file. Also, GitHub's alphabetical file ordering is not a reasoning order — it's an accident you've been treating as a reading plan for a decade.
Write it down, put it in the repo, argue about it as a team. Mine:
The logic: this list isn't "AI is bad at these." It's "the cost of being wrong here is unbounded and un-rollbackable." Automate by blast radius, not by difficulty. An agent refactoring a settings screen is a great trade. An agent silently changing a transaction isolation level is a wager you didn't know you placed.
Honestly, the differences matter less than the discipline around them — but briefly, and with the caveat that this space changes monthly, so check the docs rather than trusting a blog post:
CLAUDE.md
plus hooks: you can make your rules The meta-point: the tool that will improve your codebase most is the one whose output you actually read. If you switch to a faster agent and your review process doesn't change, you have not bought speed. You've bought debt on a shorter maturity.
Fair's fair, and I'd rather flag this than have you find it in the comments:
The models keep getting better. Every specific failure above is a moving target — a model that reliably reasons about idempotency, table locks and tenant scoping isn't unimaginable, and some of my Example 3s are already caught by better-tuned tooling.
Some of the "quality decline" data is contested. GitClear measures structural proxies (duplication, churn, moved lines), not defects in production. Reasonable people argue that some duplication is fine, that AI-era code is supposed to be more disposable, and that measuring 2026 codebases with 2015 maintainability instincts is a category error. GitClear's own 2026 data notes some of these signals flattening. I find the trend convincing; I don't think it's settled.
Nostalgia is not evidence. Human review was never the golden standard we remember. Plenty of pre-AI PRs got "LGTM" in ninety seconds from someone who'd opened the diff and immediately opened Slack. AI didn't invent the rubber stamp — it just industrialized it.
What I'm confident about is narrower and, I think, harder to argue with: generation scaled, verification didn't, and the gap between them is where your next incident lives. Whether that gap gets closed by better models or better process, it does have to get closed.
The checklist, if you take nothing else:
AGENTS.md
holding every rule you've explained twicemake verify
that gates everythingThe job title didn't change. It just stopped being author and became editor-in-chief, and nobody sent the memo. Editors don't write less carefully than authors — they read more carefully than anyone.
Ship fast. Read the diff. 🫡
What's the worst AI-generated code that made it through review on your team? I want the war stories in the comments — especially the ones where the tests were green.