{"slug": "my-checklist-for-reviewing-ai-generated-code", "title": "My Checklist for Reviewing AI-Generated Code", "summary": "A developer outlines a checklist for reviewing AI-generated code, warning that AI code is uniformly fluent and can be confidently wrong, unlike human code where defects cluster where the author struggled. The checklist prioritizes defects by cost of error, with hallucinated APIs and methods as the most critical issue to catch by running the code rather than reading it.", "body_md": "The agent handed me a function that fetched a Stripe customer, read `customer.tax_ids.data[0].value`\n\n, and used it as the VAT number for an invoice. Clean code. Typed. Named well. It read perfectly. It also assumed every customer has exactly one tax ID at index zero, that the array is never empty, and that the first entry is the VAT number rather than, say, an Australian ABN. None of those assumptions hold. The function would work in every test I'd bother to write by eye and break the first time a real customer had two tax IDs or none. I almost merged it, because nothing about reading it told me to stop.\n\nThat is the entire problem with **reviewing AI-generated code**, and it's why I keep a separate checklist for it. When I [decided where to delegate to an agent in the first place](https://iurii.rogulia.fi/blog/when-ai-is-faster-than-a-senior), I closed with a line: how to review what the agent produces is a whole discipline of its own. This is that discipline. The companion to it is [the prompts that prevent bad output before generation](https://iurii.rogulia.fi/blog/ai-agent-codebase-prompts) — that's the upstream half. This is the downstream half: what to look for once the code already exists, when prevention didn't catch everything, because it never does.\n\nHuman code and AI code fail in different places, and that difference is the whole reason a generic review is the wrong tool.\n\nWhen a person writes code, the defects cluster where the person struggled. The awkward function reads awkwardly. The half-understood API gets used hesitantly, with a comment that says \"not sure this is right.\" The reviewer's instinct — slow down where the code looks uncertain — works, because the code's surface correlates with the author's confidence.\n\nAI code has no such tell. It is uniformly fluent. The function that's subtly wrong reads exactly as smoothly as the function that's correct, because fluency is what the model optimizes — it produces the most plausible-looking continuation, and plausible-looking is the entire failure mode. The defect doesn't sit where the prose got awkward. There is no awkward prose. This is the class I call **confidently wrong**: code that is articulate, idiomatic, well-named, and incorrect in a way its own surface will never reveal.\n\nSo the reviewer's normal instinct actively misfires. \"It reads well\" is evidence of nothing. The model is good at exactly the signal you were using as a proxy for correctness. A review tuned for human code — skim the clean parts, slow down at the messy ones — sails straight past the bugs, because for AI code there are no messy parts to slow down at.\n\nThe checklist below is ordered by **cost of error**, not by how often each defect appears. That ordering is the opinion in this article. You will not catch everything — review never does, and some defects only surface under production load — so spend your attention where being wrong is most expensive. A cosmetic naming miss in a one-off script and a missing tenant filter in a billing query are not the same risk, and a checklist that treats them equally wastes the scarce thing, which is your attention.\n\nHighest on the list because it's the most AI-specific defect and the cheapest to catch — if you actually run the code instead of reading it.\n\nModels invent. They produce method names that should exist, config fields that sound right, function arguments in a plausible order, package versions that were never published. The invention is confident and consistent — the model will use the hallucinated helper three times in the same file, which makes it look deliberate.\n\n```\n// The model \"remembered\" a method that doesn't exist.\n// zod has .parse() and .safeParse() — there is no .validate().\nconst result = schema.validate(input);\n\n// Plausible argument order, wrong reality. Stripe's\n// charges.create takes one params object, not positional args.\nawait stripe.charges.create(amount, currency, customer);\n```\n\nThe fix is not \"read more carefully.\" You cannot read your way to catching a hallucinated API, because the whole point is that it looks like a real one. The fix is mechanical: let the **type checker and the compiler** do the reading. A hallucinated method on a typed library fails `tsc`\n\ninstantly. A wrong argument shape fails type-checking. For untyped surfaces — a config key, a CLI flag, an environment variable the model invented — there is no substitute for **actually running the code path**, not eyeballing it and nodding. If the line never executed in front of you, you don't know the API exists. Treat \"I read it and it looked fine\" as equivalent to \"I didn't check.\"\n\nThis is the most dangerous category and the hardest to automate, because the defect is the absence of something the model could not have known to include.\n\nYour domain has rules that live nowhere in public training data. Orders are scoped by `tenant_id`\n\n. Soft-deleted rows must be filtered. A refund can't exceed the captured amount. A user can only see their own organization's records. The model writes a flawless query against your schema and silently omits the invariant, because the invariant exists in your head and your migrations, not in the millions of repositories it learned from.\n\n```\n// The model wrote a correct, well-formed query.\n// It is also a cross-tenant data leak, because it\n// doesn't know this table is multi-tenant.\nconst orders = await db.select().from(ordersTable).where(eq(ordersTable.status, \"paid\"));\n\n// What your domain actually requires:\nconst orders = await db\n  .select()\n  .from(ordersTable)\n  .where(and(eq(ordersTable.tenantId, ctx.tenantId), eq(ordersTable.status, \"paid\")));\n```\n\nThe compiler is no help here — both versions type-check, both run, both return rows. The first one returns the *wrong* rows, and in a demo with one tenant it looks identical to the correct one. This is why I review every data-access path the agent writes against a question the model can't answer for itself: *what is true about this data that isn't written in the code?* Tenant scoping, ownership checks, status guards, monetary bounds. If your invariants live only in tribal knowledge, the agent will violate them every time, and so will every new hire — which is an argument for [writing them into the agent's persistent context](https://iurii.rogulia.fi/blog/ai-agent-codebase-prompts) so the prevention layer catches what it can before the code reaches review.\n\nThe model learned from public code, and a lot of public code is insecure. It reproduces the common patterns it saw, and the common pattern is frequently the unsafe one.\n\n```\n# String interpolation straight into SQL — the model has\n# seen this thousands of times in tutorials and answers.\nquery = f\"SELECT * FROM users WHERE email = '{email}'\"\ncursor.execute(query)\n\n# Parameterized — what you actually want.\ncursor.execute(\"SELECT * FROM users WHERE email = %s\", (email,))\n```\n\nThe recurring offenders, in order of how often I find them: SQL built by string concatenation instead of parameters; missing input validation on anything that crosses a trust boundary; secrets hardcoded or logged in plaintext; insecure defaults (permissive CORS, `verify=False`\n\non TLS, debug mode left on); and IDOR — an endpoint that takes a resource ID from the request and never checks the caller owns it. That last one overlaps with the missing-invariant category, and it's worth flagging twice precisely because it's invisible in a read: the code that fetches `order/:id`\n\nand returns it looks complete. The authorization check that should be there is, again, an *absence*, and absences don't show up when you're reading what's present.\n\nRun new code mentally against [the OWASP Top 10](https://owasp.org/www-project-top-ten/) — not as a compliance ritual, but because the model's training-data priors point at exactly those failure modes.\n\nslug=\"fractional-cto\"\n\ntext=\"Standing up the review discipline for AI-assisted teams — what gets read line by line, what a CI gate rejects automatically, who owns the merge decision — is the kind of structure my fractional CTO engagements put in place early.\"\n\n/>\n\nThe model writes the happy path beautifully and stops there. It was trained on code that demonstrates the intended use, not code that survives the inputs nobody intended.\n\nThe specific gaps, every time: empty collections (the `[0]`\n\naccess from my opening, the `.reduce`\n\nwith no initial value on an empty array); `null`\n\nand `undefined`\n\nwhere the model assumed a value; swallowed exceptions; happy-path-only logic with no else; missing timeouts and retries on external calls; numeric boundaries — zero, negative, overflow, floating-point money.\n\n```\n// The model writes this and considers the task done.\ntry {\n  await chargeCustomer(order);\n} catch {\n  // Silent. The charge failed, the order is marked paid,\n  // and nobody will know until the books don't reconcile.\n}\n```\n\nA swallowed exception is the worst of these because it doesn't just fail to handle the error — it actively hides it, turning a loud, recoverable failure into a silent, expensive one. When I see an empty `catch {}`\n\nor a `catch`\n\nthat logs and returns `null`\n\n, I reject it. Errors propagate or they're handled concretely; there is no third option. The model adds these defensively because it has seen a lot of code that does, and almost all of that code was wrong, too.\n\nThe agent is good at producing green tests. Green is not the same as meaningful, and a test suite that passes while testing nothing is worse than no suite, because it manufactures false confidence.\n\n```\n// This passes. It also tests nothing — the mock returns\n// what the assertion checks for. It's a tautology dressed\n// as a test.\nit(\"returns the user\", async () => {\n  const repo = { findById: vi.fn().mockResolvedValue({ id: 1, name: \"Test\" }) };\n  const result = await getUser(repo, 1);\n  expect(result).toEqual({ id: 1, name: \"Test\" });\n});\n```\n\nThat test verifies that a mock returns what you told the mock to return. The real logic — how `getUser`\n\nhandles a missing user, a repo error, an invalid ID — is untested, and the green checkmark says otherwise. So I review the *tests themselves*, not their presence or their pass/fail. The questions: does this test fail if I break the behavior it claims to cover? Does it mock the exact thing it's supposed to verify? Does it assert on the contract that matters, or on an incidental shape? A test you can't make fail by introducing the bug it's named after is decoration. Asking the model to test code it just wrote tends to produce exactly this — it tests the implementation's assumptions back to itself, bugs included.\n\nThe model writes code that is generically correct and locally foreign. It doesn't know your patterns, so it invents reasonable-looking new ones that quietly fork your codebase.\n\nThe signs: it duplicates a helper that already exists instead of importing it; it introduces a new error-handling style alongside your established one; it logs with `console.log`\n\nwhen you have a structured logger; it picks a different naming or file-placement convention than the surrounding code. None of it is *wrong* in isolation. All of it is drift, and drift compounds — three sessions downstream you have two ways to do everything and no one decided to.\n\nThis is where the prevention layer earns its keep: most of this is catchable before review by [feeding the agent your conventions up front](https://iurii.rogulia.fi/blog/ai-agent-codebase-prompts) and by lint rules that reject the divergence deterministically. What review adds on top is the judgment a linter can't encode — \"this is technically fine but it's not how we do it here.\" When I see a reinvented helper, the fix is one line back to the model: check `lib/`\n\nfor an existing one and use it.\n\nThe model loves to build for a future you didn't ask about. A factory where a function would do. A configuration object with options nobody will set. An abstraction layer wrapping a single concrete call, justified by \"flexibility\" and \"extensibility\" that no requirement demanded.\n\nThis is the inverse failure of the others — not a missing safeguard but a surplus of architecture. It's lower on the list because it's not a correctness bug; it's a maintenance tax. But it's an AI-specific tendency worth naming, because the model has read a lot of \"enterprise-grade\" code and reaches for its ceremonies by default. The reviewer's job here is subtraction. If an abstraction has exactly one implementation and no concrete second use case on the roadmap, it's speculative — inline it. The cost of a premature abstraction is paid by every person who later has to understand the indirection to change behavior that was never variable to begin with.\n\nWhen the agent reaches for a package, three things need checking, and none of them are visible in the diff that adds the import line.\n\nIs the package real and the version published — or did the model hallucinate it (back to category one, now in `package.json`\n\n)? Is it maintained, or an abandoned repo whose last commit was four years ago? Is the license compatible with yours — a GPL dependency pulled into a closed-source product is a legal problem, not a technical one? And what does it drag in transitively — a one-line utility that pulls a hundred packages and three megabytes is rarely worth it when the standard library or six lines of your own would do.\n\nAdding a dependency is a project-level decision the model makes as a per-task convenience. That asymmetry is exactly why the agent shouldn't add packages without a human deciding the trade-off — preferably enforced by a CI check on `package.json`\n\ndiffs rather than left to catch at review.\n\nLast because it's the most context-dependent — what's a problem at scale is invisible at demo size, so this is the category most likely to pass review and surface in production. The model optimizes for \"works,\" not \"works at ten thousand rows.\"\n\nThe classic is the N+1: a loop that runs a query per iteration, correct and fast with three records, a self-inflicted denial of service with three thousand.\n\n```\n// Correct output. One query per order. With 5,000 orders,\n// 5,000 round-trips to the database.\nfor (const order of orders) {\n  const customer = await db.query.customers.findFirst({\n    where: eq(customers.id, order.customerId),\n  });\n  order.customer = customer;\n}\n\n// One query. The model rarely reaches for this on its own.\nconst ids = orders.map((o) => o.customerId);\nconst customers = await db.query.customers.findMany({ where: inArray(customers.id, ids) });\n```\n\nThe other recurring offenders: rendering work inside loops that triggers re-renders per item, building a data structure with the wrong access pattern (a linear scan where a `Map`\n\nlookup belonged), loading a whole table to count it. Catching these requires asking a question the model doesn't ask itself — *what happens to this when the input is a thousand times bigger?* — and that question is a senior reflex, not a thing you read off the page. The code is correct. It just doesn't scale, and correctness is the only thing its surface advertises.\n\nThree, because a checklist sold as a guarantee is a lie.\n\n**It doesn't catch everything.** Review is a filter, not a proof. Race conditions, defects that only appear under concurrent load, the bug that needs production data volume to manifest — these survive any read-through and surface in production regardless of how disciplined the review was. The checklist lowers the rate and the cost of escaped defects. It doesn't drive them to zero, and anyone who tells you their review process does is selling something.\n\n**Reviewing this code can cost more than writing it.** This is the uncomfortable one. When I already hold the full solution in my head, checking the model's version of it line by line — hunting the one subtle deviation from what I intended — can take longer than just typing my own. On those tasks the delegation is a net loss even though generation felt instant. That cost is precisely *why* the checklist matters: if review is the expensive part, it has to be done well, and done well means systematic rather than vibes. Cheap review on cheap-to-verify tasks; expensive, structured review on expensive-to-verify ones. The checklist is for the second kind.\n\n**This is a 2026 snapshot.** Every item here describes a current-model tendency, and the models move. Hallucinated APIs are already less frequent than a year ago. Some categories will shrink; the root ones — missing domain invariants, security priors, the absence of consequence-modeling — depend on things a public-data interpolation engine structurally can't have, so I expect them to age slowest. Re-run the list as the tools change; the priorities will drift, the top of the list less than the bottom. And to be clear: good AI code exists. Most of what the agent writes for me on well-trodden tasks is fine, and paranoia applied uniformly is just slow. The whole point of ordering by cost of error is to *not* review everything as if it were a billing query.\n\nPriority-ordered, by cost of error. Attach it to your PR template for AI-generated changes.\n\n```\n## AI Code Review Checklist\n\nRead order: top items first — they're ordered by cost of error, not frequency.\n\"It reads well\" is not evidence. The model is fluent; fluency is the failure mode.\n\n1.  [ ] Hallucinated APIs — does it type-check AND actually run?\n        (Don't trust untyped configs/flags/env vars by reading.)\n2.  [ ] Domain invariants present — tenant scoping, ownership checks,\n        status guards, monetary bounds. What's true about this data\n        that isn't written in the code?\n3.  [ ] Security — parameterized queries (no string-built SQL),\n        input validated at trust boundaries, no secrets in code/logs,\n        no IDOR (does it check the caller owns the resource?).\n4.  [ ] Edge cases — empty collections, null/undefined, no swallowed\n        catch {}, timeouts + retries on external calls, numeric bounds.\n5.  [ ] Tests assert behavior — does each test FAIL if I break the\n        thing it names? Not testing mocks back to themselves?\n6.  [ ] Conventions — uses existing helpers, your error layer, your\n        logger, your naming. No reinvented patterns.\n7.  [ ] No over-engineering — abstractions with one implementation\n        and no concrete second use case get inlined.\n8.  [ ] Dependencies — package real, maintained, license-compatible,\n        not dragging heavy transitives. Human approved the add.\n9.  [ ] Performance — no per-iteration DB queries (N+1), right data\n        structure, no loading a table to count it. Scales past demo size.\n\nIf review costs more than rewriting would have: rewrite.\n```\n\n", "url": "https://wpnews.pro/news/my-checklist-for-reviewing-ai-generated-code", "canonical_source": "https://dev.to/iurii_rogulia/my-checklist-for-reviewing-ai-generated-code-2iop", "published_at": "2026-07-16 10:01:05+00:00", "updated_at": "2026-07-16 10:04:10.458961+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "ai-agents"], "entities": ["Stripe"], "alternates": {"html": "https://wpnews.pro/news/my-checklist-for-reviewing-ai-generated-code", "markdown": "https://wpnews.pro/news/my-checklist-for-reviewing-ai-generated-code.md", "text": "https://wpnews.pro/news/my-checklist-for-reviewing-ai-generated-code.txt", "jsonld": "https://wpnews.pro/news/my-checklist-for-reviewing-ai-generated-code.jsonld"}}