{"slug": "ai-reviewing-ai-is-not-review", "title": "AI Reviewing AI Is Not Review", "summary": "A developer argues that AI-powered code review agents are counterproductive, citing a 2026 study where 60.2% of agent-reviewed pull requests had low signal ratios and 12 of 13 review agents averaged below 60% signal. The post claims that deploying more agents to handle the surge in agent-authored code fails to address the real problem of review workload exceeding human capacity.", "body_md": "*Originally published at tddbuddy.com.*\n\n*Related reading: Where the Review Point Moved is the direct predecessor; this post argues the industry's response to that shift is doubling down on the wrong surface at higher throughput. What \"Senior\" Means When Typing Is Free and The Test Pyramid Was an Economic Argument name where signal actually lives now.*\n\nThe review agent left fourteen comments on the pull request and none of them were the reason the PR should not have merged.\n\nThat is the shape of the failure. The reviewer that shipped the review was a tool built to catch what a human reviewer no longer had time for. Three of the comments were genuine issues, unused imports, a typo in a log message, a dead branch. Eleven were style opinions, restatements of what the diff already made obvious, or false positives on patterns the codebase had chosen deliberately. The human on the PR spent more time filtering the review than reading the diff. The change that actually needed a second pair of eyes (a renamed field in a shared DTO that had already broken a downstream consumer twice this year) merged without a comment on it from either the human or the machine.\n\nThe industry response to agent-generated pull-request volume has been to deploy more agents. The response is understandable. It is also empirically counterproductive. A 2026 study measured what happens when only a code-review agent reviews an agent-authored PR: 60.2% of closed pull requests sat in the 0 to 30 percent signal-ratio range, and twelve of the thirteen review agents evaluated averaged below a 60% signal ratio. Signal is what a human reviewer needs. The review agent produces less of it per unit of reviewer attention than the diff would have without a bot in the middle.\n\nFour hundred thousand pull requests in two months from a single code-writing agent. One in five reviews on the largest hosting platform now involves an agent. Pickup time on agent-authored PRs is 5.3 times longer than on human PRs.\n\nThose three numbers describe a workflow that no longer fits the shape of human review. A human reviewer at pre-agent volume could open a PR, read the diff, ask a question, and either approve or block within an afternoon. Multiply the volume by an order of magnitude and something has to give. The reviewer skims. The reviewer batches. The reviewer approves without reading. The reviewer leaves the PR open for a week and hopes CI will catch what human attention would have. Every one of those responses is an adaptation to a workload that arrived faster than the review process was designed for.\n\nThat workload is the real problem. It is not going away. The volume of agent-authored code will grow, not shrink, as more teams adopt agent-paired workflows and as the agents themselves get faster. The question the industry is trying to answer is: what does review look like when the volume of change exceeds the throughput of the humans who used to catch its problems.\n\nThe wrong answer is \"another agent.\" The right answer starts by looking at what the extra agent actually produces when it lands in the loop.\n\nDeploy a review agent. Let it triage. Escalate only the non-trivial cases. The pitch reads like it should work because the reader's mental model is a triage nurse: someone who sees a hundred cases, correctly identifies the seventy that are minor, and hands the thirty serious ones to the specialist.\n\nThat model requires the triage agent to have a reliable signal for \"trivial\" that the specialist agrees with. The 2026 study measured that agreement, and the number is not close. 60.2% of closed pull requests the review agent handled alone landed in the bottom third of the signal-ratio distribution. Twelve of thirteen review agents averaged below a 60% signal ratio. In plain terms, most of what the review agent surfaces is noise, and the specialist the triage was supposed to protect either re-reads every triaged PR (defeating the purpose) or accepts a false-negative rate that ships bugs the human would have caught.\n\nThe failure mode is not that the review agent is bad at reading code. It is that \"signal\" for the human reviewer includes properties the review agent has no independent way to check. Vocabulary drift. Cohesion loss. Load-bearing distinctions being flattened. Contract-shape changes that will break downstream. The review agent optimizes for what it can see (the diff in isolation) and misses what it cannot (the concept the diff sits inside). The specialist has to look at every triaged case anyway, because the triage did not measure the property the specialist cares about.\n\nThe pitch survives because it maps onto a familiar workflow. The empirical result is noise amplification instead of signal concentration.\n\nSame training distribution on both sides. Same failure modes on both sides. Same blind spots on both sides.\n\nThat is the structural problem with the review-of-the-review pattern. The coding agent and the reviewing agent were trained on overlapping corpora. They share priors about what \"correct\" code looks like. Where the coding agent made an assumption that the domain expert on the team would have pushed back on, the reviewing agent shares that assumption and passes the diff through. Where the coding agent used a pattern that reads well in isolation but violates a codebase-specific convention, the reviewing agent recognizes the pattern from its training and calls it clean.\n\nA review is a check by a party with different priors and different stakes. A loop is a check by a party with the same priors and no stakes. The distinction matters because selection pressure only works when the selector can produce a signal the thing being selected did not produce. If both agents draw from the same well, running the well through itself does not filter the water. It measures how consistent the well is with itself.\n\nThe mirror problem is not exotic to code review. It is the same mechanism that makes a unit test the agent wrote against code the same agent wrote a certificate of the agent's assumptions rather than a check on them. The [test-pyramid post](https://www.tddbuddy.com/blog/the-test-pyramid-was-an-economic-argument/) named this at the unit-test layer. This post names it at the review layer. The mechanism is identical: authorship without separation cannot verify itself.\n\nA loop is not a review. Selection requires a signal the writer cannot produce.\n\nA test the agent read from the existing suite and had to keep passing is signal. A contract the platform team owns is signal. A scenario tied to a domain concept the team named on a whiteboard is signal. A boilerplate comment from another agent is not.\n\nThe pattern behind each is separation. The test in the existing suite was authored by a different session, on a different day, and the agent editing the code cannot rewrite it without triggering the append-only rule. The contract owned by the platform team lives outside the writable tree. The scenario tied to a whiteboard concept was named by a human whose taste the agent has no way to reproduce. Each produces a check the writer cannot short-circuit by editing the checker.\n\nWhere the review process concentrates on these surfaces, signal per unit of reviewer attention is high. The reviewer reading a diff can ask three questions in ninety seconds. Which named test did the diff touch, and did it stay passing on the same assertion the previous version pinned. Which contract at the seam did the diff move, and did the counterparty verify. Which domain concept did the diff modify, and did the vocabulary stay coherent with the rest of the codebase. Answering those three questions with confidence is a real review. Reading fourteen bot comments is not.\n\nThe tooling can help at the line level. Linters, style checks, obvious dead-code detection: these are useful, and calling them \"review\" is where the language slips. Triage at that layer is fine. Naming triage as review is what produced the noise amplification the study measured.\n\nSignal is a scarce resource. Concentrate it where the writer cannot produce it.\n\nReview the vocabulary drift, the cohesion of the concept, the fitness of the scenario. Let the tooling flag the line-level issues. Do not deploy a second agent to argue with the first one about them.\n\nThat reallocation is what a working post-agent review workflow looks like. The reviewer opens the PR and asks: what concept did this touch, did the test surface for that concept change, and did the vocabulary stay coherent. Three questions, ninety seconds each. If the answers are clean, the diff is approved. If any of them is unclear, the diff gets a comment naming the specific concept-level question the reviewer wants answered. If the diff modifies a load-bearing distinction, the diff is rejected as a domain merge and routed to a domain conversation.\n\n```\n// Concept-level review shape (a checklist a human can run in under two minutes)\n//\n// 1. Concept touched:  Which named domain concept does this PR modify?\n// 2. Test surface:      Did any concept-crossing test change? Did the assertion\n//                       weaken (fewer invariants pinned, looser matcher, deleted case)?\n// 3. Vocabulary:        Are any new nouns being introduced for concepts the\n//                       codebase already names? Any rename that collapses a\n//                       kept-separate distinction?\n```\n\nThe line-level work is real and should still happen. It is tooling's job now. Style checks, unused imports, obvious null-safety violations, formatting: none of those need a human, and none of them need a review agent that spends the human's attention arguing about them. The linter surfaces them, the CI blocks on them, the human never sees them. That is the correct allocation of the line layer.\n\nWhat the human does is the concept-level pass. The signal is high because the questions are ones only a party with independent context can answer. The reviewer is expensive, and the questions the reviewer answers deserve the expense.\n\nSelection pressure requires a signal that cannot be produced by the thing being selected.\n\nThat is the closing observation and the operational rule. A review agent reading an agent-authored PR is a system marking its own homework at higher throughput. Adding more agents to the loop increases the throughput of the marking and does not add a party whose taste is independent of the writer. The signal ratio does not improve. It degrades, because the human reviewer whose attention the loop was supposed to protect now has more bot output to filter.\n\nThe loop improves when a human owns one side. The human reads the concept-level review, checks the vocabulary, verifies the test surface, and rejects or approves on those grounds. Line-level tooling handles the noise. The review agent, if it exists at all, is deployed on the same layer as the linter: a tool that surfaces obvious issues to be gated by CI, not a party that produces comments the human has to read.\n\nA team that runs the loop this way spends less human attention per PR and catches more of the concept-level problems that ship bugs. A team that runs the loop with a review agent in the middle spends more human attention per PR (because the agent's comments have to be filtered) and catches fewer, because the concept-level questions are the ones neither agent is asking.\n\nThe review agent left fourteen comments and none were the reason the PR should not have merged. That was the opening. It is also the shape of the era. The loop that improves is the one whose signal comes from a party the writer cannot become. Deploying another writer as the reviewer is not review. It is a louder loop.\n\nSignal lives where authorship separates. Everything else is noise wearing a lab coat.", "url": "https://wpnews.pro/news/ai-reviewing-ai-is-not-review", "canonical_source": "https://dev.to/tmfrisinger/ai-reviewing-ai-is-not-review-33ik", "published_at": "2026-08-20 15:23:03+00:00", "updated_at": "2026-08-20 15:44:58.535311+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "developer-tools", "ai-products"], "entities": ["tddbuddy.com"], "alternates": {"html": "https://wpnews.pro/news/ai-reviewing-ai-is-not-review", "markdown": "https://wpnews.pro/news/ai-reviewing-ai-is-not-review.md", "text": "https://wpnews.pro/news/ai-reviewing-ai-is-not-review.txt", "jsonld": "https://wpnews.pro/news/ai-reviewing-ai-is-not-review.jsonld"}}