{"slug": "every-agent-pr-costs-review-time-even-if-it-never-merges", "title": "Every agent PR costs review time even if it never merges", "summary": "A developer argues that review capacity metrics should account for time spent on agent-generated pull requests that never merge, not just submissions that land. Citing Salesforce engineering data showing code volume up roughly 30 percent and review latency climbing, the analysis distinguishes reviewer overload from rational disengagement with unmergeable changes, noting the two demand different fixes: added capacity or upstream gating. A 33-week AMCIS 2026 study of GitHub developers found AI delegation significantly increases exploration tasks, inflating the share of exploratory PRs entering review queues.", "body_md": "Review capacity is usually measured against submissions, but the cost that matters is spent on reviews of changes that never actually land. When agents are producing the code, the gap between those two numbers widens, and the metric most dashboards show hides it.\n\nSalesforce published the clearest version of this pattern I have found. In [Scaling Code Reviews: Adapting to a Surge in AI-Generated Code](https://engineering.salesforce.com/scaling-code-reviews-adapting-to-a-surge-in-ai-generated-code/) (Shan Appajodu and Ravi Boyapati, January 29, 2026), internal signals show code volume rising about 30 percent, pull requests regularly extending past 20 files and 1,000 lines of change, and review latency climbing quarter over quarter. The observation that matters most is buried in the middle: review time on the largest pull requests started to plateau, then decline. At that point reviewers are no longer meaningfully engaging with the change.\n\nThat plateau is usually described as a quality problem, and it partly is. But it is also a cost-accounting problem, and the two should be separated, because they point to different fixes.\n\nEvery pull request that enters a review queue consumes reviewer time. If the change merges, that time bought merged code. If the change is abandoned, revised, superseded, or rejected, the time was still spent. There is no refund on a reviewer's attention.\n\nUnder human-authored flow this mattered less, because abandoned PRs were a smaller share of the total. Agents change the ratio. A coding agent can open many pull requests quickly, and a meaningful share of them are exploratory. A panel study of [GitHub developers over 33 weeks spanning the introduction of the Copilot Coding Agent](https://aisel.aisnet.org/amcis2026/sig_osra/sig_osra/6) (Tang, Zhao, and Karahanna, AMCIS 2026 Proceedings) finds that AI delegation significantly increases developers' engagement in exploration tasks. More exploration means more branches opened, more prototypes proposed, and more changes thrown away before they become permanent. Each of those still shows up in the queue and still asks for judgement before anyone can tell whether it is worth reviewing. Salesforce describes senior reviewers context-switching across multiple large AI-assisted changesets daily, which raises cognitive overhead even when a reviewer only triages a submission.\n\nThe cost per submission is real. The cost per merged change is what the team actually pays to ship. The two diverge exactly when agents inflate the submission count.\n\nA flat or falling review time per large PR is usually reported as a warning, and Salesforce reports it that way. That is the correct first reading. Constant time on a bigger diff means shallower review, and Salesforce ties the trend to the erosion of its second-pair-of-eyes guarantee. When review capacity cannot keep pace with submission volume, vulnerabilities and architectural regressions slip through not because of negligence but because the workflow no longer supports effective reasoning.\n\nThere is a second reading that matters for any team trying to size its review load. If a growing share of large PRs never merge, a reviewer who has learned to recognize the unmergeable ones stops spending time on them. The plateau is then partly rational disengagement with low-value submissions, not only overload. Salesforce's framing puts the weight on overload, which is fair for their system. A team diagnosing its own numbers needs to tell the two causes apart, because they need different fixes. Overload needs more capacity or better triage. Disengagement with unmergeable submissions needs tighter gating upstream, so agents stop producing work the team will throw away.\n\nOnce the submission count inflates, the first question on any PR is whether it deserves a human review at all. That triage decision is itself review labor, and it does not appear as review time on the changes that matter. A reviewer who skims five exploratory agent PRs and rejects them spends time that never shows up against a merged change, because there is no merged change. The denominator is empty for those five, and the cost is attached to nothing.\n\nThis is why submission-based review metrics understate the load so badly. Triage is the fastest-growing portion of the work, and it is the portion most likely to be invisible in per-PR averages. The changes that get reviewed look well-handled, because they are the survivors. The changes that never merge still consumed the triage minutes, and no metric captures them unless the team deliberately counts rejected and abandoned submissions.\n\nThe broader measurement problem is described directly in a survey of practitioners reported in [AI-Era Software Developer Productivity and Performance Metrics](https://aisel.aisnet.org/amcis2026/ai_systdesign/ai_systdesign/17) (Luo and Guo, AMCIS 2026 Proceedings, survey conducted February 2026). The authors find that traditional activity proxies such as lines of code and commits can rise even when end-to-end delivery capacity is constrained by review throughput, quality assurance, and architectural fit. Respondents report widespread reliance on easy-to-instrument volume measures, while expressing only moderate confidence that those measures reflect performance under AI assistance. The survey's conclusion is that AI-era productivity should be assessed as system delivery under verification and coordination constraints, not as individual output volume.\n\nPull request count is exactly this kind of volume proxy, and it moves in the wrong direction just when review becomes the bottleneck. Every agent-driven PR is recorded as output, and as a reason for a review task, before anyone knows whether it will merge. The review-tracking number that says \"n reviews this period\" counts submissions, not delivered value. It reads as progress while the queue builds.\n\nMeasure review time per merged change, not per submitted change. Concretely: total reviewer time in a period divided by the number of changes that reached production in that period. Compare that against the same number from before agents entered the flow.\n\nIf cost per merged change is stable, the plateau is mostly noise from unmergeable submissions. If it is climbing, the review system is absorbing agent-flow inefficiency, and that is a real spend even if no individual review looks too long. If it is falling, at least one of two things is happening: the team is cutting corners somewhere, which is the disengagement signal Salesforce warns about, or it is correctly skipping exploratory submissions that would never merge, which is a win.\n\nNone of these three outcomes is visible from review time per PR alone. That is the point. The number most dashboards show is review time per submission, and it systematically understates the real cost once agents start opening PRs that never land. Track the denominator, because that is the number the team actually feels. The agent that wrote the code has no opinion about the review it creates; the team paying the reviewer time does.", "url": "https://wpnews.pro/news/every-agent-pr-costs-review-time-even-if-it-never-merges", "canonical_source": "https://dev.to/tessainsley/every-agent-pr-costs-review-time-even-if-it-never-merges-2m3n", "published_at": "2026-09-18 01:15:01+00:00", "updated_at": "2026-09-18 01:53:02.092440+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "mlops"], "entities": ["Salesforce", "Shan Appajodu", "Ravi Boyapati", "GitHub", "Copilot Coding Agent", "Tang", "Zhao", "Karahanna"], "alternates": {"html": "https://wpnews.pro/news/every-agent-pr-costs-review-time-even-if-it-never-merges", "markdown": "https://wpnews.pro/news/every-agent-pr-costs-review-time-even-if-it-never-merges.md", "text": "https://wpnews.pro/news/every-agent-pr-costs-review-time-even-if-it-never-merges.txt", "jsonld": "https://wpnews.pro/news/every-agent-pr-costs-review-time-even-if-it-never-merges.jsonld"}}