I hate reading agent-generated code and it's not a good use of my time, and I wanted a staistic to quantify by how much.
My current estimate is $0.243 for a human to review one changed line and $0.114 in model spend for an agent to produce one benchmark-accepted changed line. That makes human review about 2.1 times as expensive under these assumptions.
The agent calculation uses one public GPT-5.5 OpenHands run from May 2026. Its token categories, final patches, per-task costs, and evaluation outcomes come from the same 500 attempts. I then reprice those measured tokens at GPT-5.6 Sol's current rates.
Human review: $0.24 per changed line
A SmartBear/Cisco study analyzed more than 2,500 reviews covering 3.2 million lines of code. It found that defect detection deteriorated as inspection rates rose through the 300-500 LOC/hour range. I used 400 LOC/hour as a midpoint.
The US Bureau of Labor Statistics reports a median software-developer wage of $65.38/hour for May 2025. BLS employer-compensation data for March 2026 says benefits represented 32.7% of total compensation for private-industry professional and related occupations.
Treating $65.38 as the wage component gives:
loaded hourly compensation = $65.38 / (1 - 0.327) = $97.15
review cost per LOC = $97.15 / 400 = $0.243
OpenHands publishes its complete GPT-5.5 SWE-bench Verified run. It used OpenHands v1.18.1 with GPT-5.5 at high reasoning effort and attempted all 500 benchmark tasks.
I counted changed LOC as additions plus deletions in each unified diff, excluding the +++ and --- file-header lines. "Accepted" means the SWE-bench evaluator marked the task resolved.
The run resolved 391 of 500 tasks. One successful result, psf__requests-1142, contained 16,180 changed lines. The next five largest successful patches contained 136, 129, 86, 81, and 79 lines.
That single patch accounts for 76% of all successful changed LOC. Including it would reduce the aggregate cost from $0.114 to $0.027 per LOC, which feels wrong. I therefore treat it as an outlier and removed that entire task from both cost and LOC totals. I report both results below.
After removing it, the dataset contains:
attempted tasks 499
resolved tasks 390
resolve rate 78.2%
accepted changed LOC 5,077
The remaining 499 attempts consumed:
cumulative tokens 651,273,155
GPT-5.6 Sol's promotional prices through November 21, 2026 are $4/M fresh input, $0.40/M cache reads, and $20/M output. Applying those rates:
total spend over 499 attempts $577.48
All failed attempts remain in the $577.48 numerator. Only code from the 390 resolved tasks enters the denominator:
cost per accepted changed LOC = $577.48 / 5,077
= $0.1137
tokens per accepted changed LOC = 651.27M / 5,077
= 128,279
Source: https://static1.smartbear.co/support/media/resources/cc/book/code-review-cisco-case-study.pdf
Sources: https://www.bls.gov/news.release/ocwage.t01.htm https://www.bls.gov/news.release/ecec.t04.htm
Run metadata and archive link: https://github.com/OpenHands/openhands-index-results/tree/main/results/GPT-5.5
SWE-bench Verified source data: https://huggingface.co/datasets/SWE-bench/SWE-bench_Verified
Comments URL: https://news.ycombinator.com/item?id=49456758
Points: 3