The October 2026 SlopCodeBench update — published this week and currently sitting at 610 points on Hacker News — contains a finding that should reset your AI coding agent defaults: enabling XHigh reasoning effort on Claude Opus 5.5 costs 4.22x more and produces fewer correct results than Medium. Specifically, Opus 5.5 drops from 28 strict checkpoint passes on Medium to 25 on XHigh, across 196 checkpoints on the SlopCodeBench leaderboard. GPT-6 SOL shows the same regression: XHigh costs 2.11x more and yields 4 fewer strict passes (26 vs. 30). One model did improve with XHigh — GPT-6 Astra, which gained 3 passes at a 2.91x cost premium. The finding is model-specific, but the implication is universal: stop assuming higher reasoning effort is always better.
When XHigh Reasoning Costs More and Delivers Less #
The mechanism behind the regression is what the benchmark team calls “overthinking.” In high-thinking mode, models over-reason about the current checkpoint in isolation. They ignore code they already wrote, reimplement logic from scratch — often incorrectly — and over-engineer solutions that create architectural dead-ends for future requirements. Each checkpoint in SlopCodeBench builds on prior ones, so a poor structural decision at step 2 cascades into failures at steps 4 and 5. The extended reasoning process that helps on isolated problems actively hurts on sequential ones.
According to the HumanLayer benchmark analysis, “higher effort was not a reliable way to buy more strict passes.” The cost data makes this stark: Opus 5.5 XHigh at $817 for 25 passes equals $32.68 per pass, versus $193 for 28 passes on Medium — $6.89 per pass. You are paying nearly 5x more per correct result. For teams running hundreds of agent sessions daily, this math compounds quickly.
Why SlopCodeBench Matters More Than SWE-bench #
SWE-bench tests agents on single-shot bug fixes in existing repositories. SlopCodeBench starts agents with an empty workspace and requires them to build a feature across multiple incremental requirements — the same way developers actually work. Architectural choices at checkpoint 1 propagate through to checkpoint 5. Bad abstractions, copy-paste logic, missing refactors: they all survive and compound. As Gabriel Orlanski, who created the benchmark, put it: “That’s not really how you code, it’s not how you develop software” — referring to one-shot evaluation.
The October results underscore why this distinction matters in practice. Even GPT-6 Astra — the October leader at 16.33% strict — left 1,435 to 2,831 failed test cases in final codebases across 36 problems, with regression failures dominating. No model passed more than 3 of 36 problems at final state under all test categories. The Snorkel AI analysis of code erosion identified consistent failure patterns: selective amnesia (models ignore prior code), library aversion (hand-rolling instead of using packages), and deletion phobia (refusing to remove unnecessary code).
Related: Claude Opus 5.5 Fast Mode Is GA — The Cache Miss That Will Burn Your Budget
The SlopCodeBench Leaderboard and Real Cost Story #
GPT-6 Astra on XHigh leads at 16.33% strict for $758.62. However, GPT-6.1 SOL on Medium achieves 14.80% strict for $178.51 — 1.46x less expensive for 1.5 percentage points less performance. At $6.15 per strict pass versus $23.70 for Astra XHigh, the cost-efficiency gap is significant. Opus 5.5 Medium slots in at $6.89 per pass with 14.29% strict — competitive value for Anthropic users. The worst-value entry in the benchmark: Fable 5.1 XHigh at $1,550 for 13.27% strict, 8.68x more expensive than Astra XHigh for three fewer passes.
There is also a code quality paradox worth noting. Opus 5.5 XHigh produced measurably cleaner code by slop metrics — 77.75% of lines flagged versus 94.25% for Medium — but delivered 3 fewer strict passes. Extended reasoning improved code style at the individual-file level while breaking behavioral correctness across the multi-step sequence. Cleaner code in isolation is not the same as a coherent system across iterations. This is the central irony of XHigh on iterative tasks: the extra reasoning produces better-looking output that works less reliably.
Related: MiniMax M3.1 Flash Tops Agent Benchmarks, Hides the Cost
What Developers Should Do Now #
The October SlopCodeBench results establish a clear default: use Medium reasoning effort unless you have specific evidence that XHigh improves results on your actual workload. For Opus 5.5 users, the evidence now suggests XHigh will likely hurt on iterative feature development tasks. For GPT-6 Astra users, XHigh does help — at roughly 3x the cost — which may justify it for high-stakes or production-blocking work. For everyone else, test before escalating. The original SlopCodeBench paper provides methodology for designing your own evaluation.
Key Takeaways #
- XHigh reasoning effort on Claude Opus 5.5 costs 4.22x more and produces 3 fewer strict passes than Medium — use Medium by default
- SlopCodeBench’s multi-checkpoint methodology exposes code degradation that SWE-bench single-shot testing cannot detect
- GPT-6.1 SOL Medium delivers the best cost-per-pass at $6.15 vs. Astra XHigh at $23.70 — a 3.86x gap for 1.5% less strict performance
- Cleaner code by style metrics does not equal correct code across iterative requirements — test regression failures, not just per-checkpoint pass rates
- All models still fail 1,400+ test cases at final state: AI coding agents remain tools that require human oversight, not autonomous code owners