Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck

A new arXiv study (2608.18931v1) finds that test-time scaling (TTS) methods for language models fail on open-ended tasks because exploitation—selecting the best output from a candidate pool—is the bottleneck, not exploration. In the first compute-normalised comparison of five TTS families across five benchmarks (medicine, law, finance, general chat, creative writing), reward models correlate with true quality at only ρ_v ≈ 0.12, making selection near-random, and only Fusion (synthesis across candidates) consistently improves over single-sample baselines, recovering only ~40% of available quality.

arXiv:2608.18931v1 Announce Type: new Abstract: Test-time scaling TTS improves language model outputs by spending additional inference compute - generating multiple candidates, searching over partial sequences, or iteratively refining drafts. These techniques yield large gains on mathematics and code, but have been developed and stress-tested almost exclusively on tasks where verification is straightforward. We conduct the first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing - grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation. The answer depends on which side of that decomposition you examine. Scaling exploration works: the best candidate in the pool improves steadily with compute across all settings. What breaks is exploitation - the step that converts a rich candidate pool into a final output. With state-of-the-art generators, reward models correlate at only $\rho v \approx 0.12$ with true quality, rendering selection near-random regardless of budget. Tree search amplifies this failure through diversity collapse. Refinement helps on one of five benchmarks; its apparent gains elsewhere are confounded. Only synthesis across candidates Fusion consistently improves over single-sample baselines, yet still recovers only ~40% of available quality. The candidate pool is not the bottleneck - choosing from it is.