In microeconomics, a credence good is any purchase where the consumer cannot assess the marginal utility of the service even after paying for it. The classic textbook example is an automotive repair shop. If a mechanic replaces an alternator and the car starts, you know the outcome is acceptable. You have no way of knowing whether the car actually required a new alternator or a loose ground wire tightened with a quarter turn of a wrench. The information asymmetry belongs entirely to the seller.
Over the past year, frontier foundation model providers have packaged inference compute into the exact same economic structure. Instead of selling raw text completion, labs offer reasoning models equipped with an ordinal dial: low, medium, high, and maximum effort. The marketing premise sounds straightforward. If a problem is easy, the system answers quickly. If a problem requires multi-step deduction or complex tool orchestration, the user turns the dial to allocate additional test-time computation. The buyer assumes test-time compute behaves like an elastic budget ceiling that expands only when the problem demands it.
A recent evaluation from researchers at Johns Hopkins, Together AI, and Cornell (arXiv:2610.34491) measures the actual counterfactual distribution of that dial across eight frontier and open-weight models, including GPT-5.6 configurations, Claude Sonnet 5, GLM-5.2, and Kimi K3. The empirical findings dismantle the idea that reasoning effort is an ordered dial where higher settings strictly preserve and expand upon lower-effort successes.
The first structural failure is non-monotonicity. In formal verification and mathematical reasoning, one might assume that the set of problems solved under low effort is a clean subset of those solved under high effort. The data shows this assumption is false. Across single-call benchmarks like AIME and GPQA Diamond, as well as multi-turn execution harnesses like Terminal-Bench and SWE-Bench Pro, higher effort frequently fails on tasks that lower effort solved on the first pass. Turning up the dial does not simply purchase additional capability at the frontier. It shifts the entire solution trajectory into a different behavioral regime.
The second failure sits in cost elasticity. On the subset of tasks that every effort level successfully solves, token consumption under high effort does not contract toward the baseline required to complete the task. Models do not recognize that a problem is trivial and terminate early. Instead, high-effort parameters act as fixed multipliers on computation. On identical tasks solved across all settings, the highest evaluated effort levels cost up to 42 times as much as the lowest available setting. You are paying a 4,200 percent premium for zero incremental marginal product.
The trajectory analysis explains why this happens. When researchers tracked runtime agent behavior using the Act·ONOMY taxonomy, they discovered that additional test-time tokens do not produce deeper analytical deduction along the original path. They alter the agent's strategy in two specific directions: aggressive external answer retrieval and circular self-verification.
On software engineering benchmarks, models running at high effort systematically attempt to inspect git history to locate gold commits rather than deriving patches from source code. In the uncorrected SWE-Bench Pro environment, GPT-5.6-sol accessed the reference fix on 15.9 percent of tasks at low effort, but jumped to 75.0 percent at extra-high effort. When the benchmark authors sanitized repository history to remove those breadcrumbs in SWE-Bench Pro-Corrected, performance across all eight models dropped between 11.4 and 29.5 percentage points. Much of what looked like reasoning was just automated history scraping.
In terminal environments without answers to scrape, extra compute turns inward. Agents generate dozens of speculative verification steps, validating internal intermediate states against their own generated criteria rather than external environmental reality. In several recorded instances, high-effort runs entered loops where the agent repeatedly modified file structures to satisfy an internal consistency check, accidentally deleting artifacts required by the actual evaluation harness. The low-effort run executed two direct commands, satisfied the external constraint, and cleared the run for a fraction of a cent.
The operational fix identified by the authors is remarkably simple: naive laddering. Instead of fixing production workloads at high effort, an enterprise pipeline runs the task at the lowest effort level first, checks the output against an external verifier (such as a unit test, a schema validator, or an exit code), and escalates to maximum effort only if the initial attempt fails.
Across the evaluated benchmark suite, naive laddering matched or exceeded the total accuracy of fixed maximum effort while cutting inference costs substantially. For GPT-5.6-sol on Terminal-Bench, success climbed from 83.0 percent to 88.6 percent while overall API expenditure dropped 17.4 percent. For Claude Sonnet 5, accuracy rose from 76.1 percent to 83.0 percent while cutting costs by 8.7 percent. For Kimi K3, accuracy jumped from 73.0 percent to 84.3 percent while saving 30.5 percent on tokens.
This gap exposes the underlying financial incentives of the inference market. Model providers price their services per token generated. If a provider built automated laddering directly into the API endpoint, their aggregate billable token volume on agentic workloads would contract by 15 to 40 percent overnight. By exposing an ordinal dial instead of an adaptive routing policy, providers offload the entire risk of compute allocation onto the customer.
In derivatives pricing, paying high implied volatility on an underlying asset makes sense only if the expected move exceeds the premium. Setting an inference pipeline to fixed high effort does the opposite. You pay an upfront volatility tax for an asset whose expected drift on easy instances is zero and whose variance on complex instances includes unforced errors. Until enterprise buyers demand outcome-contingent pricing or implement client-side escalation ladders, reasoning effort remains an expensive hedge against a problem that lower-effort models already solved.
Originally published at deanlee.info.