The cheapest routing decision is the one that costs no tokens

Cloudflare cut its AI code reviewer's cost per review from $2.43 to $0.77 and median review time from 4m 09s to 1m 32s, a 68% reduction in inference bill, by using shared prompt-cache prefixes, fewer LLM calls per session, and repeated model/effort A/B tests, the company detailed in a blog post. The post frames those gains as limited to optimization inside a single committed provider, model and effort level, leaving the earlier routing decision unaddressed. terraphim-llm-proxy is positioned as filling that routing layer at zero token cost.

Cloudflare published an excellent post this week: Cutting our inference bill by 68% by being obsessive about optimization https://blog.cloudflare.com/obsessive-inference-optimization/ . Their AI code reviewer went from $2.43 to $0.77 per review, and the median review time fell from 4m 09s to 1m 32s. It is a masterclass in squeezing the token economics of a single provider: shared prompt-cache prefixes, fewer LLM calls per session, and relentless model/effort A/B tests. It is also a study in what optimisation inside the call cannot reach. Every one of their levers operates after the request has already been committed to a provider, a model, and an effort level. The layer before that commitment — which lane does this request deserve? — was answered in their architecture by experiment, not by a decision mechanism. That layer is what terraphim-llm-proxy exists for, and the interesting part is that its answer costs zero tokens.