Stop Paying Full Price For Every LLM Call A developer outlined a set of techniques for cutting LLM API bills by targeting four sources of waste: repeated context, latency-tolerant work, easy questions sent to expensive models, and calls that return nothing. The approach combines prompt caching (stable prefixes first, measuring shared prefix length against the provider's minimum), batch and flex processing for non-interactive work, and a cheap-model-first routing pattern that escalates to a stronger model only when the cheap model's output fails validation. The writeup notes that reasoning tokens bill as output, that failed tool calls and malformed responses are still charged, and that teams should sum token usage by task rather than rely on cost per request. An LLM bill has 2 parts: rate and volume. You can adjust both. But a default request barely changes either part. You pay the list price on repeated context, on work that nobody waits for, on easy questions that a cheap model could answer, and on calls that return nothing. The State of FinOps 2026 https://data.finops.org/ reports that 98% of teams manage AI costs, up from 31% 2 years ago, and lists AI cost visibility as a main challenge. Managing costs doesn't explain them. This article shows which parts of that bill are optional, and how to stop paying them. Per token, output costs more than input at gpt-5.6-terra 's list rates https://developers.openai.com/api/docs/pricing . By volume, input costs more in total, because an agent resends its context every turn. Reasoning tokens are billed as output, so they raise what you pay per answer. Not every question needs them. You pay for some calls and get nothing. A failed tool call, a malformed response, and a blocked scrape are all billed for the tokens that they used. Your invoice lists tokens, not results, so cost per request doesn't show you the waste. Sum the token counts by task instead, using the usage field that your API responses already return. Prompt caching discounts the prefix, the front of your prompt, when you reuse it unchanged. Writes cost more than the normal input rate, and a single read already covers it. A prefix that you never reuse costs you the write and returns nothing, so cache only what repeats. On GPT-5.6, caching is automatic above a minimum prefix length https://developers.openai.com/api/docs/guides/prompt-caching , and below it the discount silently doesn't apply, so measure your shared prefix before you trust it. Because a cache only matches an unchanged prefix, put stable content first and changing content last. If you cache whole responses, expire them when the source data changes. On the same card, batching halves the rate on input and output, with no extra cost for writes and no minimum length. Batch jobs fit work that nobody waits for, not chat or anything that needs low latency. With the flex option you pay the same discounted rate without the async round trip, and you trade latency and availability for it. You may not keep your prefix discount when you move a workload into batch, so check cached tokens on your current traffic before you switch. Routing is cheap to build. A status code and a look at the body catch the clear blocks, and sorting the head of what remains is easy work. On the same card, input on gpt-5.6-luna is 10x cheaper than on gpt-5.6-terra , so send those pages to the cheap model first and escalate the answers that fail a check: python pip install litellm from litellm import completion, model cost CHEAP, STRONG = "gpt-5.6-luna", "gpt-5.6-terra" LABELS = {"content", "block", "captcha", "empty"} GATE = f"Label the page as one of: {', '.join sorted LABELS }. " "Reply with the label only." def classify head : """Cheap model first. Escalate when its answer is not a valid label.""" def ask m : res = completion model=m, messages= {"role": "system", "content": GATE}, {"role": "user", "content": head} return res.choices 0 .message.content.strip .lower if out := ask CHEAP in LABELS: return out, CHEAP if out := ask STRONG in LABELS: return out, STRONG raise ValueError f"neither model returned a label: {out r}" What that costs, priced from LiteLLM's own cost map. No API key needed. The call is small, far under any context limit. Uncached too: the shared prefix is the GATE string, well under the cache minimum. IN TOK, OUT TOK, N = 1200, 150, 1000 page head, label, calls price = lambda m, n: n IN TOK model cost m "input cost per token" + OUT TOK model cost m "output cost per token" flat = price STRONG, N for rate in 0.1, 0.3, 0.9 : total = price CHEAP, N + price STRONG, int N rate print f"escalate {rate: 4.0%} - ${total: 5.2f} per 1K vs ${flat:.2f} flat" The code compares the cascade with the strong model alone: php escalate 10% - $ 0.84 per 1K vs $4.20 flat escalate 30% - $ 1.68 per 1K vs $4.20 flat escalate 90% - $ 4.20 per 1K vs $4.20 flat If you escalate 3 calls in 10, you save 60%, and break-even is at 90%. A 5x gap breaks even at 80%, because break-even is 1 minus 1 divided by the rate gap, when input and output have the same gap. The code charges the same amount for a failed cheap call and for a successful call, so reasoning on the cheap model lowers the break-even point. Each model in a cascade keeps its own cache, so the saving is smaller than the rate difference. Check the label, not the confidence that the model reports. Your numbers may differ because LiteLLM refreshes its cost map on import. A valid label isn't always a correct label, so sample against the strong model before you trust the savings. Agents that read protected sites get blocked, and you pay full price for every blocked call. A better IP is the first thing to try, so we tested one. In September 2026, we sent the same httpx GET to each of 7 targets that we picked for their anti-bot protection, first from an ordinary connection, then from a US exit node on Decodo's residential proxies https://decodo.com/proxies/residential-proxies . We also sent the same 7 targets through Decodo's web scraping solutions https://decodo.com/scraping/web , which render JavaScript. | Target | Plain fetch | + residential IP | Web Scraping API | |---|---|---|---| | zillow.com/homes/for sale/ | 403 | 200, real listings | 200 | | amazon.com/s?k=laptop | 503 | 200, real prices | 200 | | walmart.com/search?q=laptop | 200, bot-check | 200, real prices | 200 | | glassdoor.com/Reviews/index.htm | 403, 147,539 tokens | 403 | 200 | | indeed.com/q-software-engineer-jobs.html | 403 | 403 | 200 | | crunchbase.com/organization/anthropic | 403 | 403 | 200 | | g2.com/products/decodo/reviews | 403 | 403 | 613, refused | | All 7 | 0 usable | 3 usable | 6 usable | Through a residential IP, the first 3 rows returned real listings and prices. Amazon returned a page in 1 of 3 runs with no delay, and in 5 of 5 runs when we spaced them. Space the retries if you run them yourself. The remaining 4 targets refused the same address on 12 of 12 retries that we spaced 6 seconds apart, so blocking depends on more than the IP. 3 of the remaining 4 targets returned pages because the API wasn't blocked, not because it rendered them, since rendering can't produce content from a refusal. Walmart returned a failure that looked like a success. On a plain fetch, it returned HTTP 200 with a bot-check body. A tool that branches on status code reports success and passes that page to the model. The agent gets no signal to retry, so check the body for the data that you want, not just the status code. Each kind of failure costs you something different . A small 403 costs one turn. A blocked page that you keep retrying costs every later turn. A 200 with no content means you never get the answer. A retry looks like one extra call. But a retry costs much more, because each turn resends the conversation so far. That Glassdoor block page is 147,539 tokens of raw HTML. We measured it with o200k base , like every token count here. We fetched it once, but it's billed on all 3 turns that follow: 891,670 input tokens, no answer . With 3 attempts, you send 6x as many tokens as with a single attempt, and you pay 11x as much, because turns 3 and 4 are large enough to be billed at a higher rate: Agent-loop cost for one web-reading step. Every turn resends the whole conversation, so a failed tool result is re-billed on every later turn. Arithmetic over measured payloads, not a billing export. Rates come from LiteLLM's cost map, uncached, which is the ceiling. Input over the tier prices wholly at the long rate, a ceiling. Output is flat. That map refreshes on import, so your figures may differ from these. Token counts throughout measured with o200k base. pip install litellm from litellm import model cost CARD = model cost "gpt-5.6-terra" IN SHORT = CARD "input cost per token" 1e6 $/1M tokens IN LONG = CARD "input cost per token above 272k tokens" 1e6 OUT RATE = CARD "output cost per token" 1e6 TIER = 272 000 named in the rate-card key above, not priced as a value SYSTEM, TASK = 1 500, 16 assumptions: your prompt, your question TOOLS PLAIN, TOOLS DECODO = 48, 301 measured: 1 fetch tool vs TOOLSETS=web CALL, ANSWER = 30, 60 def per 1k base, payload, attempts : ctx, cost, out tok = base, 0.0, 0 for in range attempts : each turn pays its own rate cost += ctx IN SHORT if ctx <= TIER else IN LONG out tok += CALL ctx += CALL + payload the failure stays in context cost += ctx IN SHORT if ctx <= TIER else IN LONG out tok += ANSWER return cost + out tok OUT RATE / 1e6 1000 plain = SYSTEM + TASK + TOOLS PLAIN for label, payload, n in "raw HTML", 147 539, 1 , "raw HTML", 147 539, 3 , "text-extracted", 785, 3 : usd = per 1k plain, payload, n print f"blocked x{n}, {label:<14} ${usd: 8.2f} per 1K, no answer" decodo = SYSTEM + TASK + TOOLS DECODO served = per 1k decodo, 951, 1 markdown, tokenLimit=2000 print f"served x1, markdown ${served: 8.2f} per 1K, one answer" The same arithmetic compares the failing runs with the successful one: bash blocked x1, raw HTML $ 302.47 per 1K, no answer blocked x3, raw HTML $ 3267.09 per 1K, no answer blocked x3, text-extracted $ 24.09 per 1K, no answer served x1, markdown $ 10.31 per 1K, one answer You pay full price for everything the agent keeps . The 136x between the 3-attempt rows comes from the content that stayed in context, not from the block, and extracting the text first https://decodo.com/blog/python-extract-text-from-html closes that gap. Input tokens grow with the square of the attempt count, so if you double your turn limit, you pay roughly 4 times as much. Caching reduces both 3-attempt numbers by roughly half, but it doesn't close the gap. Caching helps the extracted run slightly more, because a context that grows fast has less to reuse. Compaction and tool-result clearing limit the growth, but they act later, so a page that's large on arrival is billed in full on the turn it arrives. Neither one refunds the first fetch, and neither one produces the answer. 3 blocked attempts on raw HTML cost 317x a single served fetch, and they return nothing. Extracting the text first cuts the cost to $24.09, which is still more than a served fetch, and it still returns nothing. So move the blocks and retries out of the agent loop. Decodo's MCP server https://decodo.com/ai/mcp-server gives your agent Decodo's web scraping tools, and it retries outside your context. Your context then contains one response instead of every attempt. A successful fetch costs $10.31 per 1K in gpt-5.6-terra tokens alone, and you also pay Decodo's request pricing https://decodo.com/scraping/web/pricing . 6 of the 7 returned usable pages through the API, measured by calling it directly. The last target returned a structured refusal in 67 tokens, and a failed call is a stable signal, so branch on the failure rather than on the refusal text. Your own targets will have their own success rate, so measure yours before you budget for it. Definitions and results both make the context larger, and this server has a setting for each: In your agent, setup takes one config block https://help.decodo.com/docs/mcp . The MCP server is a client for the Decodo's scraping solutions and takes a credential from the free account, either base64 of user:pass or an API key: { "mcpServers": { "decodo": { "command": "npx", "args": "-y", "@decodo/mcp-server@1.2.4" , "env": { "SCRAPER API TOKEN": "