Stop Paying Full Price For Every LLM Call

An LLM bill has 2 parts: rate and volume. You can adjust both. But a default request barely changes either part. You pay the list price on repeated context, on work that nobody waits for, on easy questions that a cheap model could answer, and on calls that return nothing. The State of FinOps 2026 reports that 98% of teams manage AI costs, up from 31% 2 years ago, and lists AI cost visibility as a main challenge. Managing costs doesn't explain them. This article shows which parts of that bill are optional, and how to stop paying them.

Per token, output costs more than input at gpt-5.6-terra's list rates. By volume, input costs more in total, because an agent resends its context every turn. Reasoning tokens are billed as output, so they raise what you pay per answer. Not every question needs them.

You pay for some calls and get nothing. A failed tool call, a malformed response, and a blocked scrape are all billed for the tokens that they used. Your invoice lists tokens, not results, so cost per request doesn't show you the waste. Sum the token counts by task instead, using the usage field that your API responses already return.

Prompt caching discounts the prefix, the front of your prompt, when you reuse it unchanged. Writes cost more than the normal input rate, and a single read already covers it. A prefix that you never reuse costs you the write and returns nothing, so cache only what repeats.

On GPT-5.6, caching is automatic above a minimum prefix length, and below it the discount silently doesn't apply, so measure your shared prefix before you trust it. Because a cache only matches an unchanged prefix, put stable content first and changing content last. If you cache whole responses, expire them when the source data changes.

On the same card, batching halves the rate on input and output, with no extra cost for writes and no minimum length. Batch jobs fit work that nobody waits for, not chat or anything that needs low latency. With the flex option you pay the same discounted rate without the async round trip, and you trade latency and availability for it. You may not keep your prefix discount when you move a workload into batch, so check cached_tokens on your current traffic before you switch.

Routing is cheap to build. A status code and a look at the body catch the clear blocks, and sorting the head of what remains is easy work. On the same card, input on gpt-5.6-luna is 10x cheaper than on gpt-5.6-terra, so send those pages to the cheap model first and escalate the answers that fail a check:

from litellm import completion, model_cost

CHEAP, STRONG = "gpt-5.6-luna", "gpt-5.6-terra"
LABELS = {"content", "block", "captcha", "empty"}
GATE = (f"Label the page as one of: {', '.join(sorted(LABELS))}. "
        "Reply with the label only.")

def classify(head):
    """Cheap model first. Escalate when its answer is not a valid label."""
    def ask(m):
        res = completion(model=m, messages=[
            {"role": "system", "content": GATE},
            {"role": "user", "content": head}])
        return res.choices[0].message.content.strip().lower()
    if (out := ask(CHEAP)) in LABELS:
        return out, CHEAP
    if (out := ask(STRONG)) in LABELS:
        return out, STRONG
    raise ValueError(f"neither model returned a label: {out!r}")

IN_TOK, OUT_TOK, N = 1200, 150, 1000   # page head, label, calls
price = lambda m, n: n * (IN_TOK * model_cost[m]["input_cost_per_token"]
                        + OUT_TOK * model_cost[m]["output_cost_per_token"])

flat = price(STRONG, N)
for rate in (0.1, 0.3, 0.9):
    total = price(CHEAP, N) + price(STRONG, int(N * rate))
    print(f"escalate {rate:>4.0%} -> ${total:>5.2f} per 1K vs ${flat:.2f} flat")

The code compares the cascade with the strong model alone:

escalate  10% -> $ 0.84 per 1K vs $4.20 flat
escalate  30% -> $ 1.68 per 1K vs $4.20 flat
escalate  90% -> $ 4.20 per 1K vs $4.20 flat

If you escalate 3 calls in 10, you save 60%, and break-even is at 90%. A 5x gap breaks even at 80%, because break-even is 1 minus 1 divided by the rate gap, when input and output have the same gap. The code charges the same amount for a failed cheap call and for a successful call, so reasoning on the cheap model lowers the break-even point. Each model in a cascade keeps its own cache, so the saving is smaller than the rate difference.

Check the label, not the confidence that the model reports. Your numbers may differ because LiteLLM refreshes its cost map on import. A valid label isn't always a correct label, so sample against the strong model before you trust the savings.

Agents that read protected sites get blocked, and you pay full price for every blocked call. A better IP is the first thing to try, so we tested one. In September 2026, we sent the same httpx GET to each of 7 targets that we picked for their anti-bot protection, first from an ordinary connection, then from a US exit node on Decodo's residential proxies. We also sent the same 7 targets through Decodo's web scraping solutions, which render JavaScript.

Through a residential IP, the first 3 rows returned real listings and prices. Amazon returned a page in 1 of 3 runs with no delay, and in 5 of 5 runs when we spaced them. Space the retries if you run them yourself. The remaining 4 targets refused the same address on 12 of 12 retries that we spaced 6 seconds apart, so blocking depends on more than the IP.

3 of the remaining 4 targets returned pages because the API wasn't blocked, not because it rendered them, since rendering can't produce content from a refusal.

Walmart returned a failure that looked like a success. On a plain fetch, it returned HTTP 200 with a bot-check body. A tool that branches on status_code reports success and passes that page to the model. The agent gets no signal to retry, so check the body for the data that you want, not just the status code.

Each kind of failure costs you something different. A small 403 costs one turn. A blocked page that you keep retrying costs every later turn. A 200 with no content means you never get the answer.

A retry looks like one extra call. But a retry costs much more, because each turn resends the conversation so far.

That Glassdoor block page is 147,539 tokens of raw HTML. We measured it with o200k_base, like every token count here. We fetched it once, but it's billed on all 3 turns that follow:

891,670 input tokens, no answer.

With 3 attempts, you send 6x as many tokens as with a single attempt, and you pay 11x as much, because turns 3 and 4 are large enough to be billed at a higher rate:

from litellm import model_cost

CARD = model_cost["gpt-5.6-terra"]
IN_SHORT = CARD["input_cost_per_token"] * 1e6                  # $/1M tokens
IN_LONG  = CARD["input_cost_per_token_above_272k_tokens"] * 1e6
OUT_RATE = CARD["output_cost_per_token"] * 1e6
TIER = 272_000        # named in the rate-card key above, not priced as a value
SYSTEM, TASK = 1_500, 16              # assumptions: your prompt, your question
TOOLS_PLAIN, TOOLS_DECODO = 48, 301   # measured: 1 fetch tool vs TOOLSETS=web
CALL, ANSWER = 30, 60

def per_1k(base, payload, attempts):
    ctx, cost, out_tok = base, 0.0, 0
    for _ in range(attempts):
        cost += ctx * (IN_SHORT if ctx <= TIER else IN_LONG)
        out_tok += CALL
        ctx += CALL + payload     # the failure stays in context
    cost += ctx * (IN_SHORT if ctx <= TIER else IN_LONG)
    out_tok += ANSWER
    return (cost + out_tok * OUT_RATE) / 1e6 * 1000

plain = SYSTEM + TASK + TOOLS_PLAIN
for label, payload, n in [("raw HTML", 147_539, 1), ("raw HTML", 147_539, 3),
                          ("text-extracted", 785, 3)]:
    usd = per_1k(plain, payload, n)
    print(f"blocked x{n}, {label:<14} ${usd:>8.2f} per 1K, no answer")

decodo = SYSTEM + TASK + TOOLS_DECODO
served = per_1k(decodo, 951, 1)   # markdown, tokenLimit=2000
print(f"served  x1, markdown       ${served:>8.2f} per 1K, one answer")

The same arithmetic compares the failing runs with the successful one:

blocked x1, raw HTML       $  302.47 per 1K, no answer
blocked x3, raw HTML       $ 3267.09 per 1K, no answer
blocked x3, text-extracted $   24.09 per 1K, no answer
served  x1, markdown       $   10.31 per 1K, one answer

You pay full price for everything the agent keeps. The 136x between the 3-attempt rows comes from the content that stayed in context, not from the block, and extracting the text first closes that gap. Input tokens grow with the square of the attempt count, so if you double your turn limit, you pay roughly 4 times as much.

Caching reduces both 3-attempt numbers by roughly half, but it doesn't close the gap. Caching helps the extracted run slightly more, because a context that grows fast has less to reuse. Compaction and tool-result clearing limit the growth, but they act later, so a page that's large on arrival is billed in full on the turn it arrives. Neither one refunds the first fetch, and neither one produces the answer.

3 blocked attempts on raw HTML cost 317x a single served fetch, and they return nothing. Extracting the text first cuts the cost to $24.09, which is still more than a served fetch, and it still returns nothing. So move the blocks and retries out of the agent loop.

Decodo's MCP server gives your agent Decodo's web scraping tools, and it retries outside your context. Your context then contains one response instead of every attempt. A successful fetch costs $10.31 per 1K in gpt-5.6-terra tokens alone, and you also pay Decodo's request pricing.

6 of the 7 returned usable pages through the API, measured by calling it directly. The last target returned a structured refusal in 67 tokens, and a failed call is a stable signal, so branch on the failure rather than on the refusal text. Your own targets will have their own success rate, so measure yours before you budget for it.

Definitions and results both make the context larger, and this server has a setting for each:

In your agent, setup takes one config block. The MCP server is a client for the Decodo's scraping solutions and takes a credential from the free account, either base64 of user:pass or an API key:

{
  "mcpServers": {
    "decodo": {
      "command": "npx",
      "args": ["-y", "@decodo/mcp-server@1.2.4"],
      "env": {
        "SCRAPER_API_TOKEN": "<basic_auth_token>",
        "TOOLSETS": "web",
        "MAX_RETRIES": "2"
      }
    }
  }
}

That config registers the web toolset alone, here in Claude Desktop:

MAX_RETRIES moves the retry tax. The server makes those attempts, so the failed attempts never reach your context, and 2 is the default on 1.2.4. jsRender gives the headless rendering that the benchmark used. Approve both tools once, or allow them for a batch run.

The call that buys nothing costs you the most, so fix how you fetch data first. The other habits save less, so measure by task, cache, batch, then route. None of them costs much, because caching on this card is automatic above the minimum length, batching is opt-in, and routing takes about an afternoon.

Moving the retries out of your context stops the retry tax, because prompt tuning alone won't fix a loop that re-bills a block page every turn. With tiktoken, count the tokens that your agent's worst URL returns and pass that number to per_1k as the payload, then create a free account, point your existing agent at the server, and count again.