{"slug": "cutting-llm-api-costs-prompt-caching-batching-and-smaller-models", "title": "Cutting LLM API Costs: Prompt Caching, Batching and Smaller Models", "summary": "A guide to cutting LLM API costs shows that three levers do most of the work on Anthropic's Claude API: prompt caching, which drops cache reads to $0.10 per million tokens on Claude Sonnet 5.5 versus $2 for normal input, the Batch API's 50% discount, and Claude Haiku 5.5, priced at $0.10 input and $0.50 output per million tokens for prompts up to 100,000 tokens, 20x cheaper than Sonnet 5.5. The post pairs each lever with runnable Python cost-tracking code and a decision flowchart, noting output costs 5x input on every model listed. Anthropic's published pricing and API documentation are the source for the figures, checked 29 Oct 2026.", "body_md": "Your prototype cost a few dollars a day. Then it got real users, and the invoice looked like a typo.\n\nThat's normal. LLM costs scale with tokens, and tokens pile up quickly: long prompts sent again on every request, chatty outputs, and a top-tier model doing work a cheaper one could handle.\n\nThe good news is that three changes do most of the work. **Prompt caching** makes repeated input cheap. **Batching** halves the price of work that can wait. **Smaller models** handle the easy requests for a fraction of the cost.\n\nThis post covers each one with real prices, code that runs, and the mistakes that quietly cancel the savings. The numbers come from Anthropic's pricing and API docs. The ideas carry over to other providers, and there's a short comparison near the end.\n\n## Where the money goes\n\nYou pay per token, and input and output tokens have different prices. Here are the current Claude API rates, per million tokens:\n\n| Model | Input | Output | Batch (in / out) | Cache read | \n|---|---|---|---|---|\n| Claude Fable 5.1 | $10 | $50 | $5 / $25 | $0.25 | \n| Claude Opus 5.5 | $4 | $20 | $2 / $10 | $0.20 | \n| Claude Sonnet 5.5 | $2 | $10 | $1 / $5 | $0.10 | \n| Claude Haiku 5.5 | $0.10 | $0.50 | $0.05 / $0.25 | $0.01 | \n\nHaiku 5.5 prices shown are for prompts up to 100,000 tokens, and cached tokens count toward that length. Longer prompts cost more, at $0.50 in and $2.50 out.\n\nThree things jump out:\n\n- **Output costs 5x input** on every model here. Long answers are expensive.\n- **A cache read costs a fraction of normal input.** For Sonnet 5.5 it's $0.10 instead of $2.\n- **Haiku 5.5 is 20x cheaper than Sonnet 5.5** , on both input and output.\n\nWhich lever matters depends on the shape of your workload. A support bot with a huge instruction prompt and tiny answers is an input problem. A report writer is an output problem.\n\nHere's how to choose:\n\n``` php\nflowchart TD\n    A[\"A workload that costs too much\"] --> B{\"Is a user waiting for the answer?\"}\n    B -->|\"No\"| C[\"Batch it: 50% off\"]\n    B -->|\"Yes\"| D{\"Does every request start with the same big prefix?\"}\n    C --> D\n    D -->|\"Yes\"| E[\"Cache the prefix: reads cost 5 to 10% of normal\"]\n    D -->|\"No\"| F{\"Is the task easy?\"}\n    E --> F\n    F -->|\"Yes\"| G[\"Use a smaller model or lower effort\"]\n    F -->|\"Not sure\"| H[\"Cascade: cheap model first, escalate on failure\"]\n    F -->|\"No\"| I[\"Keep the big model and trim tokens\"]\n```\n\n### Measure before you cut\n\nYou can't fix what you can't see. Every API response reports its token usage, so you can price each request as it happens.\n\nThis small module does that. It also holds the cost model we'll use for the rest of the post:\n\n``` python\nfrom dataclasses import dataclass\n\n@dataclass(frozen=True)\nclass Price:\n    \"\"\"Dollars per million tokens.\"\"\"\n\n    input: float\n    output: float\n    cache_write: float  # 5-minute cache writes\n    cache_read: float\n\n# From Anthropic's pricing page, checked 29 Oct 2026. Prices change, so re-check before you rely on them.\n# Haiku 5.5 shows its rate for prompts up to 100,000 tokens (longer prompts cost more).\nPRICES = {\n    \"claude-opus-5-5\": Price(input=4.00, output=20.00, cache_write=5.00, cache_read=0.20),\n    \"claude-sonnet-5-5\": Price(input=2.00, output=10.00, cache_write=2.50, cache_read=0.10),\n    \"claude-haiku-5-5\": Price(input=0.10, output=0.50, cache_write=0.125, cache_read=0.01),\n}\nBATCH_MULTIPLIER = 0.5  # the Batch API charges 50% of the standard price\n\ndef request_cost(model: str, usage, batch: bool = False) -> float:\n    \"\"\"Dollar cost of one response. `usage` is the usage object from the API response.\"\"\"\n    p = PRICES[model]\n    dollars = (\n        usage.input_tokens * p.input\n        + (usage.cache_creation_input_tokens or 0) * p.cache_write  # these two can be None\n        + (usage.cache_read_input_tokens or 0) * p.cache_read       # when caching isn't used\n        + usage.output_tokens * p.output\n    ) / 1_000_000\n    return dollars * (BATCH_MULTIPLIER if batch else 1)\n\ndef cache_hit_rate(usage) -> float:\n    \"\"\"Share of this request's input tokens that were read from the cache.\"\"\"\n    read = usage.cache_read_input_tokens or 0\n    total = usage.input_tokens + (usage.cache_creation_input_tokens or 0) + read\n    return read / total if total else 0.0\n\n@dataclass(frozen=True)\nclass Workload:\n    requests: int       # per month\n    prefix_tokens: int  # identical on every request: instructions, examples, tool definitions\n    new_tokens: int     # different every time: the user's input\n    output_tokens: int\n\ndef monthly_cost(model: str, w: Workload, cache_hit_share=None, batch: bool = False) -> float:\n    \"\"\"cache_hit_share is the fraction of requests that read the prefix from cache.\n    Leave it as None for no caching. The rest pay the 1.25x cache write.\"\"\"\n    p = PRICES[model]\n    if cache_hit_share is None:\n        input_cost = w.requests * (w.prefix_tokens + w.new_tokens) * p.input\n    else:\n        hits = w.requests * cache_hit_share\n        misses = w.requests - hits\n        input_cost = (\n            hits * w.prefix_tokens * p.cache_read\n            + misses * w.prefix_tokens * p.cache_write\n            + w.requests * w.new_tokens * p.input\n        )\n    output_cost = w.requests * w.output_tokens * p.output\n    return (input_cost + output_cost) / 1_000_000 * (BATCH_MULTIPLIER if batch else 1)\n\ndef cascade_cost(cheap: str, strong: str, w: Workload, escalation_rate: float,\n                 cache_hit_share=None, batch: bool = False) -> float:\n    \"\"\"Every request goes to the cheap model. The share that fails validation is sent to the strong one too.\"\"\"\n    escalated = Workload(round(w.requests * escalation_rate), w.prefix_tokens, w.new_tokens, w.output_tokens)\n    return (monthly_cost(cheap, w, cache_hit_share, batch)\n            + monthly_cost(strong, escalated, cache_hit_share, batch))\n```\n\n`request_cost` turns a response's usage into dollars. `cache_hit_rate` tells you how much of the input came from the cache. `monthly_cost` and `cascade_cost` let you ask \"what if\" before you change anything.\n\nLog these two numbers for every request, and you'll quickly see where your money goes.\n\n## Lever 1: Prompt caching\n\nMost prompts are mostly the same. The system instructions, the examples, the tool definitions and the reference document don't change from one request to the next. Only the user's input does.\n\nPrompt caching lets the API reuse the work it already did on that repeated part. You pay a small premium once to write it, then a fraction of the normal price to read it.\n\n### How it works\n\nThe API caches a **prefix** of your prompt, in this order: tools, then system, then messages. Anything before the breakpoint is cached. Anything after it is processed fresh each time.\n\nSo the rule is simple. Put what stays the same first, and what changes last:\n\n```\nflowchart LR\n    subgraph Stable[\"Stable prefix: cache this\"]\n        T[\"Tool definitions\"] --> S[\"System instructions\"] --> E[\"Examples and reference docs\"]\n    end\n    subgraph Changes[\"Changes on every request\"]\n        U[\"User input, timestamps, per-request data\"]\n    end\n    E -->|\"cache_control goes here\"| U\n```\n\nThe prices work like this, as multipliers on the normal input price:\n\n| Cache operation | Multiplier | Lasts | \n|---|---|---|\n| Write, 5-minute cache | 1.25x | 5 minutes | \n| Write, 1-hour cache | 2x | 1 hour | \n| Read (hit) | 0.1x by default. 0.05x on Opus 5.5 and Sonnet 5.5, and 0.025x on Fable 5.1 | Same as the write | \n\nThe 5-minute cache refreshes for free every time it's used. So steady traffic keeps it warm indefinitely. Anthropic notes that the 5-minute write pays for itself after a single read.\n\n### Turn it on\n\nThe simplest way is automatic caching: add one `cache_control` field at the top level of the request. The system caches up to the last cacheable block and moves the breakpoint forward as a conversation grows.\n\nWhen your prompt has a stable prefix and a changing end, which is the usual case for classifiers and agents, mark the end of the stable part explicitly:\n\n``` python\nimport anthropic\nfrom costs import cache_hit_rate, request_cost\n\nclient = anthropic.Anthropic()\nINSTRUCTIONS = \"...your long, stable instructions and examples...\"\n\nresponse = client.messages.create(\n    model=\"claude-sonnet-5-5\",\n    max_tokens=20,\n    system=[{\"type\": \"text\", \"text\": INSTRUCTIONS, \"cache_control\": {\"type\": \"ephemeral\"}}],\n    messages=[{\"role\": \"user\", \"content\": ticket_text}],\n)\n\nprint(f\"${request_cost('claude-sonnet-5-5', response.usage):.5f}\")\nprint(f\"{cache_hit_rate(response.usage):.0%} of input came from the cache\")\n```\n\nThe first call writes the cache. Calls within the next five minutes read it. Check the usage fields to see which one you got:\n\n- `cache_creation_input_tokens` is what was written\n- `cache_read_input_tokens` is what was read\n- `input_tokens` is only what came after the last breakpoint\n\nAdd all three for your real input total. A common surprise is a small `input_tokens` number that looks wrong but is actually the sign that caching works.\n\n### The traps\n\nMost caching problems are one of these:\n\n- **The prefix is too short.** For the current models the minimum is 512 tokens. Below that, the request still works, but nothing is cached and there's no error. If both cache fields read zero, check the length.\n- **The breakpoint sits on something that changes.** Put`cache_control` on a block holding a timestamp or the user's message, and every request writes a new entry that nothing ever reads. You pay the 1.25x write each time and never get a hit. Put it on the last block that stays identical.\n- **Something in the prefix changed.** Editing tool definitions, adding or removing images, changing`tool_choice` , or changing the thinking or`effort` settings invalidates cached content. Set these once per workload instead of flipping them per request.\n- **Key order isn't stable.** Some languages, Swift and Go among them, shuffle object keys when serializing JSON. If that happens to the keys in your`tool_use` blocks, a different order is a different prefix, and the cache misses.\n- **Parallel first requests.** A cache entry only becomes available once the first response starts. Fire 50 requests at once on a cold cache and none of them can hit it. Send one first, and wait for its response to begin.\n- **The gap is longer than five minutes.** The clock starts when a request starts, not when it finishes. For traffic that comes in bursts with long pauses, use the 1-hour cache at 2x the write price. It pays off after two reads.\n\nAnd one that cuts the other way: caching isn't free if it never hits. In the example later in this post, a cache with no hits makes the bill about 21% higher. It starts to pay once roughly one request in five reads from the cache.\n\n## Lever 2: Batching\n\nIf nobody is waiting for the answer, you shouldn't pay the real-time price.\n\nThe Message Batches API takes a pile of requests and processes them asynchronously. Everything is billed at 50% of the standard price, input and output.\n\n### Where it fits\n\nBatching suits work like:\n\n- Running evals over thousands of test cases\n- Classifying or tagging an archive of old records\n- Nightly summaries and reports\n- Content moderation sweeps\n- Backfills after a prompt change\n\nIt doesn't suit anything interactive. A user staring at a spinner can't wait for a batch.\n\n### The limits\n\n- A batch holds up to 100,000 requests or 256 MB, whichever comes first.\n- Most batches finish within an hour, but the limit is 24 hours. Requests that haven't run by then expire.\n- Requests that error, are canceled or expire aren't billed.\n- Results stay available for 29 days.\n- Results come back in any order, so match them by `custom_id` .\n\nHere's the flow. This is adapted from the examples in Anthropic's docs:\n\n``` python\nimport time\n\nfrom anthropic.types.message_create_params import MessageCreateParamsNonStreaming\nfrom anthropic.types.messages.batch_create_params import Request\n\nSYSTEM = [{\"type\": \"text\", \"text\": INSTRUCTIONS, \"cache_control\": {\"type\": \"ephemeral\", \"ttl\": \"1h\"}}]\n\nbatch = client.messages.batches.create(\n    requests=[\n        Request(\n            custom_id=f\"ticket-{i}\",\n            params=MessageCreateParamsNonStreaming(\n                model=\"claude-haiku-5-5\",\n                max_tokens=10,\n                system=SYSTEM,\n                messages=[{\"role\": \"user\", \"content\": text}],\n            ),\n        )\n        for i, text in enumerate(tickets)\n    ]\n)\n\nwhile client.messages.batches.retrieve(batch.id).processing_status != \"ended\":\n    time.sleep(60)\n\nfor item in client.messages.batches.results(batch.id):\n    if item.result.type == \"succeeded\":\n        print(item.custom_id, item.result.message.content[0].text)\n```\n\nBefore you submit thousands of requests, test one with the normal Messages API. Batch validation errors only show up after the whole batch has been processed.\n\n### Batching and caching together\n\nThe two discounts stack. But there's a catch: batch requests run concurrently and in any order, so cache hits are best effort. Anthropic says typical hit rates range from 30% to 98%.\n\nThree habits raise the odds:\n\n1. Use identical `cache_control` blocks in every request.\n2. Use the 1-hour cache, since batches often take longer than five minutes.\n3. Write the cache first. Submit one request with the shared prefix, wait for it to finish, then submit the rest.\n\n### Not asynchronous, but still cheaper\n\nSome providers also sell a cheaper tier that stays synchronous. Google's Gemini API documents a **Flex** tier at 50% off, served from off-peak capacity. It's worth knowing about when you can tolerate slower or less predictable responses but don't want to rework your code around batch jobs.\n\n## Lever 3: Smaller models\n\nThis is the biggest lever and the riskiest one. If the smaller model does the job, the savings are huge. If it doesn't, you've bought a cheaper way to be wrong.\n\nAnthropic's own guidance is to use Haiku for simple tasks, Sonnet for most production work, and Opus for the hardest reasoning. Haiku 5.5 is described as built for high-volume work like classification, extraction and routing.\n\n### Compare cost per task, not per token\n\nA cheaper token isn't always a cheaper task. Two things move the real number:\n\n- **Tokenizers differ.** Claude 4.7 and later models use a newer tokenizer that produces roughly 30% more tokens for the same text. Count tokens with the specific model you plan to use.\n- **Thinking adds tokens.** A model that reasons at length can spend more on one answer than a smaller model spends on three.\n\n### The effort dial\n\nBefore you switch models, try turning the dial. The `effort` setting controls how many tokens Claude spends on a response, including text, tool calls and thinking:\n\n```\nresponse = client.messages.create(\n    model=\"claude-sonnet-5-5\",\n    max_tokens=1024,\n    output_config={\"effort\": \"low\"},\n    messages=[{\"role\": \"user\", \"content\": prompt}],\n)\n```\n\nThere are five levels: `low`, `medium`, `high`, `xhigh` and `max`. Defaults differ by model. Sonnet 5.5 defaults to high, while Opus 5.5 and Haiku 5.5 default to medium.\n\nAnthropic's advice is to use low and medium liberally wherever your evals show quality holds. Pick one level per workload, because changing it between requests breaks the cache.\n\n### The cascade\n\nYou often can't tell in advance which requests are easy. A cascade lets the data decide. Send everything to the cheap model first, check the answer, and escalate only when the check fails:\n\n``` php\nflowchart LR\n    T[\"Ticket\"] --> H[\"Haiku 5.5\"]\n    H --> V{\"Valid answer?\"}\n    V -->|\"yes\"| Done[\"Use it\"]\n    V -->|\"no\"| S[\"Sonnet 5.5\"]\n    S --> V2{\"Valid answer?\"}\n    V2 -->|\"yes\"| Done\n    V2 -->|\"no\"| Hu[\"Human review\"]\n```\n\nHere's a version for a ticket classifier. The check is deliberately plain. The answer must be one of four labels:\n\n``` python\nfrom dataclasses import dataclass, field\nfrom typing import Callable\n\nLABELS = {\"billing\", \"bug\", \"feature_request\", \"other\"}\n\n@dataclass\nclass Tier:\n    name: str\n    call: Callable[[str], str]\n\n@dataclass\nclass Outcome:\n    answer: str | None\n    tier: str | None  # which tier produced the accepted answer\n    tried: list[str] = field(default_factory=list)\n\ndef is_valid_label(answer: str) -> bool:\n    \"\"\"A cheap, programmatic check. Better than asking the model how sure it is.\"\"\"\n    return answer.strip().lower() in LABELS\n\ndef cascade(task: str, tiers: list[Tier], is_good: Callable[[str], bool] = is_valid_label) -> Outcome:\n    \"\"\"Try the cheapest tier first. Move up when its answer fails the check, or when it errors.\"\"\"\n    tried = []\n    for tier in tiers:\n        tried.append(tier.name)\n        try:\n            answer = tier.call(task)\n        except Exception:\n            continue\n        if is_good(answer):\n            return Outcome(answer.strip().lower(), tier.name, tried)\n    return Outcome(None, None, tried)  # nobody gave a usable answer: queue it for a human\n\ndef escalation_rate(outcomes: list[Outcome]) -> float:\n    \"\"\"Share of tasks that needed more than the first tier. Watch this number.\"\"\"\n    return sum(len(o.tried) > 1 for o in outcomes) / len(outcomes) if outcomes else 0.0\n```\n\nWiring it to real models takes a few lines:\n\n``` python\nimport anthropic\nfrom cascade import Tier, cascade\n\nclient = anthropic.Anthropic()\nSYSTEM = [{\n    \"type\": \"text\",\n    \"text\": \"Classify the ticket as billing, bug, feature_request or other. Reply with the label only.\\n\\n...your examples...\",\n    \"cache_control\": {\"type\": \"ephemeral\"},\n}]\n\ndef ask(model):\n    def call(ticket: str) -> str:\n        response = client.messages.create(\n            model=model,\n            max_tokens=10,\n            system=SYSTEM,\n            messages=[{\"role\": \"user\", \"content\": ticket}],\n        )\n        return response.content[0].text\n    return call\n\ntiers = [Tier(\"haiku\", ask(\"claude-haiku-5-5\")), Tier(\"sonnet\", ask(\"claude-sonnet-5-5\"))]\nprint(cascade(\"I was charged twice this month\", tiers))\n```\n\nThree things to know before you trust it:\n\n- **A valid answer isn't necessarily a correct one.** The check above catches malformed output, not wrong labels. A cheap model can confidently pick the wrong valid label. Measure accuracy on labeled examples before you ship.\n- **Don't ask the model how sure it is.** Self-reported confidence is a weak signal. Prefer checks your code can run: a schema, a regex, a test, a lookup.\n- **Watch the escalation rate.**`escalation_rate` tells you what share of traffic needed the second tier. If it creeps up, your cheap tier is getting worse, or your traffic changed.\n\n## Put it together\n\nHere's a worked example. A ticket classifier handles **1,000,000 requests a month**. Each one has a 6,000-token stable prefix (instructions and examples), 400 tokens of new input, and 150 tokens of output. All prices are from the table above, and all numbers come from `costs.py`:\n\n| Setup | Monthly cost | Saving | \n|---|---|---|\n| A. Sonnet 5.5, no caching | $14,300 | n/a | \n| B. A, plus caching at 95% hits | $3,620 | 75% | \n| C. A, plus batch only | $7,150 | 50% | \n| D. A, plus caching at 70% hits and batch | $3,610 | 75% | \n| E. Haiku 5.5 alone, caching at 95% | $209.50 | 98.5% | \n| F. Cascade: Haiku first, 30% escalate to Sonnet, caching at 95% | $1,295.50 | 91% | \n| G. Cascade, caching at 70% hits, batch | $1,274 | 91% | \n\nA few lessons hide in that table:\n\n- **Caching did most of the work in B.** The prefix is 94% of the input, and output is only about 10% of the baseline bill. For this shape of workload, shortening answers would barely matter.\n- **D barely beats B.** Batch halves the price, but I assumed a lower cache hit rate for batch, as the docs suggest. If batch hit rates reached 95%, D would cost $1,810. A batch that misses the cache gives back a lot of its discount.\n- **E is tempting and dangerous.** It's the cheapest row, but only if Haiku's quality holds on your data. That's what the cascade in F protects.\n- **The cascade is forgiving.** In this model it stays cheaper than caching Sonnet alone until about 94% of requests escalate. At a 60% escalation rate it still saves 83%. Your real constraint is accuracy and latency, not price.\n\nOne caution on F and G: the escalated requests hit the strong model at a lower volume, so its cache may go cold between requests. If traffic is thin, the longer 1-hour cache keeps that second tier warm.\n\n## Smaller wins\n\nOnce the big three are in place, these trim the rest:\n\n- **Cap** `max_tokens` to what the task needs, since output is the expensive side.\n- **Ask for compact output.** A label or a small JSON object costs far less than a paragraph.\n- **Trim the prompt.** Dead instructions and redundant examples are paid for on every call.\n- **Fetch instead of stuffing.** Retrieve the few relevant passages instead of sending the whole document set.\n- **Summarize old turns** in long conversations instead of replaying them in full.\n- **Dedupe identical requests** with an ordinary response cache for exact repeats. That skips the model call entirely.\n- **Set a spend limit** and alerts, so a retry loop can't run up a bill overnight.\n\n## Guardrails\n\nEvery one of these levers can hurt quality if you apply it carelessly.\n\n- **Build a small labeled test set first.** A few hundred real examples is enough to compare setups.\n- **Track cost per correct answer,** not just cost per request. A cheap wrong answer isn't cheap.\n- **Change one thing at a time,** so you know which change moved the number.\n- **Roll out gradually.** Send 5% of traffic through the new path before you flip the rest.\n- **Watch three numbers:** cache hit rate, escalation rate, and the share of work that runs in batch.\n- **Re-check prices every so often.** They change, and so do the cache multipliers.\n\n## Other providers\n\nThe same three levers exist almost everywhere, with different details:\n\n| Provider | Caching | Batch | \n|---|---|---|\n| **Anthropic** | Automatic or explicit. Reads at 0.1x or less. 512-token minimum on current models | 50% off, 24-hour window | \n| **OpenAI** | Automatic for prompts of 1,024 tokens or more. Reuse shows up in `cached_tokens` | 50% off, 24-hour window | \n| **Google Gemini** | Implicit (automatic) and explicit with a TTL. Caching is listed at a 90% discount, plus storage cost | 50% off, plus a 50%-off synchronous Flex tier | \n\nOpenAI's cached-input rate depends on the model, so check its pricing page for the current figure.\n\n## What to try next\n\nStart with measurement. Log `request_cost` and `cache_hit_rate` for a week and see where the money goes. Then pull the lever that matches what you find.\n\n- Mostly input, repeated prefix: caching\n- Mostly work that can wait: batching\n- Mostly easy requests: a smaller model or a cascade\n- Mostly long outputs: lower effort, tighter `max_tokens` , shorter responses\n\n## Check the math\n\nThe cost numbers in this post come from tests with hand-computed expected values. Create `test_costs.py`:\n\n``` python\nimport math\nfrom types import SimpleNamespace\n\nfrom costs import Workload, cache_hit_rate, cascade_cost, monthly_cost, request_cost\n\nSONNET, HAIKU = \"claude-sonnet-5-5\", \"claude-haiku-5-5\"\nTICKETS = Workload(requests=1_000_000, prefix_tokens=6_000, new_tokens=400, output_tokens=150)\n\ndef close(a, b):\n    return math.isclose(a, b, rel_tol=1e-9, abs_tol=1e-12)\n\ndef usage(input_tokens=0, cache_creation=0, cache_read=0, output=0):\n    return SimpleNamespace(input_tokens=input_tokens, cache_creation_input_tokens=cache_creation,\n                           cache_read_input_tokens=cache_read, output_tokens=output)\n\ndef test_one_request_priced_from_its_usage_fields():\n    u = usage(input_tokens=1_000, cache_read=5_000, output=200)\n    # 1,000 x $2 + 5,000 x $0.10 + 200 x $10, per million\n    assert close(request_cost(SONNET, u), 0.0045)\n    assert close(request_cost(SONNET, u, batch=True), 0.00225)\n\ndef test_usage_fields_can_be_none_when_caching_is_off():\n    u = usage(input_tokens=1_000, output=200)\n    u.cache_creation_input_tokens = u.cache_read_input_tokens = None\n    assert close(request_cost(SONNET, u), 0.004)       # 1,000 x $2 + 200 x $10, per million\n    assert cache_hit_rate(u) == 0.0\n\ndef test_cache_hit_rate_from_usage():\n    assert close(cache_hit_rate(usage(input_tokens=400, cache_read=6_000)), 6_000 / 6_400)\n    assert cache_hit_rate(usage()) == 0.0\n\ndef test_baseline_month():\n    assert close(monthly_cost(SONNET, TICKETS), 14_300)      # 12,800 input + 1,500 output\n\ndef test_caching_at_95_percent_hits():\n    assert close(monthly_cost(SONNET, TICKETS, cache_hit_share=0.95), 3_620)\n\ndef test_batch_alone_halves_the_bill():\n    assert close(monthly_cost(SONNET, TICKETS, batch=True), 7_150)\n\ndef test_cache_with_no_hits_costs_more_than_no_cache():\n    assert monthly_cost(SONNET, TICKETS, cache_hit_share=0.0) > monthly_cost(SONNET, TICKETS)\n\ndef test_cache_break_even_is_about_one_hit_in_five():\n    no_cache = monthly_cost(SONNET, TICKETS)\n    assert monthly_cost(SONNET, TICKETS, cache_hit_share=0.25) < no_cache\n    assert monthly_cost(SONNET, TICKETS, cache_hit_share=0.20) > no_cache\n\ndef test_smaller_model_is_far_cheaper_for_the_same_work():\n    assert close(monthly_cost(HAIKU, TICKETS), 715)\n    assert monthly_cost(HAIKU, TICKETS) < monthly_cost(SONNET, TICKETS) / 15\n\ndef test_cascade_edges():\n    assert close(cascade_cost(HAIKU, SONNET, TICKETS, 0.0), monthly_cost(HAIKU, TICKETS))\n    both = monthly_cost(HAIKU, TICKETS) + monthly_cost(SONNET, TICKETS)\n    assert close(cascade_cost(HAIKU, SONNET, TICKETS, 1.0), both)\n\ndef test_cascade_with_caching():\n    # Haiku sees everything ($209.50), Sonnet sees the 30% that failed validation ($1,086)\n    assert close(cascade_cost(HAIKU, SONNET, TICKETS, 0.30, cache_hit_share=0.95), 1_295.5)\n```\n\nAnd `test_cascade.py`:\n\n``` python\nfrom cascade import Tier, cascade, escalation_rate\n\ndef make_tier(name, answer, calls):\n    def call(task):\n        calls.append(name)\n        if isinstance(answer, Exception):\n            raise answer\n        return answer\n    return Tier(name, call)\n\ndef test_cheap_tier_wins_when_its_answer_is_valid():\n    calls = []\n    out = cascade(\"my invoice is wrong\", [make_tier(\"haiku\", \" Billing \", calls), make_tier(\"sonnet\", \"bug\", calls)])\n    assert (out.answer, out.tier, out.tried) == (\"billing\", \"haiku\", [\"haiku\"])\n    assert calls == [\"haiku\"]                         # the expensive model was never called\n\ndef test_invalid_answer_moves_up_a_tier():\n    calls = []\n    out = cascade(\"???\", [make_tier(\"haiku\", \"I think it might be a billing issue\", calls), make_tier(\"sonnet\", \"billing\", calls)])\n    assert (out.answer, out.tier, out.tried) == (\"billing\", \"sonnet\", [\"haiku\", \"sonnet\"])\n\ndef test_an_error_moves_up_a_tier():\n    calls = []\n    out = cascade(\"???\", [make_tier(\"haiku\", TimeoutError(\"slow\"), calls), make_tier(\"sonnet\", \"bug\", calls)])\n    assert out.tier == \"sonnet\" and out.answer == \"bug\"\n\ndef test_nobody_gives_a_usable_answer():\n    calls = []\n    out = cascade(\"???\", [make_tier(\"haiku\", \"dunno\", calls), make_tier(\"sonnet\", \"hmm\", calls)])\n    assert out.answer is None and out.tier is None and out.tried == [\"haiku\", \"sonnet\"]\n\ndef test_escalation_rate():\n    haiku_right = lambda task: \"billing\" if \"invoice\" in task else \"not sure\"\n    tiers = [Tier(\"haiku\", haiku_right), Tier(\"sonnet\", lambda task: \"other\")]\n    tasks = [\"invoice 1\", \"invoice 2\", \"invoice 3\", \"weird thing\"]\n    outcomes = [cascade(t, tiers) for t in tasks]\n    assert escalation_rate(outcomes) == 0.25\n    assert escalation_rate([]) == 0.0\n```\n\nRun both with `pytest -q`. No fixtures or plugins are needed.\n\n## Recap\n\n- Tokens are the bill, and output tokens cost the most.\n- **Caching:** put the stable part first, put the breakpoint at its end, and check the usage fields.\n- **Batching:** half price for anything nobody is waiting on.\n- **Smaller models:** big savings, so protect quality with evals and a cascade.\n- Combine them, but measure each step.\n\n*Prices and limits checked against Anthropic's, OpenAI's and Google's documentation on October 29, 2026. They change often, so confirm them on the pricing pages before you plan a budget.*\n\n**Sources**", "url": "https://wpnews.pro/news/cutting-llm-api-costs-prompt-caching-batching-and-smaller-models", "canonical_source": "https://zyvop.com/cutting-llm-api-costs-prompt-caching-batching-and-smaller-models-3v8xe", "published_at": "2026-10-28 06:48:00+00:00", "updated_at": "2026-10-30 06:47:00.335371+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "ai-infrastructure", "developer-tools"], "entities": ["Anthropic", "Claude Fable 5.1", "Claude Opus 5.5", "Claude Sonnet 5.5", "Claude Haiku 5.5", "Batch API"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/cutting-llm-api-costs-prompt-caching-batching-and-smaller-models", "markdown": "https://wpnews.pro/news/cutting-llm-api-costs-prompt-caching-batching-and-smaller-models.md", "text": "https://wpnews.pro/news/cutting-llm-api-costs-prompt-caching-batching-and-smaller-models.txt", "jsonld": "https://wpnews.pro/news/cutting-llm-api-costs-prompt-caching-batching-and-smaller-models.jsonld"}}