Cutting LLM API Costs: Prompt Caching, Batching and Smaller Models

A guide to cutting LLM API costs shows that three levers do most of the work on Anthropic's Claude API: prompt caching, which drops cache reads to $0.10 per million tokens on Claude Sonnet 5.5 versus $2 for normal input, the Batch API's 50% discount, and Claude Haiku 5.5, priced at $0.10 input and $0.50 output per million tokens for prompts up to 100,000 tokens, 20x cheaper than Sonnet 5.5. The post pairs each lever with runnable Python cost-tracking code and a decision flowchart, noting output costs 5x input on every model listed. Anthropic's published pricing and API documentation are the source for the figures, checked 29 Oct 2026.

Your prototype cost a few dollars a day. Then it got real users, and the invoice looked like a typo. That's normal. LLM costs scale with tokens, and tokens pile up quickly: long prompts sent again on every request, chatty outputs, and a top-tier model doing work a cheaper one could handle. The good news is that three changes do most of the work. Prompt caching makes repeated input cheap. Batching halves the price of work that can wait. Smaller models handle the easy requests for a fraction of the cost. This post covers each one with real prices, code that runs, and the mistakes that quietly cancel the savings. The numbers come from Anthropic's pricing and API docs. The ideas carry over to other providers, and there's a short comparison near the end. Where the money goes You pay per token, and input and output tokens have different prices. Here are the current Claude API rates, per million tokens: | Model | Input | Output | Batch in / out | Cache read | |---|---|---|---|---| | Claude Fable 5.1 | $10 | $50 | $5 / $25 | $0.25 | | Claude Opus 5.5 | $4 | $20 | $2 / $10 | $0.20 | | Claude Sonnet 5.5 | $2 | $10 | $1 / $5 | $0.10 | | Claude Haiku 5.5 | $0.10 | $0.50 | $0.05 / $0.25 | $0.01 | Haiku 5.5 prices shown are for prompts up to 100,000 tokens, and cached tokens count toward that length. Longer prompts cost more, at $0.50 in and $2.50 out. Three things jump out: - Output costs 5x input on every model here. Long answers are expensive. - A cache read costs a fraction of normal input. For Sonnet 5.5 it's $0.10 instead of $2. - Haiku 5.5 is 20x cheaper than Sonnet 5.5 , on both input and output. Which lever matters depends on the shape of your workload. A support bot with a huge instruction prompt and tiny answers is an input problem. A report writer is an output problem. Here's how to choose: php flowchart TD A "A workload that costs too much" -- B{"Is a user waiting for the answer?"} B -- |"No"| C "Batch it: 50% off" B -- |"Yes"| D{"Does every request start with the same big prefix?"} C -- D D -- |"Yes"| E "Cache the prefix: reads cost 5 to 10% of normal" D -- |"No"| F{"Is the task easy?"} E -- F F -- |"Yes"| G "Use a smaller model or lower effort" F -- |"Not sure"| H "Cascade: cheap model first, escalate on failure" F -- |"No"| I "Keep the big model and trim tokens" Measure before you cut You can't fix what you can't see. Every API response reports its token usage, so you can price each request as it happens. This small module does that. It also holds the cost model we'll use for the rest of the post: python from dataclasses import dataclass @dataclass frozen=True class Price: """Dollars per million tokens.""" input: float output: float cache write: float 5-minute cache writes cache read: float From Anthropic's pricing page, checked 29 Oct 2026. Prices change, so re-check before you rely on them. Haiku 5.5 shows its rate for prompts up to 100,000 tokens longer prompts cost more . PRICES = { "claude-opus-5-5": Price input=4.00, output=20.00, cache write=5.00, cache read=0.20 , "claude-sonnet-5-5": Price input=2.00, output=10.00, cache write=2.50, cache read=0.10 , "claude-haiku-5-5": Price input=0.10, output=0.50, cache write=0.125, cache read=0.01 , } BATCH MULTIPLIER = 0.5 the Batch API charges 50% of the standard price def request cost model: str, usage, batch: bool = False - float: """Dollar cost of one response. usage is the usage object from the API response.""" p = PRICES model dollars = usage.input tokens p.input + usage.cache creation input tokens or 0 p.cache write these two can be None + usage.cache read input tokens or 0 p.cache read when caching isn't used + usage.output tokens p.output / 1 000 000 return dollars BATCH MULTIPLIER if batch else 1 def cache hit rate usage - float: """Share of this request's input tokens that were read from the cache.""" read = usage.cache read input tokens or 0 total = usage.input tokens + usage.cache creation input tokens or 0 + read return read / total if total else 0.0 @dataclass frozen=True class Workload: requests: int per month prefix tokens: int identical on every request: instructions, examples, tool definitions new tokens: int different every time: the user's input output tokens: int def monthly cost model: str, w: Workload, cache hit share=None, batch: bool = False - float: """cache hit share is the fraction of requests that read the prefix from cache. Leave it as None for no caching. The rest pay the 1.25x cache write.""" p = PRICES model if cache hit share is None: input cost = w.requests w.prefix tokens + w.new tokens p.input else: hits = w.requests cache hit share misses = w.requests - hits input cost = hits w.prefix tokens p.cache read + misses w.prefix tokens p.cache write + w.requests w.new tokens p.input output cost = w.requests w.output tokens p.output return input cost + output cost / 1 000 000 BATCH MULTIPLIER if batch else 1 def cascade cost cheap: str, strong: str, w: Workload, escalation rate: float, cache hit share=None, batch: bool = False - float: """Every request goes to the cheap model. The share that fails validation is sent to the strong one too.""" escalated = Workload round w.requests escalation rate , w.prefix tokens, w.new tokens, w.output tokens return monthly cost cheap, w, cache hit share, batch + monthly cost strong, escalated, cache hit share, batch request cost turns a response's usage into dollars. cache hit rate tells you how much of the input came from the cache. monthly cost and cascade cost let you ask "what if" before you change anything. Log these two numbers for every request, and you'll quickly see where your money goes. Lever 1: Prompt caching Most prompts are mostly the same. The system instructions, the examples, the tool definitions and the reference document don't change from one request to the next. Only the user's input does. Prompt caching lets the API reuse the work it already did on that repeated part. You pay a small premium once to write it, then a fraction of the normal price to read it. How it works The API caches a prefix of your prompt, in this order: tools, then system, then messages. Anything before the breakpoint is cached. Anything after it is processed fresh each time. So the rule is simple. Put what stays the same first, and what changes last: flowchart LR subgraph Stable "Stable prefix: cache this" T "Tool definitions" -- S "System instructions" -- E "Examples and reference docs" end subgraph Changes "Changes on every request" U "User input, timestamps, per-request data" end E -- |"cache control goes here"| U The prices work like this, as multipliers on the normal input price: | Cache operation | Multiplier | Lasts | |---|---|---| | Write, 5-minute cache | 1.25x | 5 minutes | | Write, 1-hour cache | 2x | 1 hour | | Read hit | 0.1x by default. 0.05x on Opus 5.5 and Sonnet 5.5, and 0.025x on Fable 5.1 | Same as the write | The 5-minute cache refreshes for free every time it's used. So steady traffic keeps it warm indefinitely. Anthropic notes that the 5-minute write pays for itself after a single read. Turn it on The simplest way is automatic caching: add one cache control field at the top level of the request. The system caches up to the last cacheable block and moves the breakpoint forward as a conversation grows. When your prompt has a stable prefix and a changing end, which is the usual case for classifiers and agents, mark the end of the stable part explicitly: python import anthropic from costs import cache hit rate, request cost client = anthropic.Anthropic INSTRUCTIONS = "...your long, stable instructions and examples..." response = client.messages.create model="claude-sonnet-5-5", max tokens=20, system= {"type": "text", "text": INSTRUCTIONS, "cache control": {"type": "ephemeral"}} , messages= {"role": "user", "content": ticket text} , print f"${request cost 'claude-sonnet-5-5', response.usage :.5f}" print f"{cache hit rate response.usage :.0%} of input came from the cache" The first call writes the cache. Calls within the next five minutes read it. Check the usage fields to see which one you got: - cache creation input tokens is what was written - cache read input tokens is what was read - input tokens is only what came after the last breakpoint Add all three for your real input total. A common surprise is a small input tokens number that looks wrong but is actually the sign that caching works. The traps Most caching problems are one of these: - The prefix is too short. For the current models the minimum is 512 tokens. Below that, the request still works, but nothing is cached and there's no error. If both cache fields read zero, check the length. - The breakpoint sits on something that changes. Put cache control on a block holding a timestamp or the user's message, and every request writes a new entry that nothing ever reads. You pay the 1.25x write each time and never get a hit. Put it on the last block that stays identical. - Something in the prefix changed. Editing tool definitions, adding or removing images, changing tool choice , or changing the thinking or effort settings invalidates cached content. Set these once per workload instead of flipping them per request. - Key order isn't stable. Some languages, Swift and Go among them, shuffle object keys when serializing JSON. If that happens to the keys in your tool use blocks, a different order is a different prefix, and the cache misses. - Parallel first requests. A cache entry only becomes available once the first response starts. Fire 50 requests at once on a cold cache and none of them can hit it. Send one first, and wait for its response to begin. - The gap is longer than five minutes. The clock starts when a request starts, not when it finishes. For traffic that comes in bursts with long pauses, use the 1-hour cache at 2x the write price. It pays off after two reads. And one that cuts the other way: caching isn't free if it never hits. In the example later in this post, a cache with no hits makes the bill about 21% higher. It starts to pay once roughly one request in five reads from the cache. Lever 2: Batching If nobody is waiting for the answer, you shouldn't pay the real-time price. The Message Batches API takes a pile of requests and processes them asynchronously. Everything is billed at 50% of the standard price, input and output. Where it fits Batching suits work like: - Running evals over thousands of test cases - Classifying or tagging an archive of old records - Nightly summaries and reports - Content moderation sweeps - Backfills after a prompt change It doesn't suit anything interactive. A user staring at a spinner can't wait for a batch. The limits - A batch holds up to 100,000 requests or 256 MB, whichever comes first. - Most batches finish within an hour, but the limit is 24 hours. Requests that haven't run by then expire. - Requests that error, are canceled or expire aren't billed. - Results stay available for 29 days. - Results come back in any order, so match them by custom id . Here's the flow. This is adapted from the examples in Anthropic's docs: python import time from anthropic.types.message create params import MessageCreateParamsNonStreaming from anthropic.types.messages.batch create params import Request SYSTEM = {"type": "text", "text": INSTRUCTIONS, "cache control": {"type": "ephemeral", "ttl": "1h"}} batch = client.messages.batches.create requests= Request custom id=f"ticket-{i}", params=MessageCreateParamsNonStreaming model="claude-haiku-5-5", max tokens=10, system=SYSTEM, messages= {"role": "user", "content": text} , , for i, text in enumerate tickets while client.messages.batches.retrieve batch.id .processing status = "ended": time.sleep 60 for item in client.messages.batches.results batch.id : if item.result.type == "succeeded": print item.custom id, item.result.message.content 0 .text Before you submit thousands of requests, test one with the normal Messages API. Batch validation errors only show up after the whole batch has been processed. Batching and caching together The two discounts stack. But there's a catch: batch requests run concurrently and in any order, so cache hits are best effort. Anthropic says typical hit rates range from 30% to 98%. Three habits raise the odds: 1. Use identical cache control blocks in every request. 2. Use the 1-hour cache, since batches often take longer than five minutes. 3. Write the cache first. Submit one request with the shared prefix, wait for it to finish, then submit the rest. Not asynchronous, but still cheaper Some providers also sell a cheaper tier that stays synchronous. Google's Gemini API documents a Flex tier at 50% off, served from off-peak capacity. It's worth knowing about when you can tolerate slower or less predictable responses but don't want to rework your code around batch jobs. Lever 3: Smaller models This is the biggest lever and the riskiest one. If the smaller model does the job, the savings are huge. If it doesn't, you've bought a cheaper way to be wrong. Anthropic's own guidance is to use Haiku for simple tasks, Sonnet for most production work, and Opus for the hardest reasoning. Haiku 5.5 is described as built for high-volume work like classification, extraction and routing. Compare cost per task, not per token A cheaper token isn't always a cheaper task. Two things move the real number: - Tokenizers differ. Claude 4.7 and later models use a newer tokenizer that produces roughly 30% more tokens for the same text. Count tokens with the specific model you plan to use. - Thinking adds tokens. A model that reasons at length can spend more on one answer than a smaller model spends on three. The effort dial Before you switch models, try turning the dial. The effort setting controls how many tokens Claude spends on a response, including text, tool calls and thinking: response = client.messages.create model="claude-sonnet-5-5", max tokens=1024, output config={"effort": "low"}, messages= {"role": "user", "content": prompt} , There are five levels: low , medium , high , xhigh and max . Defaults differ by model. Sonnet 5.5 defaults to high, while Opus 5.5 and Haiku 5.5 default to medium. Anthropic's advice is to use low and medium liberally wherever your evals show quality holds. Pick one level per workload, because changing it between requests breaks the cache. The cascade You often can't tell in advance which requests are easy. A cascade lets the data decide. Send everything to the cheap model first, check the answer, and escalate only when the check fails: php flowchart LR T "Ticket" -- H "Haiku 5.5" H -- V{"Valid answer?"} V -- |"yes"| Done "Use it" V -- |"no"| S "Sonnet 5.5" S -- V2{"Valid answer?"} V2 -- |"yes"| Done V2 -- |"no"| Hu "Human review" Here's a version for a ticket classifier. The check is deliberately plain. The answer must be one of four labels: python from dataclasses import dataclass, field from typing import Callable LABELS = {"billing", "bug", "feature request", "other"} @dataclass class Tier: name: str call: Callable str , str @dataclass class Outcome: answer: str | None tier: str | None which tier produced the accepted answer tried: list str = field default factory=list def is valid label answer: str - bool: """A cheap, programmatic check. Better than asking the model how sure it is.""" return answer.strip .lower in LABELS def cascade task: str, tiers: list Tier , is good: Callable str , bool = is valid label - Outcome: """Try the cheapest tier first. Move up when its answer fails the check, or when it errors.""" tried = for tier in tiers: tried.append tier.name try: answer = tier.call task except Exception: continue if is good answer : return Outcome answer.strip .lower , tier.name, tried return Outcome None, None, tried nobody gave a usable answer: queue it for a human def escalation rate outcomes: list Outcome - float: """Share of tasks that needed more than the first tier. Watch this number.""" return sum len o.tried 1 for o in outcomes / len outcomes if outcomes else 0.0 Wiring it to real models takes a few lines: python import anthropic from cascade import Tier, cascade client = anthropic.Anthropic SYSTEM = { "type": "text", "text": "Classify the ticket as billing, bug, feature request or other. Reply with the label only.\n\n...your examples...", "cache control": {"type": "ephemeral"}, } def ask model : def call ticket: str - str: response = client.messages.create model=model, max tokens=10, system=SYSTEM, messages= {"role": "user", "content": ticket} , return response.content 0 .text return call tiers = Tier "haiku", ask "claude-haiku-5-5" , Tier "sonnet", ask "claude-sonnet-5-5" print cascade "I was charged twice this month", tiers Three things to know before you trust it: - A valid answer isn't necessarily a correct one. The check above catches malformed output, not wrong labels. A cheap model can confidently pick the wrong valid label. Measure accuracy on labeled examples before you ship. - Don't ask the model how sure it is. Self-reported confidence is a weak signal. Prefer checks your code can run: a schema, a regex, a test, a lookup. - Watch the escalation rate. escalation rate tells you what share of traffic needed the second tier. If it creeps up, your cheap tier is getting worse, or your traffic changed. Put it together Here's a worked example. A ticket classifier handles 1,000,000 requests a month . Each one has a 6,000-token stable prefix instructions and examples , 400 tokens of new input, and 150 tokens of output. All prices are from the table above, and all numbers come from costs.py : | Setup | Monthly cost | Saving | |---|---|---| | A. Sonnet 5.5, no caching | $14,300 | n/a | | B. A, plus caching at 95% hits | $3,620 | 75% | | C. A, plus batch only | $7,150 | 50% | | D. A, plus caching at 70% hits and batch | $3,610 | 75% | | E. Haiku 5.5 alone, caching at 95% | $209.50 | 98.5% | | F. Cascade: Haiku first, 30% escalate to Sonnet, caching at 95% | $1,295.50 | 91% | | G. Cascade, caching at 70% hits, batch | $1,274 | 91% | A few lessons hide in that table: - Caching did most of the work in B. The prefix is 94% of the input, and output is only about 10% of the baseline bill. For this shape of workload, shortening answers would barely matter. - D barely beats B. Batch halves the price, but I assumed a lower cache hit rate for batch, as the docs suggest. If batch hit rates reached 95%, D would cost $1,810. A batch that misses the cache gives back a lot of its discount. - E is tempting and dangerous. It's the cheapest row, but only if Haiku's quality holds on your data. That's what the cascade in F protects. - The cascade is forgiving. In this model it stays cheaper than caching Sonnet alone until about 94% of requests escalate. At a 60% escalation rate it still saves 83%. Your real constraint is accuracy and latency, not price. One caution on F and G: the escalated requests hit the strong model at a lower volume, so its cache may go cold between requests. If traffic is thin, the longer 1-hour cache keeps that second tier warm. Smaller wins Once the big three are in place, these trim the rest: - Cap max tokens to what the task needs, since output is the expensive side. - Ask for compact output. A label or a small JSON object costs far less than a paragraph. - Trim the prompt. Dead instructions and redundant examples are paid for on every call. - Fetch instead of stuffing. Retrieve the few relevant passages instead of sending the whole document set. - Summarize old turns in long conversations instead of replaying them in full. - Dedupe identical requests with an ordinary response cache for exact repeats. That skips the model call entirely. - Set a spend limit and alerts, so a retry loop can't run up a bill overnight. Guardrails Every one of these levers can hurt quality if you apply it carelessly. - Build a small labeled test set first. A few hundred real examples is enough to compare setups. - Track cost per correct answer, not just cost per request. A cheap wrong answer isn't cheap. - Change one thing at a time, so you know which change moved the number. - Roll out gradually. Send 5% of traffic through the new path before you flip the rest. - Watch three numbers: cache hit rate, escalation rate, and the share of work that runs in batch. - Re-check prices every so often. They change, and so do the cache multipliers. Other providers The same three levers exist almost everywhere, with different details: | Provider | Caching | Batch | |---|---|---| | Anthropic | Automatic or explicit. Reads at 0.1x or less. 512-token minimum on current models | 50% off, 24-hour window | | OpenAI | Automatic for prompts of 1,024 tokens or more. Reuse shows up in cached tokens | 50% off, 24-hour window | | Google Gemini | Implicit automatic and explicit with a TTL. Caching is listed at a 90% discount, plus storage cost | 50% off, plus a 50%-off synchronous Flex tier | OpenAI's cached-input rate depends on the model, so check its pricing page for the current figure. What to try next Start with measurement. Log request cost and cache hit rate for a week and see where the money goes. Then pull the lever that matches what you find. - Mostly input, repeated prefix: caching - Mostly work that can wait: batching - Mostly easy requests: a smaller model or a cascade - Mostly long outputs: lower effort, tighter max tokens , shorter responses Check the math The cost numbers in this post come from tests with hand-computed expected values. Create test costs.py : python import math from types import SimpleNamespace from costs import Workload, cache hit rate, cascade cost, monthly cost, request cost SONNET, HAIKU = "claude-sonnet-5-5", "claude-haiku-5-5" TICKETS = Workload requests=1 000 000, prefix tokens=6 000, new tokens=400, output tokens=150 def close a, b : return math.isclose a, b, rel tol=1e-9, abs tol=1e-12 def usage input tokens=0, cache creation=0, cache read=0, output=0 : return SimpleNamespace input tokens=input tokens, cache creation input tokens=cache creation, cache read input tokens=cache read, output tokens=output def test one request priced from its usage fields : u = usage input tokens=1 000, cache read=5 000, output=200 1,000 x $2 + 5,000 x $0.10 + 200 x $10, per million assert close request cost SONNET, u , 0.0045 assert close request cost SONNET, u, batch=True , 0.00225 def test usage fields can be none when caching is off : u = usage input tokens=1 000, output=200 u.cache creation input tokens = u.cache read input tokens = None assert close request cost SONNET, u , 0.004 1,000 x $2 + 200 x $10, per million assert cache hit rate u == 0.0 def test cache hit rate from usage : assert close cache hit rate usage input tokens=400, cache read=6 000 , 6 000 / 6 400 assert cache hit rate usage == 0.0 def test baseline month : assert close monthly cost SONNET, TICKETS , 14 300 12,800 input + 1,500 output def test caching at 95 percent hits : assert close monthly cost SONNET, TICKETS, cache hit share=0.95 , 3 620 def test batch alone halves the bill : assert close monthly cost SONNET, TICKETS, batch=True , 7 150 def test cache with no hits costs more than no cache : assert monthly cost SONNET, TICKETS, cache hit share=0.0 monthly cost SONNET, TICKETS def test cache break even is about one hit in five : no cache = monthly cost SONNET, TICKETS assert monthly cost SONNET, TICKETS, cache hit share=0.25 < no cache assert monthly cost SONNET, TICKETS, cache hit share=0.20 no cache def test smaller model is far cheaper for the same work : assert close monthly cost HAIKU, TICKETS , 715 assert monthly cost HAIKU, TICKETS < monthly cost SONNET, TICKETS / 15 def test cascade edges : assert close cascade cost HAIKU, SONNET, TICKETS, 0.0 , monthly cost HAIKU, TICKETS both = monthly cost HAIKU, TICKETS + monthly cost SONNET, TICKETS assert close cascade cost HAIKU, SONNET, TICKETS, 1.0 , both def test cascade with caching : Haiku sees everything $209.50 , Sonnet sees the 30% that failed validation $1,086 assert close cascade cost HAIKU, SONNET, TICKETS, 0.30, cache hit share=0.95 , 1 295.5 And test cascade.py : python from cascade import Tier, cascade, escalation rate def make tier name, answer, calls : def call task : calls.append name if isinstance answer, Exception : raise answer return answer return Tier name, call def test cheap tier wins when its answer is valid : calls = out = cascade "my invoice is wrong", make tier "haiku", " Billing ", calls , make tier "sonnet", "bug", calls assert out.answer, out.tier, out.tried == "billing", "haiku", "haiku" assert calls == "haiku" the expensive model was never called def test invalid answer moves up a tier : calls = out = cascade "???", make tier "haiku", "I think it might be a billing issue", calls , make tier "sonnet", "billing", calls assert out.answer, out.tier, out.tried == "billing", "sonnet", "haiku", "sonnet" def test an error moves up a tier : calls = out = cascade "???", make tier "haiku", TimeoutError "slow" , calls , make tier "sonnet", "bug", calls assert out.tier == "sonnet" and out.answer == "bug" def test nobody gives a usable answer : calls = out = cascade "???", make tier "haiku", "dunno", calls , make tier "sonnet", "hmm", calls assert out.answer is None and out.tier is None and out.tried == "haiku", "sonnet" def test escalation rate : haiku right = lambda task: "billing" if "invoice" in task else "not sure" tiers = Tier "haiku", haiku right , Tier "sonnet", lambda task: "other" tasks = "invoice 1", "invoice 2", "invoice 3", "weird thing" outcomes = cascade t, tiers for t in tasks assert escalation rate outcomes == 0.25 assert escalation rate == 0.0 Run both with pytest -q . No fixtures or plugins are needed. Recap - Tokens are the bill, and output tokens cost the most. - Caching: put the stable part first, put the breakpoint at its end, and check the usage fields. - Batching: half price for anything nobody is waiting on. - Smaller models: big savings, so protect quality with evals and a cascade. - Combine them, but measure each step. Prices and limits checked against Anthropic's, OpenAI's and Google's documentation on October 29, 2026. They change often, so confirm them on the pricing pages before you plan a budget. Sources