# Prompt Caching, Explained: How to Cut Your LLM Bill by 70-90% (With Real Math) > Source: > Published: 2026-08-19 11:07:02+00:00 In my last post I broke down how LLMs count tokens and why your bill is decided at the tokenizer. A lot of the follow-up questions were the same: *"Okay — so how do I actually pay less?"* This is the answer, and it's the single highest-leverage cost lever available on Claude, GPT, and Gemini today: **prompt caching.** It requires no model change, no quality tradeoff, and often just a few lines of code. Done well, it cuts input token costs by 70-90% on typical production workloads. Done poorly, it does nothing — or quietly makes things worse. Let's break down how it works, the real cost math, and the mistakes that leave most of the savings on the table. When a model processes your prompt, it computes attention key-value (KV) tensors for every token. That's real compute, and you pay for it on every request. Here's the thing: most production prompts are **mostly repetition**. The system prompt, the tool definitions, the few-shot examples, the big RAG document — that prefix is identical on call after call. Only the last bit (the user's actual message) changes. Prompt caching stores the already-computed KV tensors for that repeated prefix server-side. When your next request starts with the same prefix, the model **skips recomputing it** and loads the cached result instead — and bills those cached tokens at a steep discount. The mental model: you're paying full price to process your system prompt *once*, then paying a fraction of that to reuse it for the next N requests. Caching works on **prefixes**. The provider can only reuse everything up to the first point where your prompt differs from last time. That single fact dictates everything: Put stable content at the front. Put anything that changes at the very end. Structure your prompt like this: ``` [ system prompt ] ← static, cache this [ tool definitions ] ← static, cache this [ few-shot examples ] ← static, cache this [ retrieved documents ] ← semi-static -------------------------------------------------- [ conversation history ] ← changes [ user's new message ] ← changes every time ``` The instant something dynamic leaks into the front — a timestamp, a request ID, a randomly ordered tool list, even an inconsistent trailing newline — the cache invalidates for *every token after it*. This is the number-one reason teams see "90% off" pricing advertised and a 20% hit rate in reality. The three big providers all discount cached input by roughly 90%, but the mechanics and fine print differ enough to change your architecture. You opt in by marking stable blocks with `cache_control` (up to 4 breakpoints). Claude splits your bill into: Default TTL is 5 minutes, and every read refreshes the timer. There's also an automatic mode now, but explicit breakpoints give you the most control. The write surcharge typically pays for itself on the *first* hit. Caching happens automatically on supported models — no markers, no cache objects. The catch is a hard **1,024-token minimum prefix**: a 900-token system prompt will *never* cache, no matter how consistent it is. Retention runs from a few minutes of inactivity up to 24 hours on newer models. Simplest to adopt, least to tune. Gemini offers implicit caching (automatic, ~10% read rate) and explicit caching (you create a named cache object and reference it by ID). The gotchas: Gemini caching rewards big-document workloads, not typical short prompt engineering. | Claude | GPT | Gemini | | |---|---|---|---| | Mode | Explicit (`cache_control` ) + auto | Automatic | Implicit + explicit | | Read discount | ~90% off | ~50-90% off | ~90% off | | Write cost | 1.25×-2× base | none extra | storage/hour | | Min prefix | small | 1,024 tokens | very large | | Default TTL | 5 min (refreshes) | up to 24h | configurable | *(Mechanics and rates shift often — confirm on each provider's current docs before budgeting.)* Let's make it concrete. Say you have a **5,000-token system prompt** reused across **10,000 requests/day**, on a model at **$3 / 1M input** with a cache read at **$0.30 / 1M** (90% off) and a 5-minute write at **$3.75 / 1M**. Every request pays full price for those 5,000 tokens: ``` 10,000 req × 5,000 tokens = 50,000,000 input tokens/day 50,000,000 / 1,000,000 × $3 = $150/day → ~$4,500/month ``` *(That's just the cached-portion input — actual bills add the dynamic tail and output.)* Assume the cache is written fresh a handful of times a day as it expires — say ~50 writes — and everything else is a read: ``` Writes: 50 × 5,000 / 1,000,000 × $3.75 ≈ $0.94/day Reads: 9,950 × 5,000 / 1,000,000 × $0.30 ≈ $14.93/day ------------------------------------------------------------ Total ≈ $15.87/day → ~$476/month ``` That's the repeated-prefix cost dropping from **~$4,500 to ~$476 a month — about 89% off** — by making one part of your prompt cacheable. The often-cited real-world version of this is an agent that went from **$720/month to $72/month** by adding three cache breakpoints. The savings scale with two things: **how big your static prefix is** and **how often you reuse it within the TTL**. Big system prompt + high request rate = enormous savings. Tiny prompt + sporadic traffic = little to none. Caching fails quietly. You still get correct responses — you just don't get the discount, and nothing errors out to tell you. Watch for these: Don't trust the marketing — trust the usage metadata. Every provider exposes cache stats (field names differ): `cache_read_input_tokens` , `cache_creation_input_tokens` `cached_tokens` A quick health check: ``` hit_rate ≈ cached_input_tokens / total_cache_eligible_input_tokens ``` If that number is consistently near zero, dynamic content has leaked into your prefix. A sudden drop usually means something changed in the reusable part of your prompt. Caching won't fix a badly chosen model or a bloated context. But for the very common case of "big stable prompt, called a lot," it's the closest thing to free money in the LLM stack. Most teams are leaving 70-90% of it on the table. *Have you shipped prompt caching in production? What was your real hit rate — and what broke it? I'd love to hear the war stories in the comments.*