{"slug": "prompt-caching-is-the-cheapest-way-to-cut-your-ai-bill-and-most-people-still-do", "title": "Prompt Caching Is the Cheapest Way to Cut Your AI Bill, and Most People Still Do Not Use It", "summary": "Prompt caching, a feature offered by major AI providers that reduces the cost of repeated input tokens by up to 90 percent, remains underused in production apps despite requiring minimal code changes and offering identical output quality. The technique works by storing the processed state of static prompt prefixes, enabling models to skip reprocessing identical content on subsequent requests. Widespread adoption could cut AI bills by a quarter to half for many applications.", "body_md": "*If your app sends the same system prompt, the same tool definitions, or the same reference documents on every single request, you’re paying full price to reprocess identical text over and over. Prompt caching makes those repeated tokens up to 90 percent cheaper, with no change to the output, and it takes almost no code. It’s the single highest-value cost lever available on every major AI provider right now, and it’s quietly sitting unused in a huge number of production apps. Here’s exactly how it works, the honest math on when it pays off, and how to turn it on.*\n\nThere’s a specific kind of waste that almost every AI application commits without noticing. Imagine a chatbot with a 5,000-token system prompt, the instructions, the tone guidance, the company policies, that get sent to the model on every request. If that chatbot serves 10,000 conversations a day, it’s sending those same 5,000 tokens 10,000 times. That’s 50 million tokens a day of nothing but the system prompt, reprocessed from scratch on every call, before a single user message is even considered. You’re paying, over and over, for the model to re-read text it has already read a thousand times that hour.\n\nPrompt caching is the fix, and it’s close to free money. It’s a feature every major provider now offers that stores the processed state of repeated content so the model can skip reprocessing it, and bills those repeated tokens at a steep discount. The savings are large, the output is identical, and the implementation ranges from zero code to a single extra field. And yet a striking number of teams building real products haven’t turned it on, and are leaving a quarter to half of their input bill on the table. Let’s fix that.\n\nWhen you send a prompt to a language model, the model has to process every token of your input before it generates anything. That processing step, computing what the field calls the key-value tensors for each token, is real work and you pay for it. The important detail is that this cost is per token of input, and it happens every single time, even if 99 percent of the input is identical to your last request.\n\nPrompt caching stores the result of that processing for a chunk of your prompt. The first time the model sees your big static system prompt, it processes it normally and saves the computed state. On the next request that starts with the exact same content, instead of reprocessing those tokens, the model loads the saved state and bills you a fraction of the normal price for them. The key thing to understand is that this isn’t an approximation or a shortcut that changes the answer. The model does exactly the same math it would have done, it just skips recomputing something it already computed. The output is byte-for-byte identical. Caching is a pure cost and speed optimization with no quality tradeoff whatsoever, which is what makes it such an easy win.\n\nThere’s one hard rule that governs everything about how you use it. Caching works on exact prefix matches, meaning it caches from the beginning of your prompt up to the point where the content stops being identical. The moment something changes, the cache stops applying from that point on. This single fact drives the one design principle that matters, put your static content first and your changing content last.\n\nThe headline numbers are genuinely large. As of 2026, all three major providers discount cached input tokens by 90 percent, meaning a cached token costs one tenth of what it normally would. That wasn’t always uniform, the discount used to be smaller on some providers, but the current generation of flagship models across the major labs has converged on the same 90 percent cached-read discount.\n\nPut that against a realistic workload and the effect is dramatic. Consider a customer support assistant with a 4,000-token system prompt of instructions and policies, plus 6,000 tokens of reference documents pulled in per query, so 10,000 tokens of largely static context on every request. Without caching, you pay full input price for all 10,000 tokens every time. With caching and a realistic high hit rate, the bulk of those tokens bill at one tenth the price. Worked out over 10,000 queries a day, that is the difference between something like 10,000 dollars a month and roughly 2,400 dollars a month on that input, a reduction in the neighborhood of 75 percent of the total bill for that workload. Documented production cases back this up, one engineering team reported cutting total model spend by 59 to 70 percent by raising their cache hit rate from a poorly-structured 7 percent to a well-structured 84 percent.\n\nBut there are two honest caveats that separate the headline from your actual invoice, and you need both.\n\nThe first is that caching only ever discounts input tokens. It does nothing to the cost of the tokens the model generates. Any savings estimate that applies the discount to your output spend is simply wrong. You model the input side only and treat the generated-token bill as unchanged. For workloads that send a lot of context and generate short answers, that’s a huge win. For workloads that send little and generate a lot, caching helps less.\n\nThe second is that on some providers there’s a small write premium. The mechanics differ by provider, and this is where they diverge most, so it’s worth being precise.\n\nThe three big providers landed on structurally different designs for the same feature, and which suits you depends on your workload.\n\nOne major provider, Anthropic, uses explicit caching. You mark which parts of your prompt to cache with a specific control field in your request, which gives you precise control over exactly what gets cached and where the boundaries are. In practice that means adding a small cache_control marker to the end of the block you want cached, like this.\n\n```\nresponse = client.messages.create(    model=\"claude-sonnet-4-6\",    max_tokens=1024,    system=[        {            \"type\": \"text\",            \"text\": LONG_STATIC_SYSTEM_PROMPT,  # your big reusable block            \"cache_control\": {\"type\": \"ephemeral\"},  # cache everything up to here        }    ],    messages=[{\"role\": \"user\", \"content\": user_message}],  # the variable part)\n```\n\nThat single cache_control line is the whole implementation. Everything from the start of the request up to that marker gets cached, and the variable user message after it stays uncached, which is exactly the ordering you want. The tradeoff is a write premium, the first time you write something to the cache, you pay about 1.25 times the normal input price for it, in exchange for the 90 percent discount on every subsequent read. That write premium pays for itself after a single cache hit, so for any workload where the same content is reused even a couple of times, you come out ahead almost immediately. Anthropic's cache also has a default lifetime of five minutes that refreshes each time it is used, with a one-hour option at a higher write premium.\n\nAnother major provider, OpenAI, uses automatic caching. There’s nothing to mark and no code to write, the system detects when your request shares a prefix with a recent one and applies the discount automatically for prompts above a minimum length. There’s no separate write premium. The tradeoff is less control, you can’t precisely choose the cache boundaries, the system decides based on the longest matching prefix. Historically OpenAI’s discount was smaller, but on its current-generation models the cached-read discount reaches the same 90 percent, so the gap that older articles describe has largely closed.\n\nThe practical rule between these two. If your traffic is high-volume with large, stable prompts, the explicit approach with precise control tends to maximize savings, and the write premium is trivial when amortized over thousands of hits. If your traffic is sporadic or your prompts change often, the automatic approach with no write premium and no code is the cleaner choice. Both get you to roughly the same 90 percent on the cached portion in the end. A third major provider, Google, offers both an explicit caching mode with a per-hour storage fee and a zero-setup automatic mode, landing somewhere between the two.\n\nHere is where most of the teams that “turned on caching” but saw little benefit go wrong. Caching matches from the start of your prompt until the first difference, so the entire game is keeping your static content at the front and your dynamic content at the back.\n\nThe classic mistake is injecting something that changes on every request near the top of the prompt. The single most common culprit is putting the current date and time at the start of a system prompt. Because that string is different on every call, it breaks the cache immediately, and everything after it has to be reprocessed at full price no matter how static it is. Concretely, this is the version that quietly destroys your cache.\n\n```\nToday is 2026-07-10 14:32:01.          <- changes every call, ruins everything belowYou are a support assistant for...     <- 4,000 static tokens, now uncacheable[6,000 tokens of policy docs]          <- also uncacheable, reprocessed in full\n```\n\nAnd this is the fix, the exact same information, reordered so the static frame comes first and the one changing value moves to the end.\n\n```\nYou are a support assistant for...     <- 4,000 static tokens, cached[6,000 tokens of policy docs]          <- cached--- cache_control marker here ---Current time: 2026-07-10 14:32:01.     <- the only uncached partUser question: ...\n```\n\nSame content, but now 10,000 tokens cache cleanly and only a few uncached tokens get full-price processing. The fix is simply to move the changing content, the date, the user’s name, the session details, to the end of the prompt or into the user message, and let the long static frame at the front cache cleanly.\n\nThe other common cache-killers are subtle variations that you might not even notice. Regenerating your system prompt string on every call instead of reusing the same one, which can introduce tiny differences in whitespace or line endings that invalidate the match. Even a single changed character anywhere in the cached prefix invalidates the cache from that point. So the discipline is to build your static prompt once, keep it byte-identical across calls, and never let anything variable sneak in before it.\n\nThe way to know if it’s working is to look. Every provider returns cache statistics in the response. On Anthropic you check the usage block for the cache fields, and a working cache shows a large read count on repeat requests.\n\n```\nprint(response.usage)# cache_creation_input_tokens: 10000   <- the one-time write (first call)# cache_read_input_tokens:     10000   <- the 90%-off read (every call after)# input_tokens:                   45   <- only the fresh, uncached tokens\n```\n\nOn OpenAI the equivalent is a cached_tokens figure inside the usage object. If those cache fields are zero on every request, your cache isn't activating, usually because your prompt is below the minimum cacheable length or because dynamic content is sitting in front of your static content. Track your cache hit rate as a first-class metric, cached tokens divided by total input tokens, and if it is below 50 percent on a long-context workload, your static and dynamic content are almost certainly in the wrong order.\n\nHonesty means naming when this doesn’t help, because caching isn’t universal.\n\nIf your calls are genuinely stateless and short, single-shot requests under the minimum cacheable length with little repeated content, there’s nothing to cache and the minimum thresholds will simply exclude you. For those, the right move is a smaller, cheaper model, not caching. If every byte of your prompt genuinely depends on the user, a fully personalized prompt with no shared frame at all, then there’s no common prefix to cache, and caching cannot help until you restructure the prompt to put a long shared section first. And for workloads that generate far more than they consume, caching helps only at the margin, because it never touches output cost.\n\nThe pattern where caching is transformative is the opposite of these, a large, stable block of context, system prompts, tool definitions, reference documents, few-shot examples, reused across many requests, with the variable part being small. That describes chatbots, retrieval pipelines, and agentic loops almost perfectly, which is exactly why caching is the highest-value lever for those workloads specifically.\n\nPrompt caching is the rare optimization where the code change is tiny and the bill change is enormous. For any production workload that re-sends the same static content across requests, and that describes most chatbots, most retrieval systems, and essentially every agent, it cuts the input cost of that repeated content by around 90 percent with zero effect on the quality of the output. The savings on a real workload routinely land in the range of cutting total spend by half or more.\n\nThe reason it goes unused isn’t that it’s hard, it’s that it’s invisible. Nothing breaks if you skip it, the bill is just quietly larger than it needs to be, and the fix requires knowing the feature exists and structuring your prompt to use it. So here’s the whole thing in one action. Start with the provider you already use. Put your static content first and your dynamic content last. Turn on caching, either by adding the cache marker or by confirming automatic caching is engaged for your model. Then check the cache-read tokens in your responses the next day. The numbers will make the case for doing the rest. It’s close to the only place in AI engineering where you can cut a bill by half in an afternoon without changing what your product does at all.\n\n*Pricing, discount rates, cache lifetimes, and minimum thresholds vary by provider and by model generation and change frequently. The figures here reflect widely reported 2026 pricing, but confirm the current numbers on your provider’s official documentation before architecting around them.*\n\n[Prompt Caching Is the Cheapest Way to Cut Your AI Bill, and Most People Still Do Not Use It](https://pub.towardsai.net/prompt-caching-is-the-cheapest-way-to-cut-your-ai-bill-and-most-people-still-do-not-use-it-d7dbb9f474b9) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/prompt-caching-is-the-cheapest-way-to-cut-your-ai-bill-and-most-people-still-do", "canonical_source": "https://pub.towardsai.net/prompt-caching-is-the-cheapest-way-to-cut-your-ai-bill-and-most-people-still-do-not-use-it-d7dbb9f474b9?source=rss----98111c9905da---4", "published_at": "2026-07-12 17:01:02+00:00", "updated_at": "2026-07-12 17:11:14.430566+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-products", "ai-tools", "ai-infrastructure"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/prompt-caching-is-the-cheapest-way-to-cut-your-ai-bill-and-most-people-still-do", "markdown": "https://wpnews.pro/news/prompt-caching-is-the-cheapest-way-to-cut-your-ai-bill-and-most-people-still-do.md", "text": "https://wpnews.pro/news/prompt-caching-is-the-cheapest-way-to-cut-your-ai-bill-and-most-people-still-do.txt", "jsonld": "https://wpnews.pro/news/prompt-caching-is-the-cheapest-way-to-cut-your-ai-bill-and-most-people-still-do.jsonld"}}