# How Does Prompt Caching Work for LLMs, and Why It Cuts Bills in Half > Source: > Published: 2026-09-08 22:22:31+00:00 *Prompt caching can cut a Claude or GPT bill by 50 to 90 percent, but only if you understand what actually gets stored, how long it lives, and why the order you write your prompt in decides whether you hit the cache at all.* - Anthropic charges 1.25x standard input price to write a 5-minute cache and 2x to write a 1-hour cache, but only 0.1x, a 90 percent discount, to read from it - Caching only works above a minimum size: Anthropic won't cache prefixes under 1,024 tokens on Sonnet and Opus, or 2,048 tokens on Haiku, and it fails silently with no error - Cache hits depend on exact prompt order: static system prompts and tool definitions must come first, and anything that changes per call, like a timestamp or session ID, must come last or it invalidates everything after it - Notion cut its Claude API costs by 90 percent and cut latency by up to 85 percent after restructuring its prompts around caching, according to Anthropic's own published case study - Anthropic's default cache TTL quietly reverted from 1 hour to 5 minutes for weeks in early 2026 with no announcement, according to developer reports on GitHub, spiking cache costs 20 to 32 percent for teams that didn't notice Ask most founders how does prompt caching work for LLMs and you'll get some version of you turn it on and the bill drops. That's the pitch, not the mechanism. Prompt caching is a storage system with its own pricing tiers, its own expiration clock, and its own rules about what silently breaks it, and most teams shipping AI products right now are paying full price for tokens they assume are cached. Here's what's actually happening under the hood. Every token you send has to be processed before the model can generate anything, and that processing, not the output, is where most of an LLM bill goes on agentic or RAG-heavy applications. Prompt caching lets Anthropic or OpenAI store the internal state of a prompt prefix so a later request reusing that exact prefix skips reprocessing it. Same idea as a CDN caching a webpage, except the page here is your system prompt, your tool definitions, and whatever static context rides along on every call. The part that trips people up is that caching isn't free the first time you use it. Writing to the cache costs more than a normal token, not less. On Anthropic's API, a cache write costs 1.25 times the standard input price with a 5-minute time-to-live, or 2 times the standard price with a 1-hour TTL. That premium pays for storing the prefix. Every request afterward that reuses the same prefix inside that window counts as a cache read, and a cache read costs just 0.1 times the standard input rate, a 90 percent discount. Run the numbers on Claude Sonnet, where standard input costs $3 per million tokens. A 5-minute cache write costs $3.75 per million tokens. A cache read costs $0.30 per million tokens. Write once, read it ten times, and the blended cost per token falls by more than 70 percent before you've touched a single other part of the system. That's llm token cost optimization at its most literal: pay more once, then pay far less for as long as the window stays open. [How Does the Model Context Protocol Work, and What It Kills for Startups](https://startupfortune.com/how-does-the-model-context-protocol-work-and-what-it-kills-for-startups/) How does the Model Context Protocol actually work, and why should founders care? MCP standardizes how AI agents connect to outside tools, and two years after Anthropic open-sourced it, that standardization is quietly commoditizing the AI integration startups whose whole pitch was doing that wiring for you. Here's the client-host-server mechanics... - [model context protocol for AI agent integration](https://startupfortune.com/how-does-the-model-context-protocol-work-and-what-it-kills-for-startups/) - [what happened to AI integration startup business models](https://startupfortune.com/how-does-the-model-context-protocol-work-and-what-it-kills-for-startups/) OpenAI takes a different approach, and the tradeoff runs the other way. GPT-4o and its successors cache automatically past 1,024 tokens, no setup required, and matched tokens bill at 50 percent of the standard rate. There's no write charge and nothing to configure, but there's also no control: you can't stretch the window, you can't request a deeper discount, and you can't force a prefix to stay warm on demand. Anthropic makes you mark up to four cache breakpoints yourself. It pays you back with a steeper discount for doing the work. Google's Gemini API runs a comparable system with its own explicit context caching, so the ordering discipline below applies whether the stack sits on Claude, GPT, or Gemini. ## The Silent Failure Mode Nobody Gets Warned About There's a failure mode here that never throws an error, which is exactly why it's so common. Anthropic won't cache a prefix shorter than 1,024 tokens on Claude Sonnet and Opus, or 2,048 tokens on Claude Haiku. Mark a 600-token system prompt with a cache_control breakpoint and the API accepts the request, runs it, and returns an answer, quietly billing it at full price every time, because the prefix never cleared the minimum. No warning. No rejected request. Nothing in the response flags it unless you already know to look for it. A team can ship a product it believes is caching for months without ever collecting a single discount. ## Why Prompt Order Decides Whether You Hit The Cache This is the part almost nobody gets right on the first try, and it's the real reason bills stay high even after a team turns caching on. A cache match is judged against an exact token prefix, checked from the very start of the prompt. The instant a token fails to match, everything after that point is a miss, no matter how much of the rest of the prompt is identical to before. That makes prompt structure a cost decision, not a style choice. Static material, your system instructions, tool schemas, long reference documents, needs to sit first. Anything that changes on every call, the user's latest message, a session ID, a timestamp accurate to the second, needs to sit last. Anthropic's own documentation says this outright: put cacheable content at the start of the prompt and dynamic content at the end. Teams that skip this and drop a live timestamp into the system prompt for logging purposes pay the 1.25x write premium every single call, because that timestamp sits ahead of the cache breakpoint and drags the entire prefix behind it down with it. It's a one-line prompt bug that turns a caching feature into a caching tax. Randomizing few-shot examples for variety, rotating which tool shows up first in a schema list, or slotting a running conversation summary above the static instructions instead of below them does the same damage. None of it looks like a caching bug. It looks like ordinary prompt engineering, which is exactly why it survives code review. ## It's An Exact Match, Not A Smart One It helps to be blunt about what caching is not. It isn't a semantic system that recognizes two prompts as close enough to count. It's a literal, byte-for-byte prefix match. Reorder two sentences in your system prompt, add a single extra space, or change your tool schema by one field, and the entire cache invalidates, even though the prompt's meaning hasn't shifted at all to a human reading it. That's why version-controlling a system prompt like code, rather than editing it ad hoc in a config file, matters more once caching is in the picture. A one-character fix pushed at 2 a.m. resets every cached session for every user until the new prefix gets rewritten and warmed back up. [What Is an AI Agent Swarm and How These Systems Actually Coordinate](https://startupfortune.com/what-is-an-ai-agent-swarm-and-how-these-systems-actually-coordinate/) What is an AI agent swarm? It's a group of specialized AI agents working under an orchestrator that splits tasks, hands off subtasks, and merges results, rather than one model doing everything alone. This piece breaks down how orchestrator-worker coordination actually works, where it fails, and what Anthropic and OpenAI's own engineering teams... - [how AI agent swarms coordinate](https://startupfortune.com/what-is-an-ai-agent-swarm-and-how-these-systems-actually-coordinate/) - [multi agent systems architecture explained](https://startupfortune.com/what-is-an-ai-agent-swarm-and-how-these-systems-actually-coordinate/) ## Caching A Conversation That Keeps Growing Chat products complicate this because the prompt isn't fixed, it grows every turn. The fix isn't to cache the whole conversation as one blob and hope for a hit. It's to cache incrementally: mark a breakpoint after the system prompt and tool definitions, then another after the conversation history as it stood at the end of the previous turn, and let only the newest user message sit uncached at the very end. Each new turn reuses everything before it as a cheap read and pays the write premium only on the small slice that's actually new. Place the breakpoint after the latest message instead of before it, and every turn rewrites the entire history from scratch, which is the single most common way agent-style chat products blow through a token budget without anyone figuring out why. ## Cache Hit Vs Cache Write Pricing In Real Numbers Take an agent carrying a 20,000-token system prompt, tool definitions plus a slice of product documentation, running 50 calls in one working session. Without caching, that's 50 times 20,000, a million input tokens billed at the standard $3-per-million rate: three dollars just to repeat the same instructions fifty times. Structure the prompt correctly and the first call pays the 5-minute write premium on 20,000 tokens, about seven cents. The remaining 49 calls read that same prefix at the 90 percent discount, about 29 cents. Total cost: roughly 37 cents instead of three dollars. An 88 percent reduction, and none of it required touching the application logic. ## How To Actually Check Your Cache Hit Rate The only reliable way to know whether any of this is working is to read the usage object Anthropic returns with every response. It reports cache_creation_input_tokens for what got written and cache_read_input_tokens for what got read from cache, sitting right alongside the normal input and output counts. If cache_read_input_tokens comes back zero across a run of calls that should all be hitting the same prefix, the cache isn't working, whatever the code claims to be doing. Most teams never check this field. They add one cache_control block, assume the discount applies, and move on. ## Don't Trust The Default TTL, Confirm It The TTL isn't even a fixed constant you can assume and forget. Developers on GitHub and DEV Community reported that Anthropic's default cache TTL quietly shifted from 1 hour back down to 5 minutes for a stretch of calls in early 2026, with no blog post, no changelog entry, and no warning attached anywhere. Teams that had built their cost models around a 1-hour window watched cache creation costs climb 20 to 32 percent without changing a line of their own code, simply because more of their traffic started falling outside a window that had quietly gotten five times shorter. If a long TTL matters to your cost model, don't trust whatever the API defaults to. Set the ttl parameter to 1h explicitly inside the cache_control block on every call, and check the usage object to confirm you're actually getting the window you asked for. ## Notion's Numbers Show This Isn't A Theoretical Saving This isn't a hypothetical optimization somebody ran in a sandbox. Notion built its AI assistant on Claude, and according to Anthropic's own published case study, restructuring its prompts around caching cut its API costs by 90 percent and cut response latency by as much as 85 percent. Notion co-founder Simon Last said prompt caching made Notion AI faster and cheaper, all while maintaining quality. That's not a marginal tuning win. That's the difference between a feature that survives contact with a real user base and one that gets quietly shelved for being too expensive to run. ## Choosing Between A 5-Minute And A 1-Hour Cache The TTL you pick should match how your product actually gets used, not whatever default sat in a tutorial you copied. A 5-minute cache is cheaper to write and fits agent loops or coding assistants that fire off a burst of calls back to back, then stop. Accessing a cached prefix resets its clock, so as long as calls keep landing inside that 5-minute window, the cache never expires and the write premium never repeats. A 1-hour cache costs double the standard rate to write instead of 1.25 times, but it's the better bet for products with real gaps between requests: a support tool where an agent reads a ticket, thinks it over, and replies a few minutes later, or any workflow where the model might sit idle longer than five minutes between turns. Pick the short window for a bursty workload and you save on every write. Pick it for a slow, gappy one instead and you'll pay the write premium over and over for a cache that keeps timing out before it earns its cost back. Anthropic lets you set up to four cache breakpoints inside a single request: one after the system prompt, one after tool definitions, one after long reference material, one after conversation history. Most teams calling the API right now set zero. That gap, not the discount rate and not the TTL, is where most of the money sits for anyone serious about reducing LLM API costs. **Also read:** [How Does Invoice Factoring Work for Startups With No Collateral](https://startupfortune.com/how-does-invoice-factoring-work-for-startups-with-no-collateral/) • [How Does a Merchant of Record Work, and Why SaaS Founders Are Switching](https://startupfortune.com/how-does-a-merchant-of-record-work-and-why-saas-founders-are-switching/) • [Here's Why AI Agent Retry Logic Quietly Multiplies Your API Costs](https://startupfortune.com/heres-why-ai-agent-retry-logic-quietly-multiplies-your-api-costs/) *This article is filed under [AI News](https://startupfortune.com/category/ai/). Check it for more stories.* ## Join the discussion [Open in the community →](/community/) Almost there. Sign in and your reply posts straight away.