Prompt caching — the LLM discount most teams aren't using A developer's analysis of prompt caching shows that cached-prefix tokens cost 5% to 50% of standard input tokens across major LLM providers, cutting input bills by 50-95% for workloads with long stable prefixes. The writeup compares caching implementations at Anthropic, OpenAI, Google Gemini and DeepSeek, noting that teams without caching enabled pay 2-20x more per request. It identifies long reused system prompts, repeated document Q&A and code assistants over a fixed codebase as the workloads that benefit most. Most LLM applications pay full input price on the same prefix tokens, request after request. A 20,000-token system prompt sent 100 times a day is 2 million tokens billed at the standard input rate — most of them repeat. Prompt caching is the API-level feature that stops the billing clock on those repeats. On the current pricing of every major provider, cached-prefix tokens cost between 5% and 50% of standard input tokens, which translates to 50-95% off the input bill for workloads with long stable prefixes. By late 2026, every major hosted LLM supports prompt caching in some form. Teams that haven't turned it on are paying 2-20x more per request than teams that have. This article covers what caching is, how each provider implements it, when it helps and when it doesn't, with the current late 2026 pricing shape. An LLM processes a prompt token-by-token, building up an internal representation the KV cache as it goes. For a stable prefix — a long system prompt, a document pasted in before a question, a tool-schema preamble — that internal state is identical every time the prefix is sent. Recomputing it from scratch on every request wastes GPU time at the provider's side and bills the user for tokens the provider didn't really have to process. Prompt caching exposes that reuse. The provider stores the KV cache for recent long prefixes, and when the next request arrives with the same prefix, it skips the recompute. The savings get passed to the user as a steep discount on cached-input tokens, and the request also runs faster because the first N tokens of inference are already done. Two things determine whether caching helps: Everything else is implementation detail. Rough shape across the four major providers. Verify current numbers on each provider's live pricing page before budgeting; this landscape moves monthly. | Provider | Cached-input discount | Cache-write surcharge | Minimum cacheable prefix | Default TTL | Model/mode | |---|---|---|---|---|---| | Anthropic | 90% off 0.1× base on most models; 95% off 0.05× on Opus 5.5 and Sonnet 5.5 | +25% on first write 1.25× base for 5-min TTL; +100% 2× base for 1-hour TTL | 1,024 tokens varies by model | 5 minutes default , 1 hour opt-in | Explicit cache control field | | OpenAI | ~90% off cached input 0.10× base across current GPT-5.6 / GPT-6 families; 50% off on older GPT-4-era models | None on current models cache writes billed at uncached-input rate | 1,024 tokens | ~30 minutes typical; up to 24 hours on extended-retention models | Automatic + optional explicit breakpoints on newest models | | Google Gemini | 90% off cached input Gemini 2.5+ ; 75% off Gemini 2.0 | Cache creation billed at standard input rate; plus a per-token-hour storage charge on explicit caches | 2,048 tokens Gemini 2.5 , 4,096 tokens Gemini 3.x , varies | Implicit: automatic; explicit: user-chosen TTL with storage cost | Implicit automatic + explicit API | | DeepSeek | ~90% off cached input | None automatic | Varies by model | Hours disk-based | Automatic | Three practical takeaways from the table: cache control marker Three workload shapes see the biggest gains: Long system prompts reused across users. A chatbot with a 10,000-token system prompt covering tone, policies, allowed tools, and example interactions. Every user turn sends that 10K prefix again. With Anthropic 5-minute caching, after the first request each subsequent request within 5 minutes pays 10% of standard for those 10K tokens — a 90% discount on the dominant input line. Document Q&A with the same document. User uploads a 50-page PDF, then asks 20 follow-up questions over the next 10 minutes. Without caching: 20 × 50-page prefix = full price. With caching: 1 full-price request + 19 cached reads = ~95% off the document portion. Code assistants over the same codebase. Cursor, Claude Code, and similar tools re-send large chunks of the codebase on every turn as context. The codebase changes slowly within a session, so cache hit rates can be high. Both Anthropic's and OpenAI's APIs are built with this workload specifically in mind. Three workload shapes see little or no gain: Anthropic's SDK requires an explicit marker on each content block the caller wants cached. Everything up to and including that marker becomes the cache boundary. python from anthropic import Anthropic client = Anthropic LONG SYSTEM PROMPT = """\ You are an assistant for a legal document review platform. Answer questions only using the attached contract. Cite clause numbers. ... 10,000 more tokens of policy, examples, allowed-tools, formatting rules """ def ask question: str - str: response = client.messages.create model="claude-sonnet-5-5", max tokens=1024, system= { "type": "text", "text": LONG SYSTEM PROMPT, "cache control": {"type": "ephemeral"}, default 5-minute TTL } , messages= {"role": "user", "content": question} , return response.content 0 .text The first call writes the cache at 1.25× standard input price for the system prompt tokens. Every subsequent call within 5 minutes reads from the cache at 0.1× or 0.05× on Sonnet 5.5 and Opus 5.5 . For a 10,000-token prompt called 20 times in 5 minutes, that's roughly: For a 1-hour TTL useful when reuse is sparser , add "ttl": "1h" : cache control = {"type": "ephemeral", "ttl": "1h"} The 1-hour cache costs 2× on write but still 0.1× on read, so it pays off after two reads instead of one. The 1-hour TTL became generally available in August 2025 and no longer requires a beta header. OpenAI's caching requires zero code change on current models. Any prompt prefix at least 1,024 tokens long that gets re-sent within the cache window typically 30 minutes on GPT-5.6 and later; up to 24 hours on extended-retention models gets charged at the cached-input rate automatically. python from openai import OpenAI client = OpenAI LONG SYSTEM PROMPT = "..." 2000 10,000+ tokens def ask question: str - str: response = client.chat.completions.create model="gpt-5.6-terra", messages= {"role": "system", "content": LONG SYSTEM PROMPT}, {"role": "user", "content": question}, , Cache hit telemetry cached = response.usage.input tokens details.cached tokens print f"Cached tokens hit: {cached}/{response.usage.prompt tokens}" return response.choices 0 .message.content Two things to notice: cached tokens field on usage.input tokens details . cached tokens / prompt tokens to catch cache misses early — a small change in the system prompt a date rotated in, a dynamic value inserted at position 50 will silently drop the hit rate without any error. On GPT-6 family models, developers can also set explicit cache breakpoints to force the cache boundary where they want it, in addition to the automatic behaviour. Useful for prompts where the stable prefix ends mid-paragraph and automatic detection might cache less than the full stable portion. Gemini supports two caching modes: Implicit caching — automatic, no code change, 90% discount on hits on Gemini 2.5 and 3.x models , no storage fee. The system detects repeated prefixes and routes to the cached version transparently. Explicit caching — the developer creates a named cache object, which stores the prefix on Google's side for a chosen TTL. Reads are 90% off standard input, but there is a per-token-hour storage charge while the cache is alive. python from google import genai from google.genai import types client = genai.Client Create an explicit cache from a 50-page PDF uploaded earlier cache = client.caches.create model="models/gemini-3.1-pro-preview", config=types.CreateCachedContentConfig display name="legal-contract-123", system instruction="You are a legal reviewer. Answer only from the attached contract.", contents= my uploaded pdf , ttl="3600s", 1 hour; billed for each hour of storage , Use the cache in later requests response = client.models.generate content model="models/gemini-3.1-pro-preview", contents="What is the termination clause?", config=types.GenerateContentConfig cached content=cache.name , The storage cost matters. As of October 2026, Flash-tier models bill around $1.00 per million tokens per hour with Gemini 3.8 Flash discounted to $0.50 through December 31, 2026, then rising to $1.00 on January 1, 2027 , and Pro-tier both Gemini 2.5 Pro and 3.1 Pro Preview is $4.50 per million tokens per hour. For a 100,000-token cached PDF held for a 1-hour conversation on Flash, storage adds $0.05-$0.10 — negligible relative to the ~90% savings on reads if there are more than one or two queries. For a 1-million-token cached corpus held for 24 hours on Gemini Pro, storage adds ~$108 — only worth it with substantial reuse. Verify current rates on Google's pricing page before planning. Implicit caching sidesteps the storage charge entirely. The trade-off: no guarantee the cache will be there when needed, since the system decides what to retain. Six patterns that silently kill cache hit rates: A timestamp or current date rendered into the system prompt. "Today is 2026-11-24" means every request has a unique prefix. Move the dynamic data into the user message, or round the date to the day/hour and accept slight staleness. Streaming responses that join concatenated chat history differently each turn. The join logic has to be byte-identical across calls. Different whitespace in message separators invalidates the cache. Tool/function definitions changing order between calls. Even if the tools themselves are the same, re-ordering their position in the request changes the prefix bytes. Sort tool lists deterministically. A/B tests or feature flags that rotate parts of the prompt. Each flag variant has its own cache, and total hit rate drops. Keep the flag variants above the cache boundary, not inside it. A userId or sessionId interpolated into the system prompt. Every user has their own cache, hit rates stay poor across users. Move user-specific data to the user message. Model upgrades without re-planning cache budgets. Each model has its own cache namespace. The day a service switches from Sonnet 5 to Sonnet 5.5, every cache resets. If the workload has none of the three "helps most" shapes above, caching may not be the biggest cost lever. Three alternatives: Caching is one tool in the cost toolbox. Teams that treat it as the only lever often leave bigger savings elsewhere. Prompt caching is a near-free optimisation for any LLM workload with a stable prefix over 1,024 tokens that gets reused within minutes. On the current pricing of every major provider, it cuts input cost 50-95% for the cached portion. The engineering cost is roughly one hour: identify the stable prefix, add the right marker or just let auto-caching work , log the hit rate, and move on. For teams that haven't turned it on, this is almost certainly the highest-ROI cost cut sitting in their LLM bill today. For teams that have turned it on but see poor hit rates, the six gotchas above cover 90% of the real-world misses. Verify current pricing on each provider's live docs — the specific percentages above are accurate as of late 2026 but the LLM pricing landscape moves monthly. Related reading: