The Third Price: I Measured Prompt Caching Across 393 LLMs and Found a 90% Discount Hiding Behind One JSON Field

A developer measured prompt caching costs across 393 LLMs and found that a single JSON field can unlock a 90% discount on repeated prompts. Tests showed Anthropic's Claude Sonnet required an explicit cache_control marker to cache, while OpenAI's GPT-4o-mini cached automatically, cutting costs by 48% and latency by 30%. The developer warns that without configuration, Anthropic traffic can cost up to 10x more than necessary.

Same model. Same prompt. Ten minutes apart. $0.00464 per request $0.00069 per request The difference was one JSON field. I spent last week measuring what LLM requests actually cost, expecting to write about tokenizer differences. I found something more useful instead. Open any LLM pricing page. You get price per input token and price per output token. Every comparison article, every spreadsheet, every "which model is cheapest" post uses those two numbers. Providers publish a third one: prompt $0.000002 fresh input input cache read $0.0000002 cached input <- 10x cheaper input cache write $0.0000025 writing to cache <- 25% MORE For RAG and agent workloads, that third line is where most of your money lives. Here's why. A typical RAG support prompt breaks down like this measured, not estimated : system prompt 96 tokens 32% identical on every request retrieved context 143 tokens 48% often stable across a session conversation history 55 tokens 18% prefix is stable user message 6 tokens 2% the only genuinely new part 98% of that prompt is repeated content being billed at full price. I sent a 2,600-token system prompt to Claude Sonnet twice, ten minutes apart, through an API gateway so both providers ran through identical client code. Then the same test against GPT-4o-mini. Then I pulled pricing for every model on the platform. Three separate findings came out of it. First run, no cache configuration at all: cold: input=2253 cached=0 cost=$0.00470600 warm: input=2251 cached=0 cost=$0.00464200 Nothing cached. Full price both times. No error, no warning, no hint that a 90% discount was available. Then the identical prompt with a cache control marker on the system message: body = { "model": "anthropic/claude-sonnet-4.5", "max tokens": 20, "messages": { "role": "system", "content": { "type": "text", "text": PREFIX, "cache control": {"type": "ephemeral"} <- this line } }, {"role": "user", "content": question}, , } Result: cold: input=2628 cached=0 written=2614 cost=$0.00676300 warm: input=2626 cached=2614 written=0 cost=$0.00068680 90% cheaper on every request after the first. Note the first request cost more than the uncached version: $0.00676 vs $0.00464, about 46% more. That's the cache write premium. You're ahead by request two, but if you send a prompt exactly once, caching loses you money. Same test, GPT-4o-mini, no configuration whatsoever: cold: input=1355 cached=0 cost=$0.00021525 1737ms warm: input=1355 cached=1280 cost=$0.00011145 1223ms 1,280 of 1,355 input tokens served from cache. 48% cheaper and 30% faster , with no code change at all. So the discount exists on both providers. On one it's automatic. On the other you have to know to ask, and nothing tells you that you're paying full price. If your team assumed caching was automatic everywhere, your Anthropic traffic has been costing up to 10x more than necessary and no signal in the response would have told you. I changed a single character at the front of the prompt: warm identical prefix : cached=1280 broken 1 char changed : cached=0 Every token after that character had to be reprocessed at full price. This is documented behavior, but seeing it is different from reading it. Caching matches on the prefix , exactly. Anthropic's docs describe it as covering tools, then system, then messages in that order, up to the cache breakpoint. Azure's guidance puts the practical rule bluntly: stable content at the beginning, dynamic content at the end. Which means this is a structural property of how you build prompts, not a config flag: CACHES WELL messages = {"role": "system", "content": SYSTEM PROMPT}, stable {"role": "system", "content": RETRIEVED CONTEXT}, stable per session {"role": "user", "content": user question}, changes CACHES NOTHING messages = {"role": "system", "content": f"Time: {now }\n{SYSTEM PROMPT}"}, timestamp {"role": "user", "content": user question}, kills prefix A timestamp at the top of your system prompt invalidates the cache on every single request. So does a user ID, a session ID, or anything else that varies. I pulled pricing for all 393 models available on the platform: models priced: 393 with cache pricing: 248 63% without: 145 discount min 1.0x median 10.0x max 120.8x write premium charged by 50 of 71 models publishing one distribution: under 5x 68 5-9x 53 ~10x 120 11-25x 1 over 25x 6 Three things worth pulling out: About a third of models don't support caching at all. That makes cache support a model selection criterion, not just a configuration step. The median discount is exactly 10x , but the range runs from 1.0x to 120x. The 1.0x entries — a published cache price identical to fresh input — turned out to be small open-weight models like Granite 8B and gpt-oss-20b , where input is already nearly free. Not traps, just models where caching is economically pointless. 70% of models publishing a write price charge a premium for it. This appears to be recent: OpenAI models before the GPT-5.6 family charged nothing for cache writes, and newer ones can. Minimum prefix length. OpenAI skips caching below 1,024 tokens 2,048 on older models . Below the bar you pay full price and get no error — cached tokens just returns 0. My original test prompt was 317 tokens and would have gotten nothing. I added a guard to my script specifically because I nearly published savings numbers for a prompt that could never have been cached. Short TTL. Cache entries expire after roughly five minutes of inactivity Anthropic offers a paid one-hour window . There's a subtle trap here: Anthropic measures the lifetime from the start of the request, and generation time counts against it. A response that streams for four minutes leaves about one minute for the follow-up to land a hit. Routing you don't control. OpenAI's cached states live on individual machines. Above roughly 15 requests per minute, overflow routing can send your request to a machine with no matching entry. There's a prompt cache key parameter to improve that. Your hit rate is partly an infrastructure property, not purely a property of your prompt. I measured three cost levers on the same workload: | Lever | Impact | Quality cost | |---|---|---| | Switch to a cheaper model | up to 26x | changes answers | Cache a stable prefix | 39–90% | none | | Trim retrieved context 60% | ~10% | changes answers | Caching is the only one on that list that costs you nothing in quality. Trimming context can change answers. Switching models definitely changes answers. Caching is the same tokens at a lower price. Yet it's the one absent from every "reduce your LLM costs" article I've read, all of which are about prompt engineering — the lever measured at about 10%. Here's the whole verification, about a cent to run: python import os, time, requests KEY = os.environ "OPENROUTER API KEY" UNIT = "The customer success team reviews consumption trends weekly to " "identify accounts where adoption has plateaued or declined. " PREFIX = UNIT 45 ~2,600 tokens, comfortably over the 1,024 minimum def go label, question, use cache control=False : system = {"type": "text", "text": PREFIX, "cache control": {"type": "ephemeral"}} if use cache control else PREFIX body = { "model": "anthropic/claude-sonnet-4.5", "max tokens": 20, "usage": {"include": True}, "messages": {"role": "system", "content": system}, {"role": "user", "content": question}, , } u = requests.post "https://openrouter.ai/api/v1/chat/completions", headers={"Authorization": f"Bearer {KEY}"}, json=body, timeout=120, .json .get "usage", {} d = u.get "prompt tokens details" or {} print f"{label}: input={u.get 'prompt tokens' } " f"cached={d.get 'cached tokens', 0 } " f"written={d.get 'cache write tokens', 0 } " f"cost=${u.get 'cost', 0 :.8f}" go "cold", "Summarize in one sentence.", use cache control=True time.sleep 2 go "warm", "Summarize in five words.", use cache control=True Flip use cache control to False and run it again. The gap between those two runs is what you've been leaving on the table. The key thing to check is whether cached tokens comes back non-zero. Whether you should be getting cache hits and whether you are getting them are different questions, and only one of them is answerable from a pricing page. If you run this and see something different from what I measured, I'd genuinely like to hear about it. Every LLM cost comparison uses two prices. There's a third — cached input, typically 10x cheaper — and for RAG and agent workloads that's where most of the money is. But only if your prompt clears the minimum, your prefix is byte-identical, you're inside the TTL, and your provider doesn't require you to ask.