Prompt Caching: Your Prompt's Prefix Is a Cache Key

A developer's technical writeup explains that prompt caching keys on an exact token prefix, so prompt assembly order determines whether requests hit the cache. It details Anthropic's opt-in cache_control breakpoints (up to four, ordered tools then system then messages) versus OpenAI's automatic caching above a 1,024-token minimum with an optional prompt_cache_key, and notes cache writes cost 1.25x (5-minute TTL) or 2x (1-hour) while reads cost about 0.1x base input. A worked example shows 10 requests over a 6,000-token static block costing roughly 12,900 token-equivalents with caching versus 60,000 without.

Look at the input token count on a typical LLM feature and most of it is the same on every request. The system prompt, the tool definitions, the twelve few-shot examples, the product docs you stuffed in for grounding: thousands of tokens, identical every time, paid for every time. The user's actual question is a rounding error at the end. Prompt caching exists for exactly this. The provider saves the model's processed state for the start of your prompt, and the next request that begins with the same tokens skips that work. It's cheaper and faster. The catch is that it matches on an exact prefix, which means the order you assemble your prompt in suddenly matters a lot more than it used to. The cache key is the prompt itself, read from the beginning. If request two starts with the exact same tokens as request one, up to a cache point, it can reuse request one's work. Change one token near the top, a timestamp, a user's name, a reshuffled tool list, and everything after that token is a miss. Both major APIs follow this model, with different defaults. On Anthropic's Messages API you opt in with cache control . You can either drop a single top-level cache control and let the API place the breakpoint on the last cacheable block it moves forward as a conversation grows , or put cache control on specific blocks yourself, up to four breakpoints. Prefixes are built in a fixed order: tools, then system, then messages. Changing a tool definition invalidates the whole cache; changing a message only invalidates the messages level. OpenAI caches automatically once a prefix crosses the minimum length 1,024 tokens on GPT-5.6 and later , and lets you pass a prompt cache key to improve routing so similar requests land where the cache lives. Newer models also support explicit breakpoints. The pricing shape is similar on both. Writing to the cache costs a bit more than normal input 1.25x for Anthropic's default 5-minute TTL, 2x for the 1-hour option , and reading from it costs a fraction: 0.1x base input on most models, lower on some newer ones. Check your model's pricing page before you build a budget around it. Here's a support assistant using Anthropic's TypeScript SDK. The stable content goes first and carries the breakpoint; anything that changes per request goes after it. python import Anthropic from "@anthropic-ai/sdk"; const client = new Anthropic ; export async function answer question: string, user: { name: string; plan: string }, { const response = await client.messages.create { model: "claude-sonnet-5-5", max tokens: 1024, tools: SUPPORT TOOLS, // stable, same order every request system: { type: "text", text: SUPPORT INSTRUCTIONS + PRODUCT DOCS, // thousands of static tokens cache control: { type: "ephemeral" }, }, , messages: { role: "user", // per-request context lives AFTER the breakpoint content: Customer: ${user.name} ${user.plan} plan \n\n${question} , }, , } ; const { cache creation input tokens, cache read input tokens, input tokens } = response.usage; console.log { cache creation input tokens, cache read input tokens, input tokens, } ; return response; } The tempting version puts Customer: ${user.name} at the top of the system prompt because it "reads better." That one line makes every customer's prefix unique, and the cache never hits across users. Do the math on why ordering is worth caring about. Say the static block is 6,000 tokens and you get 10 requests inside the TTL window. Without caching you pay for 60,000 input tokens. With caching you pay one write at 1.25x 7,500 token-equivalents plus nine reads at 0.1x 5,400 , so 12,900 total. That's roughly 21% of the uncached cost for the static portion, and the time to first token drops too, because the model isn't reprocessing those tokens. Caching fails silently. A prompt that misses the cache still works; it just costs full price. So the most important habit is logging the usage fields. On Anthropic, if both cache creation input tokens and cache read input tokens are 0, nothing was cached. On OpenAI, look at cached tokens under usage.input tokens details Responses API or usage.prompt tokens details Chat Completions . The usual culprits: cache control is ignored with no error. On Anthropic it ranges from 512 tokens on the newest models up to 4,096 on some older ones. tool choice , images, or thinking settings invalidates part of the cache. Pick them per feature, not per request, where you can. That last one is worth taking seriously. If your requests are sparse and your prompts are short, caching adds a write surcharge and complexity for no gain. It pays off for chat, agents with long tool lists, RAG over a stable corpus, and batch jobs that hit the same instructions thousands of times. Start by logging cache fields on every call for a day. If the read numbers are near zero, the fix is almost always ordering: stable things first, changing things last, and nothing clever in between. Anthropic's prompt caching docs https://platform.claude.com/docs/en/build-with-claude/prompt-caching and OpenAI's prompt caching guide https://developers.openai.com/api/docs/guides/prompt-caching both have per-model minimums and pricing tables worth bookmarking.