{"slug": "prompt-caching-your-prompt-s-prefix-is-a-cache-key", "title": "Prompt Caching: Your Prompt's Prefix Is a Cache Key", "summary": "A developer's technical writeup explains that prompt caching keys on an exact token prefix, so prompt assembly order determines whether requests hit the cache. It details Anthropic's opt-in cache_control breakpoints (up to four, ordered tools then system then messages) versus OpenAI's automatic caching above a 1,024-token minimum with an optional prompt_cache_key, and notes cache writes cost 1.25x (5-minute TTL) or 2x (1-hour) while reads cost about 0.1x base input. A worked example shows 10 requests over a 6,000-token static block costing roughly 12,900 token-equivalents with caching versus 60,000 without.", "body_md": "Look at the input token count on a typical LLM feature and most of it is the same on every request. The system prompt, the tool definitions, the twelve few-shot examples, the product docs you stuffed in for grounding: thousands of tokens, identical every time, paid for every time. The user's actual question is a rounding error at the end.\n\nPrompt caching exists for exactly this. The provider saves the model's processed state for the start of your prompt, and the next request that begins with the same tokens skips that work. It's cheaper and faster. The catch is that it matches on an exact prefix, which means the order you assemble your prompt in suddenly matters a lot more than it used to.\n\nThe cache key is the prompt itself, read from the beginning. If request two starts with the exact same tokens as request one, up to a cache point, it can reuse request one's work. Change one token near the top, a timestamp, a user's name, a reshuffled tool list, and everything after that token is a miss.\n\nBoth major APIs follow this model, with different defaults. On Anthropic's Messages API you opt in with `cache_control`. You can either drop a single top-level `cache_control` and let the API place the breakpoint on the last cacheable block (it moves forward as a conversation grows), or put `cache_control` on specific blocks yourself, up to four breakpoints. Prefixes are built in a fixed order: tools, then system, then messages. Changing a tool definition invalidates the whole cache; changing a message only invalidates the messages level.\n\nOpenAI caches automatically once a prefix crosses the minimum length (1,024 tokens on GPT-5.6 and later), and lets you pass a `prompt_cache_key` to improve routing so similar requests land where the cache lives. Newer models also support explicit breakpoints.\n\nThe pricing shape is similar on both. Writing to the cache costs a bit more than normal input (1.25x for Anthropic's default 5-minute TTL, 2x for the 1-hour option), and reading from it costs a fraction: 0.1x base input on most models, lower on some newer ones. Check your model's pricing page before you build a budget around it.\n\nHere's a support assistant using Anthropic's TypeScript SDK. The stable content goes first and carries the breakpoint; anything that changes per request goes after it.\n\n``` python\nimport Anthropic from \"@anthropic-ai/sdk\";\n\nconst client = new Anthropic();\n\nexport async function answer(\n  question: string,\n  user: { name: string; plan: string },\n) {\n  const response = await client.messages.create({\n    model: \"claude-sonnet-5-5\",\n    max_tokens: 1024,\n    tools: SUPPORT_TOOLS, // stable, same order every request\n    system: [\n      {\n        type: \"text\",\n        text: SUPPORT_INSTRUCTIONS + PRODUCT_DOCS, // thousands of static tokens\n        cache_control: { type: \"ephemeral\" },\n      },\n    ],\n    messages: [\n      {\n        role: \"user\",\n        // per-request context lives AFTER the breakpoint\n        content: `Customer: ${user.name} (${user.plan} plan)\\n\\n${question}`,\n      },\n    ],\n  });\n\n  const { cache_creation_input_tokens, cache_read_input_tokens, input_tokens } =\n    response.usage;\n  console.log({\n    cache_creation_input_tokens,\n    cache_read_input_tokens,\n    input_tokens,\n  });\n  return response;\n}\n```\n\nThe tempting version puts `Customer: ${user.name}` at the top of the system prompt because it \"reads better.\" That one line makes every customer's prefix unique, and the cache never hits across users.\n\nDo the math on why ordering is worth caring about. Say the static block is 6,000 tokens and you get 10 requests inside the TTL window. Without caching you pay for 60,000 input tokens. With caching you pay one write at 1.25x (7,500 token-equivalents) plus nine reads at 0.1x (5,400), so 12,900 total. That's roughly 21% of the uncached cost for the static portion, and the time to first token drops too, because the model isn't reprocessing those tokens.\n\nCaching fails silently. A prompt that misses the cache still works; it just costs full price. So the most important habit is logging the usage fields. On Anthropic, if both `cache_creation_input_tokens` and `cache_read_input_tokens` are 0, nothing was cached. On OpenAI, look at `cached_tokens` under `usage.input_tokens_details` (Responses API) or `usage.prompt_tokens_details` (Chat Completions).\n\nThe usual culprits:\n\n`cache_control` is ignored with no error. On Anthropic it ranges from 512 tokens on the newest models up to 4,096 on some older ones.`tool_choice`, images, or thinking settings invalidates part of the cache. Pick them per feature, not per request, where you can.\nThat last one is worth taking seriously. If your requests are sparse and your prompts are short, caching adds a write surcharge and complexity for no gain. It pays off for chat, agents with long tool lists, RAG over a stable corpus, and batch jobs that hit the same instructions thousands of times.\n\nStart by logging cache fields on every call for a day. If the read numbers are near zero, the fix is almost always ordering: stable things first, changing things last, and nothing clever in between. Anthropic's [prompt caching docs](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) and OpenAI's [prompt caching guide](https://developers.openai.com/api/docs/guides/prompt-caching) both have per-model minimums and pricing tables worth bookmarking.", "url": "https://wpnews.pro/news/prompt-caching-your-prompt-s-prefix-is-a-cache-key", "canonical_source": "https://dev.to/grimicorn/prompt-caching-your-prompts-prefix-is-a-cache-key-1im7", "published_at": "2026-10-31 10:11:38+00:00", "updated_at": "2026-10-31 10:19:35.494421+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools", "developer-tools"], "entities": ["Anthropic", "OpenAI", "Claude Sonnet", "GPT-5.6", "Messages API"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/prompt-caching-your-prompt-s-prefix-is-a-cache-key", "markdown": "https://wpnews.pro/news/prompt-caching-your-prompt-s-prefix-is-a-cache-key.md", "text": "https://wpnews.pro/news/prompt-caching-your-prompt-s-prefix-is-a-cache-key.txt", "jsonld": "https://wpnews.pro/news/prompt-caching-your-prompt-s-prefix-is-a-cache-key.jsonld"}}