Prompt Caching on the Claude API: 12 Steps, 60 Min [2026]

Prompt caching lets the Claude API reuse a long, unchanged prompt prefix instead of billing and processing it from scratch on every call. Anthropic’s pricing documentation lists cache reads at a small fraction of the normal input rate (generally 0.1x, with model-specific exceptions reported for newer models), and a cache write costs a premium over a plain input token. For any app that resends the same system prompt, tool definitions or reference document, that gap is the single biggest line item you can cut.

Pricing keeps moving, which is why the numbers in this tutorial live in a config file instead of the code. Anthropic cut Sonnet 5.5 cache-read pricing by half, from $0.20 to $0.10 per million tokens, according to its own announcement, and shipped Haiku 5.5 with its own cache prices. We covered the cheaper small model in our look at the Claude Haiku 5.5 API price cut. The project below turns caching on, proves it works from the response metadata, prices every call, and flags the silent mistakes that make a cache miss while you still pay for a write.

Plan on about 60 minutes if you already have an Anthropic API key. The finished repository holds a baseline call, cache breakpoints on system prompt, tools and conversation history, a benchmark harness that compares cached and uncached runs, a prefix guard that catches cache-busting changes, and a cost report you can paste into a pull request.

Don't miss new tech stories on Google

Add Shattered once in the Google app and our stories show up in your news suggestions.

Add now

How prompt caching works and why it pays #

A large language model processes your prompt as a sequence of tokens, and the expensive part is the attention work done on every token before it writes the first word of the answer. When two requests start with exactly the same tokens, that work is identical. Prompt caching stores the processed state of the shared prefix on the provider’s side, so the second request skips straight to the part that changed. You pay a reduced rate for the reused tokens, and the response usually starts sooner.

The mechanics on the Claude API are explicit. You mark a block with a cache_control field, and everything from the start of the request up to and including that block becomes a cacheable prefix. The first request that sees a new prefix writes the cache, the later requests inside the cache lifetime read it. Anthropic’s documentation, updated October 30, 2026 according to our research, lists two lifetimes: 5 minutes and 1 hour. Each read within the window refreshes the short lifetime, which is why a busy endpoint keeps its cache warm for free while an idle one pays the write again.

Three numbers drive the economics, and you should confirm all three on the current pricing page before you trust any estimate. The cache-read multiplier is generally reported at 0.1x of the base input price. The 5-minute write is reported at 1.25x, and the 1-hour write at 2x. Reported figures for Sonnet 5.5 suggest a read price of $0.10 per million tokens after the October cut, against a $2 per million base input price, which would put reads at roughly 0.05x. Treat that as a reason to read prices per model, never from a global constant.

The break-even point is short. If a 5-minute write costs 1.25x and a read costs 0.1x, the cache pays for itself on the second request that lands inside the window, because one write plus one read costs 1.35x while two uncached calls cost 2x. With a 1-hour write at 2x, you need a second read before you come out ahead, which makes the long lifetime a poor fit for prompts you hit once or twice an hour. Our ChatGPT vs Claude pricing breakdown shows how those API rates sit next to the consumer plans.

Caching is not free in every sense. It changes how you must build prompts, because a single changed character early in the prefix invalidates everything after it. Timestamps, request IDs, shuffled tool lists and per-user greetings placed at the top of a system prompt are the classic ways to turn a 90 percent discount into a 0 percent discount. Most of this tutorial is about building prompts in a stable order and measuring whether the cache actually hit.

Other providers offer the same idea with different controls. OpenAI’s announcement of GPT-6 Sol and Luna on November 2, 2026 reportedly promised higher cache-hit rates by default and a 90 percent discount on cached input reads. Google announced 95 percent off cached input for Gemini 4 Argon on October 13, 2026. OpenRouter documents cache pricing with separate write and read prices. Step 12 compares those mechanisms, but the working project targets the Claude API because that is where explicit control gives you the most to learn.

What gets cached and what does not

Only the prefix is cacheable. The order is tools first, then the system prompt, then the conversation messages. If you change a tool definition, the system prompt and the messages after it lose their cache. If you change the system prompt, the messages lose theirs, but the tools survive. If you change only the newest user message, everything before it still hits. Design your requests so the things that change least sit at the top and the things that change most sit at the bottom.

Output tokens are never cached. The reduced rate applies only to input tokens that match a stored prefix, and a model’s answer is generated fresh every time. A caching project therefore helps most on workloads with a high ratio of input to output, such as document question answering, code review over a large repository, long agent sessions and support bots with a thick policy manual.

Prerequisites and versions #

You need a machine with Python, a terminal and an Anthropic API key with billing enabled. The commands below assume Linux or macOS and also work in WSL on Windows. Nothing here requires a GPU, since all the heavy lifting happens on the provider’s servers.

Requirement	What to use	How to check
Python	Not listed	python3 –version
Anthropic SDK	Latest release of the anthropic package	pip show anthropic
Model	A current Claude model for the main runs and a smaller one for cheap tests, set through one environment variable	Listed on the Anthropic models page
Account	API key with a funded balance, kept in an environment variable	echo ${ANTHROPIC_API_KEY:+set}
Disk and RAM	A few megabytes, any laptop works	None
Time and budget	About 60 minutes, a few dollars of API spend at most	Set a spend limit in the console first

Model identifiers come from Anthropic’s published model names as relayed in our research: Sonnet 5.5 is claude-sonnet-5-5 and Haiku 5.5 is claude-haiku-5-5. Opus 5.5 is the newest flagship, announced on September 22, 2026, but the working project skips it because a benchmark that repeats the same big prompt dozens of times is cheaper on a smaller model. Swap the identifier in one config line when you want to test another.

We deliberately pin no library version numbers beyond “latest”. The SDK moves quickly, and the fields you need (cache_control on content blocks and the cache counters on the usage object) have been stable across recent releases. If your installed SDK is old enough to lack them, pip install --upgrade anthropic fixes it, and Step 4 includes a check that tells you immediately.

Before you spend anything, set a monthly spend cap in the Anthropic console and use a dedicated key for this project. A benchmark loop with a bug can fire hundreds of calls, and the cheapest insurance is a limit you set before you start. For wider context on how the AI tooling market is evolving, our AI and machine learning coverage tracks model launches and API changes week by week.

Steps 1 to 3: project, key and uncached baseline #

You cannot measure a saving without a baseline, so the first three steps build the smallest honest uncached call and record what it costs. Resist the urge to skip ahead to the cache_control field. Every later number in this tutorial is a comparison against what you capture here.

Step 1: create the project and install the SDK

Create a folder, a virtual environment and a requirements file. Keeping the environment local means the benchmark script behaves the same on your laptop and in CI.

mkdir prompt-cache-lab && cd prompt-cache-lab
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade anthropic
pip freeze | grep -i anthropic > requirements.txt
mkdir -p lab data reports
touch lab/__init__.py

Step 2: store the key and the model settings

Never paste a key into source code. Export it in your shell, and put model names and prices in a small config module that reads from the environment with safe defaults. The price defaults below are round example numbers, not quotes from any pricing page. Overwrite them with the values you read on the current Anthropic pricing page for your model.

export ANTHROPIC_API_KEY="sk-ant-your-key-here"
export CACHE_LAB_MODEL="claude-sonnet-5-5"
export BASE_INPUT_PER_MTOK="2.00"
export OUTPUT_PER_MTOK="10.00"
export CACHE_READ_PER_MTOK="0.20"
export CACHE_WRITE_5M_PER_MTOK="2.50"
export CACHE_WRITE_1H_PER_MTOK="4.00"
python
import os

MODEL = os.environ.get("CACHE_LAB_MODEL", "claude-sonnet-5-5")

def _price(name, default):
    return float(os.environ.get(name, default))

PRICES = {
    "input": _price("BASE_INPUT_PER_MTOK", "2.00"),
    "output": _price("OUTPUT_PER_MTOK", "10.00"),
    "cache_read": _price("CACHE_READ_PER_MTOK", "0.20"),
    "cache_write_5m": _price("CACHE_WRITE_5M_PER_MTOK", "2.50"),
    "cache_write_1h": _price("CACHE_WRITE_1H_PER_MTOK", "4.00"),
}
MAX_OUTPUT_TOKENS = 300

The defaults line up with the multipliers described earlier (reads at 0.1x, 5-minute writes at 1.25x, 1-hour writes at 2x) applied to a $2 base price. Sources relayed in our research suggest the real Sonnet 5.5 read price is lower than 0.1x after October 7, so your measured savings may beat the example.

Step 3: build a large, stable document and call the API with no cache

Caching only engages above a minimum prefix length that differs by model, and Anthropic documents the exact thresholds. A tiny prompt will quietly never cache, which is the number one reason beginners think the feature is broken. So the project generates a long, deterministic policy handbook that is safely above any threshold. Real projects would load a contract, a codebase summary or a knowledge base article instead.

SECTIONS = [
    ("Refunds", "Customers may request a refund within the stated window. Agents verify the order, check the item condition, and issue the refund to the original payment method. Exceptions need a team lead."),
    ("Shipping", "Standard shipping uses tracked carriers. Agents quote delivery windows from the carrier table, never from memory, and escalate lost parcels after the stated number of days."),
    ("Security", "Agents never ask for full card numbers or passwords. Account changes require the verified email on file. Suspicious requests go to the fraud queue with the full ticket text."),
    ("Privacy", "Personal data is shared only with the ticket owner. Deletion requests follow the retention schedule. Agents log every export of customer data."),
    ("Warranty", "Hardware is covered for the stated period. Water damage and unauthorized repairs void coverage. Agents collect serial numbers before opening a repair case."),
    ("Escalation", "Issues involving safety, legal threats or repeated failures go to a senior agent within one business hour. Agents summarize the case in three lines."),
]

def build_handbook(repeats: int = 40) -> str:
    parts = []
    for n in range(repeats):
        for title, body in SECTIONS:
            parts.append(f"Policy {n + 1}.{title}: {body}")
    return "\n\n".join(parts)

if __name__ == "__main__":
    text = build_handbook()
    print(len(text.split()), "words")

With 40 repeats the handbook runs to roughly 5,000 words, which is on the order of several thousand tokens. Run python -m lab.corpus to see the word count, and raise the repeat count if a later step reports zero cache writes. The baseline client below sends the handbook as a plain system string with no cache marker, then prints the usage counters.

import anthropic
from lab.config import MODEL, MAX_OUTPUT_TOKENS
from lab.corpus import build_handbook

client = anthropic.Anthropic()
HANDBOOK = build_handbook()
SYSTEM = "You are a support analyst. Answer only from the handbook.\n\n" + HANDBOOK

def ask(question: str):
    return client.messages.create(
        model=MODEL,
        max_tokens=MAX_OUTPUT_TOKENS,
        system=SYSTEM,
        messages=[{"role": "user", "content": question}],
    )

if __name__ == "__main__":
    r = ask("What is the escalation rule for legal threats?")
    print(r.content[0].text[:200])
    print(r.usage)

Run it with python -m lab.baseline. The usage object should show a large input_tokens value, and the cache counters should read zero or be absent. Write that input token count down. It is the size of the prefix you are about to cache, and it feeds the cost model in Step 9.

Steps 4 and 5: add the first cache breakpoint and read the usage fields #

Now the same call gets one change. The system prompt moves from a plain string to a list of content blocks, and the last block carries a cache_control marker. Everything up to and including that block becomes the cacheable prefix. The model, the question and the output limit stay exactly as in the baseline, so any difference in the usage numbers comes from caching alone.

Step 4: mark the system prompt as cacheable

The system text is split in two. The short role line sits first, and the long handbook follows in its own block with the marker. Both blocks together form the prefix, so editing the role line would invalidate the cache. That is intentional here, since the role line is static.

import anthropic
from lab.config import MODEL, MAX_OUTPUT_TOKENS
from lab.corpus import build_handbook

client = anthropic.Anthropic()
HANDBOOK = build_handbook()

def system_blocks(ttl: str = "5m"):
    marker = {"type": "ephemeral"}
    if ttl == "1h":
        marker = {"type": "ephemeral", "ttl": "1h"}
    return [
        {"type": "text", "text": "You are a support analyst. Answer only from the handbook."},
        {"type": "text", "text": HANDBOOK, "cache_control": marker},
    ]

def ask(question: str, ttl: str = "5m"):
    return client.messages.create(
        model=MODEL,
        max_tokens=MAX_OUTPUT_TOKENS,
        system=system_blocks(ttl),
        messages=[{"role": "user", "content": question}],
    )

if __name__ == "__main__":
    for q in ["What is the escalation rule for legal threats?",
              "Who can receive personal data?"]:
        r = ask(q)
        u = r.usage
        print(q)
        print("  input:", u.input_tokens,
              "write:", getattr(u, "cache_creation_input_tokens", None),
              "read:", getattr(u, "cache_read_input_tokens", None),
              "output:", u.output_tokens)

Step 5: prove the hit from the response metadata

Never trust that caching worked because the code ran without errors. The response carries three input counters that tell the full story. input_tokens counts the tokens after the last cache breakpoint, which were not served from or written to the cache. cache_creation_input_tokens counts tokens written to the cache on this call. cache_read_input_tokens counts tokens served from the cache. The sum of the three is your true prompt size.

Run python -m lab.cached twice within five minutes. A healthy first run looks like the sample below, and a healthy second run flips the write counter into the read counter. The values here are illustrative, so your token counts will differ with your handbook length.

What is the escalation rule for legal threats?
  input: 21 write: 7430 read: 0 output: 64
Who can receive personal data?
  input: 17 write: 0 read: 7430 output: 58

That pattern is the target: one write, then reads of exactly the same size. If the second question also shows a write, the prefix changed between calls or the cache lifetime expired. If both show zero for write and read, the prefix fell under the model’s minimum cacheable length or the marker never reached the API. The troubleshooting section lists every cause with a fix.

Add one small safety check so the project fails loudly on an SDK that drops the fields. The helper below returns a normalized dictionary, raising an error if the counters are missing entirely.

def read_usage(resp) -> dict:
    u = resp.usage
    if not hasattr(u, "cache_read_input_tokens"):
        raise RuntimeError("SDK too old: upgrade the anthropic package")
    return {
        "fresh": u.input_tokens or 0,
        "write": getattr(u, "cache_creation_input_tokens", 0) or 0,
        "read": getattr(u, "cache_read_input_tokens", 0) or 0,
        "output": u.output_tokens or 0,
    }

Anthropic also describes how its own coding tool relies on this mechanism. The company’s Claude Code prompt caching page explains why a stable system prompt and tool list matter for long sessions, and it is a good second read once your harness works.

Step 6: design the prefix so it stays stable #

Marking a block is the easy part. Keeping the bytes before the marker identical on every request is the real work, and it decides whether you save 90 percent or nothing. Treat the prefix like a compiled artifact. It is built once, hashed, versioned and never edited at request time.

The stable-prefix rules

• Order matters. Put tools first, then the system prompt, then static reference material, then conversation history, and put the fresh user message last.
• Keep dynamic values out of the prefix. Dates, user names, request IDs, feature flags and A/B variants belong in the final user message, not in the system prompt.
• Sort anything that comes from a set or a dictionary. A tool list built from an unordered collection can reorder between processes and silently bust the cache.
• Serialize JSON the same way every time. Use fixed key order and fixed whitespace for any schema you embed.
• Change prompts through releases, not at runtime. A prompt edit is a deliberate cache flush, so ship it on purpose and expect one write at the old price.

Anthropic announced on May 28, 2026 that the Messages API lets you place system entries inside the messages array, which allows instruction changes mid-conversation without breaking the cache for the content before them, according to the sources behind this article. If your product needs to inject late instructions, such as a policy reminder after the user uploads a file, that route is better than editing the top-level system prompt.

Add tool definitions to the cached prefix

Tools sit before the system prompt in the cache order, so they are the most expensive thing to change. Put the marker on the last tool in the list, and everything above it is cached together. The example below defines two small tools in a fixed order.

TOOLS = [
    {
        "name": "lookup_order",
        "description": "Look up an order by its numeric ID and return status and items.",
        "input_schema": {
            "type": "object",
            "properties": {"order_id": {"type": "integer"}},
            "required": ["order_id"],
        },
    },
    {
        "name": "open_case",
        "description": "Open a support case with a short summary and a priority.",
        "input_schema": {
            "type": "object",
            "properties": {
                "summary": {"type": "string"},
                "priority": {"type": "string", "enum": ["low", "normal", "high"]},
            },
            "required": ["summary", "priority"],
        },
        "cache_control": {"type": "ephemeral"},
    },
]

Pass tools=TOOLS to messages.create alongside the system blocks from Step 4. With both markers present, you get two cache entries: one ending at the tools and one ending at the handbook. A later edit to the handbook then keeps the tool cache alive, which matters when your tool list is large.

Anthropic limits how many breakpoints one request can carry, and the documented ceiling is small. Spend them on the boundaries that change at different speeds: tools, system and conversation history. A marker on every paragraph wastes the budget and adds no savings.

Step 7: cache a multi-turn conversation #

Single questions against a big document are the easy case. Chat apps and agents hit the API with a growing message list, and each turn resends everything before it. That is the second-biggest caching win after static documents, because the earlier turns never change.

The pattern is to move the breakpoint forward each turn. After the assistant answers, you append the answer and the next user message, and you mark the newest user block. The cache then covers the whole earlier conversation plus the system prompt. Each new call reads the previous prefix and writes only the incremental tokens.

import anthropic
from lab.config import MODEL, MAX_OUTPUT_TOKENS
from lab.cached import system_blocks
from lab.usage import read_usage

client = anthropic.Anthropic()

class CachedChat:
    def __init__(self, ttl: str = "5m"):
        self.ttl = ttl
        self.history = []
        self.log = []

    def _marked(self, text: str):
        marker = {"type": "ephemeral"}
        if self.ttl == "1h":
            marker = {"type": "ephemeral", "ttl": "1h"}
        return [{"type": "text", "text": text, "cache_control": marker}]

    def say(self, text: str) -> str:
        for m in self.history:
            if m["role"] == "user" and isinstance(m["content"], list):
                for block in m["content"]:
                    block.pop("cache_control", None)
        self.history.append({"role": "user", "content": self._marked(text)})
        resp = client.messages.create(
            model=MODEL,
            max_tokens=MAX_OUTPUT_TOKENS,
            system=system_blocks(self.ttl),
            messages=self.history,
        )
        answer = resp.content[0].text
        self.history.append({"role": "assistant", "content": answer})
        self.log.append(read_usage(resp))
        return answer

A subtle point: stripping the old marker does not break the cache. The cache keys on the content before a breakpoint, not on where earlier markers sat, and Anthropic’s docs describe a lookback behavior that finds earlier cached prefixes. Moving the marker forward each turn is the documented pattern for conversations, and it keeps you inside the breakpoint limit.

Run five turns and print chat.log. The first turn should write the handbook plus the first message. Turn two should read the handbook and the first exchange, then write only the new tokens. By turn five your read counter should dwarf the fresh counter. If instead every turn shows a full write, one of the earlier blocks is mutating, which is the subject of Step 11.

When you need a sanity check on routing decisions across models, our guide to building a Claude vs GPT router shows why sticky routing matters here. A router that bounces one conversation between models throws away the cache every time, since caches do not transfer between models.

Step 8: choose between the 5-minute and 1-hour lifetime #

The default lifetime is 5 minutes, and each cache read inside the window resets the clock. Anthropic’s documentation, updated October 30, 2026 according to our research, also offers a 1-hour option at a higher write price. Describing only the 5-minute policy is outdated now, so the project supports both through the ttl field you already wired into system_blocks.

The right choice comes down to request spacing. If the same prefix gets hit more often than every five minutes, the short lifetime renews itself and you pay the cheaper 1.25x write once. If gaps between requests run from six minutes to an hour, the short cache expires between calls, and you pay a write every time. The long lifetime costs more up front but survives those gaps.

Traffic pattern	Typical gap between hits	Better lifetime	Why
Busy chat endpoint	Seconds	5 minutes	Every read renews the window at the lower write price
Batch job with steady calls	Under 5 minutes	5 minutes	Cache stays warm for the whole run
Internal tool used a few times an hour	10 to 40 minutes	1 hour	Short cache expires between calls
Nightly report over one big document	Hours	Neither, or batch the calls together	A single write per day rarely pays back
Agent sessions with long tool waits	1 to 10 minutes	Test both	Depends on tool latency and reuse

A quick way to decide is the break-even formula. With a write multiplier W, a read multiplier R and N calls inside one window, caching beats no caching when W + (N – 1) x R is lower than N. For W = 1.25 and R = 0.1, that holds from N = 2 (1.35 against 2). For W = 2 and R = 0.1, N = 2 loses (2.1 against 2), and the cache only wins from N = 3 (2.2 against 3). Put real multipliers from your pricing page into the formula, because reported Sonnet 5.5 read prices are lower than 0.1x.

There is one more trap on the long lifetime. If you mix lifetimes in one request, Anthropic’s rules say the longer lifetime entries must come before the shorter ones in the prompt order. Mixing them casually can cause errors or misses, so most teams pick one lifetime per prefix and stay with it.

Test the choice with data instead of intuition. Step 10 adds a gap parameter to the benchmark so you can simulate a request every 30 seconds, every 6 minutes and every 20 minutes, then compare the totals.

Step 9: build a cost model that reads prices from config #

Raw token counters are hard to reason about, and dollars are easy. The cost module turns a normalized usage record into a price using the config values from Step 2. It never hard-codes a number, so a price change is a one-line environment edit.

from lab.config import PRICES

def call_cost(u: dict, ttl: str = "5m") -> float:
    per = 1_000_000
    write_key = "cache_write_1h" if ttl == "1h" else "cache_write_5m"
    return (
        u["fresh"] * PRICES["input"] / per
        + u["write"] * PRICES[write_key] / per
        + u["read"] * PRICES["cache_read"] / per
        + u["output"] * PRICES["output"] / per
    )

def uncached_equivalent(u: dict) -> float:
    per = 1_000_000
    prompt_tokens = u["fresh"] + u["write"] + u["read"]
    return (
        prompt_tokens * PRICES["input"] / per
        + u["output"] * PRICES["output"] / per
    )

def savings_pct(records: list, ttl: str = "5m") -> float:
    actual = sum(call_cost(r, ttl) for r in records)
    base = sum(uncached_equivalent(r) for r in records)
    return 0.0 if base == 0 else round(100 * (base - actual) / base, 2)

The second function answers the question finance will ask: what would these exact calls have cost without caching? It rebuilds the full prompt size from the three counters and prices it at the plain input rate. The gap between the two totals is your savings.

A worked example with round numbers

Take a 20,000-token prefix, 100 requests inside one warm window, a $2 per million base input price and the example multipliers. Without caching, the prefix costs 100 x 20,000 x $2 / 1,000,000, which is $4.00. With caching, one 5-minute write costs 20,000 x $2.50 / 1,000,000, or $0.05, and 99 reads cost 99 x 20,000 x $0.10 / 1,000,000. Add those two amounts and compare the sum with the uncached $4.00: the saving on the prefix alone is large.

Output tokens and the fresh tail of each prompt dilute that number in real life. A request with a 300-token answer at a $10 per million output price adds $0.003 per call, which is $0.30 across 100 calls and not affected by caching at all. Report savings on total spend, not only on the prefix, so the figure survives an audit. Our comparison of DeepSeek vs Claude API pricing is a useful reference when you want to see how cached prices change the ranking between vendors.

Third-party trackers also compare what each provider charges for cache reads and writes. The Benchclaw prompt caching comparison is one such page, but treat any outside table as a lead and verify it on the provider’s own pricing page before it goes into a budget.

Steps 10 and 11: benchmark harness and prefix guard #

Two small tools finish the core project. The benchmark runs the same questions with and without caching and writes a report. The prefix guard hashes everything that sits before your breakpoint and warns when the hash changes between calls, which catches the silent cache-busting bugs before they show up on an invoice.

Step 10: write the benchmark harness

The harness runs a fixed question list in two modes. In uncached mode it sends the baseline request. In cached mode it sends the Step 4 request. A gap argument sleeps between calls so you can model different traffic patterns. Keep the question list short while you debug, since each run spends real money.

import json, sys, time
from datetime import datetime, timezone
import anthropic
from lab.config import MODEL, MAX_OUTPUT_TOKENS
from lab import baseline, cached
from lab.usage import read_usage
from lab.cost import call_cost, uncached_equivalent, savings_pct

QUESTIONS = [
    "What is the escalation rule for legal threats?",
    "Who can receive personal data?",
    "What do agents collect before a warranty repair?",
    "How are lost parcels handled?",
    "What must agents never ask a customer for?",
    "What exceptions need a team lead?",
]

def run(mode: str, ttl: str, gap: float):
    records = []
    for q in QUESTIONS:
        t0 = time.perf_counter()
        resp = baseline.ask(q) if mode == "uncached" else cached.ask(q, ttl)
        elapsed = round(time.perf_counter() - t0, 3)
        rec = read_usage(resp)
        rec["seconds"] = elapsed
        rec["cost"] = call_cost(rec, ttl) if mode == "cached" else uncached_equivalent(rec)
        records.append(rec)
        time.sleep(gap)
    return records

def main():
    ttl = sys.argv[1] if len(sys.argv) > 1 else "5m"
    gap = float(sys.argv[2]) if len(sys.argv) > 2 else 1.0
    out = {"model": MODEL, "ttl": ttl, "gap": gap,
           "when": datetime.now(timezone.utc).isoformat()}
    for mode in ("uncached", "cached"):
        out[mode] = run(mode, ttl, gap)
    out["savings_pct"] = savings_pct(out["cached"], ttl)
    path = f"reports/bench-{ttl}-{int(gap)}s.json"
    with open(path, "w") as fh:
        json.dump(out, fh, indent=2)
    unc = sum(r["cost"] for r in out["uncached"])
    cac = sum(r["cost"] for r in out["cached"])
    print(f"uncached ${unc:.4f}  cached ${cac:.4f}  savings {out['savings_pct']}%")
    print("avg seconds uncached:", round(sum(r["seconds"] for r in out["uncached"]) / len(QUESTIONS), 2))
    print("avg seconds cached:  ", round(sum(r["seconds"] for r in out["cached"]) / len(QUESTIONS), 2))

if __name__ == "__main__":
    main()

Run it with python -m lab.bench 5m 1. The first number pair is the point of the whole exercise. Because the cached run pays a write on question one, a six-question run shows a smaller saving than a 100-question run. Raise the question count with repeats if you want to see the curve approach the worked-example figure.

Latency also matters, but treat it as noisy. Network jitter, server load and output length move response time more than caching does on short answers. Run the benchmark several times at different hours and compare medians instead of single runs. Never publish a latency claim from one execution.

Step 11: add a prefix guard that catches cache busters

Most cache misses come from a prefix that changed without anyone noticing. The guard below hashes the serialized tools and system blocks with fixed JSON formatting and keeps the last hash in memory. If the hash changes between two calls in the same process, it prints a warning with a diff hint. Wire it in front of every messages.create call.

import hashlib, json

_last = {"hash": None, "dump": None}

def prefix_hash(system, tools=None) -> tuple:
    dump = json.dumps({"tools": tools or [], "system": system},
                      sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(dump.encode("utf-8")).hexdigest()[:16], dump

def check_prefix(system, tools=None) -> bool:
    h, dump = prefix_hash(system, tools)
    changed = _last["hash"] is not None and h != _last["hash"]
    if changed:
        old = _last["dump"]
        i = next((n for n, (a, b) in enumerate(zip(old, dump)) if a != b), min(len(old), len(dump)))
        print(f"[guard] prefix changed {_last['hash']} -> {h} near char {i}: ...{dump[max(0, i - 30):i + 30]}...")
    _last["hash"], _last["dump"] = h, dump
    return not changed

The warning prints the position where the two serialized prefixes first differ, with thirty characters of context on each side. That is usually enough to spot a timestamp or an unsorted list. In production, send the hash to your metrics system instead of printing it, and alert when the hash changes outside a deploy window.

Pair the guard with the usage counters. A prefix hash that stays constant while cache_read_input_tokens drops to zero points to expiry, a model switch or a provider-side eviction. A hash that changes points to your code. That split cuts debugging time sharply.

Step 12: compare other providers and gateways #

The same idea exists elsewhere, but the controls differ, and moving a prompt between vendors without checking them is how teams end up paying full price. The table below compares mechanisms only. It carries no prices, because the figures reported in our research are single-source and change often, so read each vendor’s pricing page for the model you actually call.

Provider	How you turn it on	What you control	What to verify before you rely on it
Anthropic Claude API	Explicit cache_control markers on blocks	Breakpoint position, 5-minute or 1-hour lifetime	Per-model read and write prices, minimum prefix length
OpenAI API	Automatic reuse of repeated prefixes, per OpenAI’s announcements	Prompt structure and cache-related request settings in the Responses API docs	Cached-input price per model, how hits are reported in usage
Google Gemini API	Context caching, with support that depends on the model	Cache objects and their storage lifetime	Whether your model supports caching, storage charges
OpenRouter	Passes provider cache controls through to supported models	Provider choice and routing	Cache read and write price entries for the provider you land on

OpenAI described its November 2, 2026 GPT-6 release as bringing higher cache-hit rates by default, and its pricing pages separate ordinary input from cached input. Google announced 95 percent off cached input for Gemini 4 Argon on October 13, 2026, and its developer forum said on November 2, 2026 that the Gemini 3.8 Live model supports neither explicit nor implicit context caching. Those two facts show why a one-size-fits-all wrapper fails: support varies by model, not only by vendor.

OpenRouter documents cache pricing as separate entries for ordinary prompt tokens, cached prompt tokens and cache writes, with write prices that can differ by lifetime. Read its pricing documentation before you route cached Claude traffic through it, because cache savings only appear if the gateway forwards your markers and the provider it picks supports them.

If you run multi-vendor comparisons, test caching separately from quality. Our ChatGPT vs Claude vs Gemini API test harness shows how to log per-call usage for all three, and you can feed the same cached-versus-fresh counters into the cost module from Step 9. Keep one rule: compare vendors on the same prefix length and the same request spacing, or the numbers mean nothing.

A provider-neutral usage record

Normalize the counters early, so the rest of your code does not care which vendor produced them. The adapter below maps a Claude response into the same four-field record used above, and you can add sibling functions for other vendors as you test them.

def from_anthropic(resp) -> dict:
    u = resp.usage
    return {
        "fresh": u.input_tokens or 0,
        "write": getattr(u, "cache_creation_input_tokens", 0) or 0,
        "read": getattr(u, "cache_read_input_tokens", 0) or 0,
        "output": u.output_tokens or 0,
    }

def from_openai_style(usage: dict) -> dict:
    details = (usage.get("prompt_tokens_details") or {})
    cached = details.get("cached_tokens", 0) or 0
    prompt = usage.get("prompt_tokens", 0) or 0
    return {
        "fresh": max(prompt - cached, 0),
        "write": 0,
        "read": cached,
        "output": usage.get("completion_tokens", 0) or 0,
    }

Check the field names against the vendor’s current API reference before you ship, since usage schemas change. The goal of the adapter is only to keep vendor quirks in one file.

The working project: layout, report and full run #

Everything so far fits one small repository. The tree below lists every file, and the report script turns the JSON benchmark output into a table you can paste into a pull request or a ticket.

prompt-cache-lab/
  requirements.txt
  data/
  reports/
  lab/
    __init__.py
    config.py      # model and price settings from env
    corpus.py      # deterministic handbook generator
    baseline.py    # Step 3 uncached call
    cached.py      # Step 4 cache_control on system blocks
    usage.py       # Step 5 counter normalizer
    tools.py       # Step 6 cached tool list
    chat.py        # Step 7 multi-turn with a moving breakpoint
    cost.py        # Step 9 cost model
    bench.py       # Step 10 harness
    guard.py       # Step 11 prefix hash
    adapters.py    # Step 12 provider-neutral records
    report.py      # markdown report
python
import glob, json

def row(path):
    d = json.load(open(path))
    unc = sum(r["cost"] for r in d["uncached"])
    cac = sum(r["cost"] for r in d["cached"])
    reads = sum(r["read"] for r in d["cached"])
    writes = sum(r["write"] for r in d["cached"])
    return f"| {d['model']} | {d['ttl']} | {d['gap']}s | {writes} | {reads} | ${unc:.4f} | ${cac:.4f} | {d['savings_pct']}% |"

def main():
    print("| model | ttl | gap | written | read | uncached | cached | saved |")
    print("|---|---|---|---|---|---|---|---|")
    for p in sorted(glob.glob("reports/bench-*.json")):
        print(row(p))

if __name__ == "__main__":
    main()

Run the full sequence from the project root with the commands below. Start with the cheaper Haiku model if you want to debug at the lowest cost, then switch back to Sonnet for the numbers you plan to quote.

export CACHE_LAB_MODEL="claude-haiku-5-5"
python -m lab.corpus
python -m lab.baseline
python -m lab.cached
python -m lab.bench 5m 1
python -m lab.bench 1h 1
python -m lab.report

A successful report looks like the sample below. These values are illustrative, since your token counts, prices and savings depend on the model, the handbook length and the question count. What matters is the shape: writes appear once per mode, reads dominate, and the cached cost lands well under the uncached cost.

| model | ttl | gap | written | read | uncached | cached | saved |
|---|---|---|---|---|---|---|---|
| claude-haiku-5-5 | 5m | 1s | 7430 | 37150 | 0.0894 | 0.0211 | 76.4% |
| claude-haiku-5-5 | 1h | 1s | 7430 | 37150 | 0.0894 | 0.0236 | 73.6% |

Notice that the 1-hour run saves slightly less in this sample, because the write costs more and the questions all land inside one short window. A run with 6 to 20 minute gaps would reverse that ranking, and that reversal is the experiment worth running before you choose a lifetime for production.

Commit, version and review

Check the lab into version control, minus the reports folder if it holds anything sensitive. Treat the handbook text as part of the prefix: a change to it is a deliberate cache flush. Add a short note in your changelog whenever the prefix changes, so whoever reads next month’s invoice can match a spike to a release.

Common pitfalls that quietly kill the cache #

Most failed caching projects trace back to the same handful of mistakes. Each one leaves the code working and the answers correct, so nothing tells you anything is wrong except the bill.

1. Prompts below the minimum length. Each model has a minimum cacheable prefix. Below it, the marker is accepted and ignored, and both cache counters stay at zero. Check the threshold for your model in Anthropic’s docs and pad the stable part of the prompt only if it makes sense for the product.
2. Dynamic values in the prefix. A line such as “Today is November 4, 2026” at the top of the system prompt changes the prefix every day, and a per-request ID changes it every call. Move volatile text to the last user message.
3. Unstable ordering. Tool lists built from sets, dictionaries merged in different orders and JSON serialized with random key order all produce a different byte sequence for the same logical content. Sort and fix the format.
4. Switching models mid-conversation. Caches belong to one model. A router that sends turn one to Sonnet and turn two to Haiku writes two caches and reads neither. Keep a conversation pinned to one model, as covered in Step 7.
5. Marking one-shot prompts. A prompt used once pays the write premium and never earns a read. Cache only what you will reuse inside the lifetime, and measure before you add markers everywhere.
6. Expecting caches to cross boundaries. Do not assume a cache written under one API key, workspace or organization serves another. Anthropic documents isolation between organizations, and your own deployment may split traffic across keys without you noticing.
7. Ignoring policy and content rules. Cached text is still subject to the provider’s usage rules, and those rules change. Anthropic’s recent usage-policy change on arrest-related calls is a reminder to review long-lived system prompts when the policy shifts.

Troubleshooting: 11 symptoms and their fixes #

Use the usage counters from Step 5 as your compass. Each symptom below names what the counters show, the usual cause and the fix.

1. Both cache counters stay at zero. The prefix is probably under the model’s minimum cacheable length, or the marker sits on a block that is too short. Print the sum of the three input counters to see your real prompt size, then grow the stable part or move the marker to the end of the longest static block.
2. Every call shows a write and never a read. The prefix changes between calls. Turn on the Step 11 guard and look at the character position it reports. Timestamps, random IDs and unsorted lists are the usual culprits.
3. Reads work for a few minutes, then a write appears again. The 5-minute lifetime expired between calls. Either send traffic more often, add a keep-alive call, or switch the prefix to the 1-hour lifetime and rerun the Step 10 benchmark with a longer gap.
4. The SDK raises an error on the cache_control field. Your installed anthropic package is old, or the marker sits in a place the API does not accept. Runpip install --upgrade anthropic , and check that the marker is inside a content block dictionary, not at the top level of the request.
5. An error mentions the number of cache breakpoints. You exceeded the per-request limit. Keep markers on tools, system and the newest conversation block only, and strip old markers from history as the Step 7 class does.
6. Savings look much smaller than the worked example. Short benchmark runs pay one write across few reads, and output tokens are never discounted. Run more calls per window, and report savings on total spend with output included.
7. Reads vanish after you changed a tool description. Tools sit at the top of the cache order, so any edit flushes the system prompt and messages that follow. Batch tool changes into planned releases and expect one write at the new price.
8. Costs spike right after a deploy. A prompt edit flushed the cache on every worker at once, so each active prefix paid a write. Warm the cache with one cheap call per prefix before sending production traffic.
9. Hit rates differ between environments. Staging and production may use different API keys, workspaces or organizations, and caches do not cross those lines. Compare the usage counters with the same key and the same prefix hash.
10. The cost report disagrees with the invoice. Your config prices are probably stale or belong to a different model. Re-read the pricing page, update the environment values, and rerun the report. Remember that Anthropic’s pricing documentation changed again on November 4, 2026, according to a monitoring service, though that detection does not by itself prove any price moved.
11. Latency did not improve. Caching mainly cuts input cost, and the latency gain varies with prompt size and server load. Compare medians over many calls before you decide it failed.

If none of these match, log the full serialized request for one failing call and one succeeding call, then diff them byte by byte. The bug is nearly always visible in the diff.

Advanced tips for production caching #

Once the basics work, a few habits keep the savings from eroding as the product grows.

• Track a hit ratio. Compute reads divided by the sum of reads, writes and fresh input tokens, per prefix and per hour. A healthy steady-state endpoint trends toward 1.0, and a sudden drop is a deploy bug until proven otherwise.
• Pre-warm on deploy. Send one request withmax_tokens set to 1 for each long prefix right after release. The first real user then reads instead of paying the write and the extra wait.
• Keep hot prefixes warm. For prefixes with gaps just over five minutes, a cheap scheduled ping can cost less than repeated writes. Price the ping against the write premium before you add it.
• Cache instructions, not retrieved chunks. In retrieval pipelines, the retrieved passages change on every query, so they belong after the marker. Put the stable instructions, schemas and few-shot examples before it.
• Pin conversations to one model. Sticky routing protects the cache. If you must fail over to another model, accept one fresh write and log it as an event.
• Use per-tenant prefixes with care. A prefix that includes tenant-specific text creates a cache per tenant. That is fine for large tenants and wasteful for thousands of small ones, so group small tenants behind a shared prefix.
• Test gateways before trusting them. Route a hundred identical requests through your gateway and confirm the read counter climbs. A gateway that rewrites or reorders content blocks can erase your cache without any error.
• Watch for new server-side state features. A secondary source reported on November 2, 2026 that the Messages API gained a beta mode that keeps conversation state on Anthropic’s servers, with threads expiring five minutes after the last request. Anthropic’s own documentation was not confirmed in our research, so read it yourself before you change an architecture around it.

Cloud marketplaces add one more wrinkle. Anthropic reportedly gets close to half of its sales through three large clouds, as our report on Anthropic’s cloud revenue split shows, and feature support there can lag the first-party API. If you call Claude through a marketplace, rerun the Step 5 check on that endpoint and do not assume the same markers and lifetimes work.

The takeaway: caching pays when the prefix is large, stable and reused inside its lifetime, and it fails silently when any of those three breaks. Measure with the counters, price with a config you control, and guard the prefix like production code.

FAQ: prompt caching on the Claude API #

What is prompt caching?

Prompt caching stores the processed form of a repeated prompt prefix on the provider’s side, so later requests that start with the same tokens skip that work and pay a reduced rate for those tokens. On the Claude API you mark blocks with a cache_control field, and the response reports how many tokens were written to and read from the cache.

How much money can it save?

It depends on prefix size, request spacing and the model’s prices. In the worked example with round numbers, a 20,000-token prefix reused 100 times inside one window saved about 89 percent of the prefix cost. Real savings on total spend are lower, because output tokens and the uncached tail of each prompt are billed at normal rates. Anthropic advertises large caching discounts on its newer models, so check the current rates for yours.

How long does a cache last?

Anthropic’s documentation offers a short default lifetime and a longer option, and reads inside the short window refresh it. Confirm the current lifetimes and write prices on the pricing page for your model, since both can change.

Does caching change the model’s answers?

It should not. The model receives the same prompt either way, and caching only changes how the provider processes and bills the repeated prefix. Run your own evaluation set with and without markers if your product depends on exact output, and treat any difference as a bug to report.

How do I know the cache worked?

Read the usage object on the response. A first call should show tokens in the cache-write counter, and a later call with the same prefix should show those tokens in the cache-read counter. If both stay at zero, the prefix is probably too short or the marker did not reach the API.

Is cached data safe to use with private documents?

Treat it like any other prompt data. Read Anthropic’s data retention and privacy terms for your plan, check whether features such as zero data retention affect caching for your account, and avoid placing secrets in prompts that do not need them. The research behind this article did not confirm retention details, so do not assume them.

Does caching work on Haiku 5.5 and the cheaper models?

Anthropic publishes model-specific cache prices for its small models, so yes, with their own read and write rates. Each model keeps its own cache, and a cache written under one model never serves another.

Should I cache everything?

No. Cache prefixes that are large, stable and reused within the lifetime. A prompt used once pays the write premium for nothing, and a prefix that changes on every call writes a new entry each time. Start with your biggest repeated prefix, measure it with the harness above, and expand only where the counters show a payoff.

source & further reading

shattered.io — original article Claude vs ChatGPT Voice Match: 12 Steps, 80 Min [2026] Fake Job Interview Led to Binance’s $6.3B Loss [2026] Gemini 4 Carbon Tops AA Index at 59, $2 Input [2026]