Anthropic emailed our org last week: prompt cache hit rate is low, and caching repeated content "could save up to 64% of direct API spend." The email is careful with scope — direct API only, Claude Code excluded because it manages caching itself. I went to check what that 64% was made of.
Our direct API traffic over the period: 16 calls, $0.25 total, average prompt 90 tokens. They come from traceguard's rerun harness, which replays self-contained consults as fresh single-turn calls — every prompt unique by construction. A workload like that cannot hit cache: there is no repeated prefix across calls, and 90 tokens is far below the minimum Anthropic will cache at all (1,024 tokens on Opus 4.8, the model those calls ran on; 512 on Opus 5 / Fable 5). Hit rate 0%, structurally. 64% of $0.25 is $0.16.
The metric was right. The money wasn't there.
The interesting numbers sit in the traffic the email correctly excluded. We ingest our own Claude Code session logs into traceguard's trace store (Claude Code transcripts carry no cost field, so we compute list price from usage
). 71 days, 158 sessions, 58,753 API messages. This is section 1 of the command output, unedited:
| model | messages | prompt tok | hit rate | input cost | no-cache | saved | saved % |
|---|---|---|---|---|---|---|---|
| claude-opus-4-8 | 23,759 | 5,535,807,891 | 96.1% | $4,616.39 | $27,691.44 | $23,075.05 | 83.3% |
| claude-fable-5 | 15,291 | 3,583,420,473 | 96.2% | $5,847.08 | $35,834.20 | $29,987.13 | 83.7% |
| claude-opus-5 | 6,316 | 1,750,760,343 | 97.7% | $1,205.08 | $8,753.80 | $7,548.72 | 86.2% |
| claude-sonnet-5 | 11,139 | 1,326,223,701 | 95.1% | $422.96 | $2,652.45 | $2,229.49 | 84.1% |
| claude-haiku-4-5-20251001 | 1,573 | 57,312,161 | 93.9% | $9.76 | $57.31 | $47.56 | 83.0% |
| claude-opus-4-7 | 64 | 11,045,966 | 96.6% | $9.04 | $55.23 | $46.19 | 83.6% |
| claude-sonnet-4-5-20250929 | 2 | 42,410 | 0.0% | n/a | n/a | n/a | n/a |
| (none) | 609 | 0 | n/a | n/a | n/a | n/a | n/a |
| TOTAL | 58,753 | 12,264,612,945 | 96.2% | $12,110.31 | $75,044.44 | $62,934.13 | 83.9% |
Hit rate is token-weighted; costs are the input side only, at list price. The two n/a rows are deliberate — no published price for a model means no guessed money, tokens counted anyway.
Caching nobody at our org configured cuts the input side by 83.9%. With output included, the whole bill lands at 1/5.5 of the no-cache counterfactual. There is nothing left to optimize here; the remaining spend is mostly first-write premiums on genuinely new content.
There's a folk optimization going around: ping your session on a timer so the cache stays warm and the next turn hits. The mechanics say it could work — the cache TTL is 5 minutes by default, every hit refreshes it for free, and a 1-hour TTL costs 2× on writes. Instead of arguing, we ran the counterfactual on our own gap data.
97.3% of our in-session gaps are under 5 minutes; they need nothing. The addressable prize is 422 gaps longer than an hour: the context rewrites right after them cost at most $1,913 over 71 days (an overestimate, since it counts genuinely new content as rewrite). Bridging those same gaps with a ping every 55 minutes would have taken 6,765 pings costing $2,009 in cache reads alone, before counting the pings' own output tokens. The strategy loses money on our data, under assumptions tilted in its favor.
The reason is boring: the 1-hour TTL removed the ping's niche. Our logs show Claude Code already routes nearly all cache writes into the 1h bucket (opus-4-8: 155.9M tokens written at 1h TTL vs 56.4M at 5m). Pinging is a hack from before extended TTL existed. If a tool sells you auto-ping in 2026, ask it for the counterfactual on your logs first.
While writing this I checked our own SDK and found the same class of bug I keep filing against other trackers. traceguard's wrap_anthropic
recorded tokens_in = usage.input_tokens
. On Anthropic's API, input_tokens
excludes cache reads and writes — and on our traffic, cache reads run about 1,000× larger than raw input, so the wrapper under-counted by three orders of magnitude. Every field read correctly, the quantity still wrong (the splitrail #220 pattern). Fixed in PR #39, and the fix is what makes the next part possible.
A rate without a denominator is not a finding. Percentages travel well in emails; dollars make decisions. Before acting on a metric email, multiply it by the money it applies to.
This analysis is now one command, merged in PR #40: python -m traceguard.routing_audit.cache_audit
. Point it at your ingested Claude Code transcripts or a traceguard trace store, and get the table above for your own data — per-model hit rate, actual vs no-cache cost at list price, gap distribution, and a yes/no on whether keep-alive pings would pay for you (ours printed NOT WORTH IT). Ships in traceguard 1.2.0 — pip install -U traceguard
. The last line of ours came out like this:
Claude Code caching already saves us 84% ($12,110.31 vs $75,044.44 list). Checked with: python -m traceguard.routing_audit.cache_audit
If you run it, send me your numbers. Especially if pings win somewhere — I'd like to see the traffic shape that makes them rational.
Li Zhuojun