{"slug": "claude-s-20-block-cache-lookback-silently-kills-agent-loops", "title": "Claude's 20-block cache lookback silently kills agent loops —", "summary": "Anthropic's Claude prompt caching silently fails in agent loops when the conversation exceeds 20 content blocks between cache breakpoints, causing cache reads to drop to zero and triggering full-prefix cache writes at 1.25× the read price — a 12× cost swing per turn on Opus 5 ($5/MTok input), according to a technical analysis. The issue arises because each agentic turn with 8 parallel tool calls emits 18 blocks, so two turns push the trailing breakpoint past the 20-block lookback limit, and the article proposes rolling breakpoints with a stride of 15 blocks to maintain cache hits.", "body_md": "# Claude's 20-block cache lookback silently kills agent loops —\n\n`cache_read_input_tokens`\n\nat 40K and climbing. Twelve tool calls later, reads drop to zero and `cache_creation_input_tokens`\n\nspikes to the full conversation length — every single turn. Prompt didn't change. No timestamp, no reordered tools, no model switch. The prefix is byte-identical.You just hit the 20-block lookback ceiling. It's the single most expensive gotcha in [Claude](/en/tags/claude/) prompt caching that nobody puts in their retro.\n\n## Why the lookup fails mid-loop\n\nCache lookup is bounded. A `cache_control`\n\nbreakpoint walks **backward at most 20 content blocks** hunting for an existing entry. One agentic turn with 8 parallel tool calls emits:\n\n- Assistant: 1 thinking + 1 text + 8\n`tool_use`\n\n= 10 blocks - User: 8\n`tool_result`\n\n= 8 blocks **Total: 18 blocks in one round trip**\n\nTwo turns and your single trailing breakpoint sits 36 blocks past the last cached point. Silent miss — no error, no warning field, just\n\n`cache_read_input_tokens: 0`\n\nand a full-prefix cache-write charge at 1.25× the read price. On Opus 5 ($5/MTok input) that's ~$0.50/MTok read vs ~$6.25/MTok write. A 12× swing per turn triggered by a config detail you never set.Chat apps never see this — one user turn = one block, one assistant turn = one block. You'd need ten round trips to move 20 blocks. Agent loops are a different beast entirely.\n\n## Tiered invalidation — what actually busts the cache\n\n| Trigger | Cache impact |\n\n|---------|--------------|\n\n| `tool_choice`\n\nchange, images added/removed, thinking toggle | Preserves tools+system cache |\n\n| Tool definitions change, model switch | Full rebuild |\n\n| System prompt replaced mid-run | Nukes everything downstream |\n\n**Workaround for system-prompt edits** (Opus 5, Opus 4.8, Fable 5 only): append `{\"role\": \"system\", ...}`\n\nto `messages[]`\n\ninstead of replacing the top-level `system`\n\nfield. Sonnet 5 doesn't support this.\n\n## Rolling breakpoints — the pattern that holds\n\nStop putting one marker on the last block. Rotate a small set through the message list at a stride shorter than the lookback window. Every breakpoint is both a write point *and* a read point, so a trailing chain means each new request always finds a prior entry within 20 blocks.\n\n**Budget: 4 breakpoints per request total** (tools + system + messages). Spend one on the last system block — it caches tools and system together since render order is `tools → system → messages`\n\n. Rotate the remaining three.\n\n```\nCACHEABLE = {\"text\", \"image\", \"tool_use\", \"tool_result\", \"document\"}\nSTRIDE = 15  # blocks apart, safely under the 20-block lookback\n\ndef add_rolling_breakpoints(messages: list[dict], stride: int = STRIDE) -> None:\n    \"\"\"Mutates plain-dict messages in place. Round-trip SDK objects with .model_dump() first — you cannot set cache_control on a response object.\"\"\"\n    flat = []\n    for msg in messages:\n        for block in msg.get(\"content\", []):\n            if block.get(\"type\") in CACHEABLE:\n                flat.append((msg, block))\n\n    # clear existing\n    for _, block in flat:\n        block.pop(\"cache_control\", None)\n\n    # place markers every STRIDE blocks from the end\n    for i, (_, block) in enumerate(reversed(flat)):\n        if i % stride == 0:\n            block[\"cache_control\"] = {\"type\": \"ephemeral\"}\n\n# usage each turn:\nadd_rolling_breakpoints(messages)\nresponse = client.messages.create(model=\"claude-opus-4-20250514\", messages=messages, ...)\n```\n\nThe system+tools breakpoint stays fixed. The three message breakpoints leapfrog: newest turn gets one, the two prior turns keep theirs. Next turn, the oldest drops off, the new one lands — always three live markers spaced ≤15 blocks apart. Lookup never misses.\n\n## Dashboard trap\n\n`input_tokens`\n\nin the usage block is **uncached remainder only**. Total prompt size = `input_tokens + cache_creation_input_tokens + cache_read_input_tokens`\n\n. Grafana panels graphing `input_tokens`\n\nalone show a flat line while you burn cache writes at 12× the read cost. Add the other two series or you're flying blind.\n\n**Bottom line:** the 20-block limit isn't documented in the quickstart. If you run multi-tool agent loops on Claude, rolling breakpoints every ~15 blocks is the difference between a $0.50/MTok read path and a $6.25/MTok rewrite every turn. Ship the snippet, watch `cache_read`\n\nstay non-zero, and stop lighting money on fire.\n\n[Next Qwen 3.5 4B actually runs a full agent loop on iPhone 15 Pro →](/en/threads/7180/)", "url": "https://wpnews.pro/news/claude-s-20-block-cache-lookback-silently-kills-agent-loops", "canonical_source": "https://promptcube3.com/en/threads/7182/", "published_at": "2026-08-21 16:12:20+00:00", "updated_at": "2026-08-21 16:42:44.329556+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "developer-tools"], "entities": ["Anthropic", "Claude", "Opus 5", "Opus 4.8", "Fable 5", "Sonnet 5"], "alternates": {"html": "https://wpnews.pro/news/claude-s-20-block-cache-lookback-silently-kills-agent-loops", "markdown": "https://wpnews.pro/news/claude-s-20-block-cache-lookback-silently-kills-agent-loops.md", "text": "https://wpnews.pro/news/claude-s-20-block-cache-lookback-silently-kills-agent-loops.txt", "jsonld": "https://wpnews.pro/news/claude-s-20-block-cache-lookback-silently-kills-agent-loops.jsonld"}}