Do Dynamic Tools Break Prompt Caching?

A production audit of 10,186 assistant turns across OpenAI, Google Gemini, and OpenRouter found that dynamic tool activation causes prompt cache misses on only 2.4% to 3.4% of turns, while avoiding 71.2 million unused schema tokens and reducing average turn costs by 32.5%. The study, conducted over 93 multi-turn coding sessions in the Pi coding agent, attributes over 82.5% of cache misses to routine multi-file reads, context compaction, and cloud provider 5-minute idle timeouts, not tool switching.

Do Dynamic Tools Break Prompt Caching? Short answer: No. Dynamic tool activation does not destroy provider prompt caching. In an audit of 10,186 assistant turns across OpenAI, Google Gemini, and OpenRouter, tool swapping caused a cache miss on only 2.4% to 3.4% of turns , while avoiding 71.2 Million unused schema tokens and reducing average turn costs by 32.5% . Over 82.5% of prompt cache misses stem from routine multi-file reads, context compaction, and cloud provider 5-minute idle timeouts, not tool switching. This study is the empirical follow-up to my architecture deep-dive: How I Cut 80%+ of Context Overhead in My Coding Agent . When I published my architecture for dynamic tool activation in AI coding agents, the most common counterargument on Hacker News was immediate: "Adding tools dynamically seems fine, but removing them is a terrible idea. Swapping schemas changes the prompt prefix and constantly nukes your provider prompt cache." On paper, this objection sounds logical. Major LLM providers Anthropic, OpenAI, Google cache prompt prefixes. If you modify the system prompt or tool schemas, the prefix hash changes, and the model must write a fresh cache entry at standard or elevated write pricing. To see whether dynamic tool activation actually harms prompt caching in production, I audited 93 multi-turn coding sessions and 10,186 assistant turns in Pi across OpenAI, Google Gemini, and OpenRouter. Here is what the empirical telemetry reveals. The dataset I analyzed all interactive coding sessions over an 11-day period where dynamic tool activation was active in daily engineering workflows. Total Sessions: 93 sessions Total Assistant Turns: 10,186 turns Total Tool Calls: 11,461 executions Models Tested: OpenAI GPT-5.6 Sol, GPT-5.6 Luna, Codex , Google Gemini 3.7 Flash, and OpenRouter community endpoints. Total Tokens Processed: 1.22 Billion tokens 1. How often do agents actually call standby tools? The core assumption behind the "cache nuke" concern is that an agent constantly swaps tools in and out every two turns. In practice, software development follows a strict power-law distribution. Across 11,461 total tool executions: Core 4 tools 10,805 calls bash , read , edit , write : 94.28% . Standby tools activated on demand: 656 calls 5.72% . Core tools bash, read, edit, write : ██████████████████████████████ 94.28% Standby tools browser, loops, image : █ 5.72% Over 94% of an agent's work consists of reading files, editing lines, and running shell commands. Because the 4 default tools never leave the prompt, the tool schema prefix remains 100% identical and cached for over 85% to 90% of the entire session. 2. What actually causes cache misses? Across all 10,186 assistant turns, there were 2,030 cache miss or zero-cache events a 19.93% total miss rate, giving an overall 80.08% cache hit rate . I categorized every single cache miss by its technical trigger: Figure 1: Root causes of cache misses across 10,186 assistant turns. Over 82.5% of misses stem from routine file operations and cloud TTLs, not tool switching. | Root Cause | Cache Misses | Share of Misses | Share of ALL 10,186 Turns | |---|---|---|---| Large File Reads & Context Compaction | 1,422 | 70.05% | 13.96% | Tool Activation & TTL Expiration | 354 | 17.44% | 3.47% | Provider Idle Timeouts 5 min | 150 | 7.39% | 1.47% | Turn 0 Session Warmup | 89 | 4.38% | 0.87% | Model Switching | 15 | 0.74% | 0.15% | Total | 2,030 | 100% | 19.93% | Key takeaways from the miss data: Tool activation accounts for only 3.47% of turns. Across all providers and models, swapping tools or letting a tool expire after its 2-turn TTL caused a cache transition in only 354 turns. The other 96.53% of turns ran with zero tool-related cache disruption. On OpenAI models, the tool miss rate was 2.40%. In 4,164 OpenAI turns, tool activation caused exactly 100 cache transitions. Over 82.5% of cache misses had nothing to do with tools. The primary driver of cache churn is dumping 2,000-line files into the prompt, which shifts the context boundary and forces session compaction. 3. Why cache hit rates vary by provider When comparing hit rates across providers, the differences come down to how each vendor structures its caching engine: Figure 2: Empirical cache hit rates across models and provider endpoints. | Provider / Model | Total Turns | Cache Hit Rate | Minimum Token Threshold | Documented Cache TTL | |---|---|---|---|---| OpenAI GPT-5.6 / Codex | 4,164 | 86.5% | 1,024 tokens | 30 minutes | Google Gemini 3.7 Flash | 3,683 | 78.8% | 32,768 tokens 32k | 1 hour | OpenRouter / Free Endpoints | 2,237 | 65.2% | Variable | None / Node cycling | The Google Gemini 32k threshold Google Gemini's 78.8% hit rate initially looked lower than OpenAI's 86.5%. Looking into Google Cloud's documentation explains why: - OpenAI begins caching prompts automatically as soon as the input exceeds 1,024 tokens . - Google Gemini's context caching engine requires prompts to exceed 32,768 tokens before caching activates. In short sessions or early turns where context was under 32k tokens, Gemini returned cacheRead: 0 by design. Out of 772 Gemini cache misses, 471 misses 61.0% occurred solely because the prompt had not yet reached Google's 32k threshold. Once sessions grew past 32k tokens, Gemini's cache hit rate climbed past 90%. 4. The financial math: cache writes vs. schema dragging Prompt cache reads are heavily discounted, but they are not free. OpenAI charges 10% to 50% of base input price for cached reads $0.30 to $1.25 / MTok on GPT-5.6 / GPT-4o . When you keep 79 static tools loaded in an environment like Codex, you send roughly 12,000 extra schema tokens on every turn . Here is the exact financial balance sheet from my 4,164 OpenAI turns: | Strategy | Extra Schema Tokens Dragged | Cache Rebuild Cost | Cache-Read Fees | Net Cost Impact | |---|---|---|---|---| Static Tools 79 tools loaded 100% of the time | 49.96 Million tokens | $0.00 | +$62.45 | +$62.45 penalty | Dynamic Tools 4 core + on-demand standby | 0 tokens | ~$0.80 100 writes | $0.00 | -$61.65 net savings | Dragging 79 static tools forces you to pay a cache-read fee on 12,000 unused tokens on every single turn for the entire day. By pruning standby tools after 2 turns of idle time, I spent ~$0.80 on 100 cache rebuilds to save $62.45 in cache-read fees. That is a ~77x return on investment . Across all models and sessions, my average cost per turn dropped from $0.0609 to $0.0411 per turn a 32.5% pure invoice reduction . How to maximize cache stability in your agent Based on these 10,000 turns, here are the three rules for maintaining prompt cache stability: 1. Promote high-frequency search tools to core In my initial setup, web search and web fetch were on standby. Because web search accounted for 36% of all tool activations, promoting both to the default tool set eliminated 36 cache transitions for just ~350 extra tokens. Keep heavy engines browser use , multi-agent loops, image generators on standby, and leave lightweight, high-frequency lookups in core. 2. Enforce deterministic prompt sorting Standby tool lists should be sorted alphabetically standby.sort before prompt injection. If tool discovery returns items in varying order across runs, the byte string changes and breaks prefix matching. 3. Use bounded reads instead of full file dumps Large file reads cause 70% of all cache misses. Adding a single guideline to your project instructions stops agents from dumping 2,000-line files: Search with rg -n to locate target lines first, and use read with offset and limit 100–200 lines instead of loading entire large files into context. Conclusion Dynamic tool activation does not destroy prompt caching. Because 94% of coding tasks only use basic file and shell tools, your prompt prefix stays stable across the vast majority of turns. The few cache transitions that do occur cost pennies in cache writes, while saving tens of millions of billed schema tokens and keeping the model's context window clean.