{"slug": "trimming-context-can-make-your-agent-more-expensive", "title": "Trimming Context Can Make Your Agent More Expensive", "summary": "Two studies dated 26 Oct found that deleting old tool output from coding-agent runs can raise costs despite shortening prompts: keeping only the last five tool outputs made prompts 33.3% shorter but increased uncached prefill by 608.5%, and deleting old content saved money in none of 42 configurations once prompt caching was priced in. The mechanism is that agents resend the full conversation on every call, and Anthropic and OpenAI bill an unchanged cached prefix at 0.1x the base input price while charging 1.25x to write the 5-minute or default 30-minute cache, so any edit to earlier turns invalidates everything after the first changed token. The Physical Cost of Context (Yu and others, UC San Diego and UMass Amherst) ran mini-SWE-agent on 100 SWE-bench Verified tasks on a local GPU box with prefix caching on, comparing an Eager policy that keeps every tool output against Mask-5, which retains only the last five.", "body_md": "# Trimming Context Can Make Your Agent More Expensive\n\nTwo new studies deleted old tool output from coding-agent runs. The prompts got shorter and the bills got bigger. Here's why, when trimming still pays, and how to check it in your own usage numbers. Cut one block out of the middle and everything after it has to be paid for again. TL;DR Your agent\n\nresends the whole conversation on every call. Prompt caching is what keeps that cheap: an unchanged prefix is billed at about a tenth of the normal input price. Deleting or rewriting old tool output changes that prefix. Everything after the first changed token is billed as fresh input again.\n\nIn one study, keeping only the last five tool outputs made prompts 33.3% shorter, but uncached prefill went up 608.5%. In another, deleting old content saved money in none of 42 configurations once caching was priced in. Trimming still pays when the cache is cold anyway, or when you do it rarely and in big batches. Trimming a little on every call is the expensive pattern.\n\nBefore you install a context trimmer, look at cache reads and cache writes per call, not total tokens. There's a thread on r/opencodeCLI from 1 Nov that sums up the problem. The poster has a conversation of about 100,000 tokens. They answer the agent within a minute, and the next call bills almost all 100,000 as input.\n\nIn one screenshot it's 148,114 input tokens and 0 cache read. Their words: \"it's making conversations more expensive.\" The replies got to the point quickly. Every harness sends the whole conversation on every call, and the provider's cache is what makes that affordable. When the cache misses, you pay full price for all of it.\n\nOne reply listed the usual suspects: a plugin that changes the start of the prompt, a timestamp at the front instead of the back, edited tool definitions or AGENTS file. There's one more suspect, and it's one people install on purpose: context trimming. Tools that drop old tool output to \"save tokens\" also change the prompt. Two papers from 26 Oct measured what that costs.\n\nI covered the basics in Compact, Clear or Hand Off?, so here's the short version. Anthropic's prompt-caching docs describe the cached prefix as tools, then system prompt, then messages, in that order. A cache hit needs that prefix to be 100% identical. Change something at one level and that level and everything after it is invalidated.\n\nOpenAI's docs say the same in fewer words: \"Append new messages rather than rewriting earlier turns. Summarization, compaction, or context truncation can change the prefix and reset cache reuse.\" The price gap is what makes this matter. On Anthropic's API, a cache read costs 0.1x the base input price. Writing to the 5-minute cache costs 1.25x, and the 1-hour cache costs 2x.\n\nOpenAI's current models charge 1.25x to write the default 30-minute cache and 0.1x to read it. Some newer models on both sides read even cheaper. OpenAI's own example: ten requests on a cached prefix cost 2.15x the price of one uncached request, against 10x without caching. So a long agent session is cheap to continue only as long as it stays append-only.\n\nPicture call 40 of a session. The first 39 calls are already cached. If a trimmer now replaces the output of call 12 with \"[output removed]\", the prompt gets shorter by that much. But everything from call 12 onwards is no longer a cache hit.\n\nIt's billed at full price and written to the cache again at 1.25x. On the next call, the trimmer removes call 13's output and it all happens again. The Physical Cost of Context (Yu and others, UC San Diego and UMass Amherst, 26 Oct) measures what a context policy costs the hardware, not just the token count. They ran mini-SWE-agent on 100 SWE-bench Verified tasks with an open-weights model on a local GPU box.\n\nThe serving stack had prefix caching on. The comparison that matters here is between two policies: Eager keeps every tool output in the conversation. Mask-5 keeps only the last five tool outputs and replaces older ones with a short placeholder. It's the same idea as SWE-agent's \"last N observations\" setting, and it's what many trimmers do.\n\nMask-5 did what it promises on tokens: prompts per task went from 225.6k to 150.5k, 33.3% less. But the authors also counted how much of each prompt the server actually had to compute fresh, because it wasn't in the cache. That number went from 10.0k to 70.8k tokens per task, up 608.5%. Masking removed 75.1k tokens, but it threw away 135.9k tokens that would have been cache hits.\n\nWith Eager, cached prefixes covered about 96% of the prompt. With Mask-5, 53%. The rest followed from that. Wall time went up 36.9% and their GPU energy measure went up 125.6%.\n\nMask-5 also solved 26 tasks against Eager's 29, so it failed the authors' quality bar too. Their summary line is blunt: \"repeated observation masking turned a 33.3% prompt saving into a 608.5% rise in uncached prefill.\" Shorter prompt, more fresh work. Numbers per task, from Yu and others (arXiv 2610.34763), Table 2. A third policy in the paper shows what makes the difference.\n\nCompact rewrites the history once, after the agent has found the relevant file, and then leaves it alone. It cut prompts 6.5% and raised uncached prefill 22%. Not much of a saving, but a long way from 608%. Mask-5 rewrites on every call once its window is full.\n\nCompact rewrites once. Per task Eager: keep all Compact: rewrite once Mask-5: last 5 outputs Prompt tokens 225.6k 210.8k (−6.5%) 150.5k (−33.3%) Uncached prefill 10.0k 12.2k (+22.0%) 70.8k (+608.5%) Wall time 122.7 s 116.6 s (−5.0%) 167.9 s (+36.9%) GPU energy proxy 1.956 kJ 1.917 kJ (−2.0%) 4.412 kJ (+125.6%) The Hidden Costs of Forgetting (Vallecillos-Ruiz, Bouzenia, Pradel and Moonen, Simula and CISPA, also 26 Oct) looks at the bill. They ran two agents, mini-SWE-agent and Trae Agent, on SWE-bench Verified with DeepSeek-V4-Flash. They tried 42 ways of deleting history.\n\nThese varied in what got deleted (reasoning, tool output or both), how much (25% to 100%), whether whole turns or individual tokens went, and whether deletion waited two turns or happened straight away. In total there were 9,400 runs. They priced every run twice. Flat pricing bills all input tokens the same.\n\nCaching-aware pricing bills cached input at 0.1x and output at 5x, which is within the range of real price lists. It's an estimate, not an invoice. Under flat pricing, deletion made 26 of the 42 setups cheaper. Under caching-aware pricing, none of them got cheaper, and 26 were significantly more expensive than leaving the history alone.\n\nDeleting all tool output turn by turn in mini-SWE-agent came to 0.71 of the baseline cost under flat pricing and 1.20 with caching. Same change, saving or loss depending on how you count. The worst setup cost almost five times the baseline. Their explanation matches the theory.\n\nTotal tokens did go down, by up to a third in 31 of the 42 setups. But tokens billed at the full input price went up in all 42, from about 1.25x to nearly 10x the baseline. In the authors' words, every rewrite \"re-bills the prompt from the first changed token onwards\". And this is the optimistic version.\n\nThe cost model ignores the cache-write premium and assumes every unchanged prefix is a hit. The authors call it a lower bound. There was a quality cost too. Agents often went back for what had been deleted.\n\nDeleting tool output made them re-run tool calls up to 7.5 times as often as baseline with the two-turn delay, and 10.6 times without it. With the delay, 37 of 42 setups solved about as many tasks as baseline. Without it, 19 of 42 solved significantly fewer, nearly all of them setups that deleted tool output; reasoning-only deletion held up (13 of 14 no significant change). How the same 42 setups look under two price lists.\n\nFrom Vallecillos-Ruiz and others (arXiv 2610.35496). Both papers measured a cache that was working well. Study 1 served one request at a time on a single box, so nothing pushed its cache out. Study 2's cost model assumes every matching prefix is a hit.\n\nOn a busy hosted API, or after you've been away from the keyboard, that isn't always true. Study 1 gives a simple way to think about this. Say your cache only delivers a fraction ρ of the hits they saw. Then masking changes your fresh prefill by −75.1k + 135.9k × ρ tokens per task.\n\nAbove ρ ≈ 0.55, masking costs you more fresh prefill. Below it, masking saves you some. With no cache at all, masking is a straight saving. That 0.55 comes from one agent, one model and one workload, so don't treat it as a constant.\n\nBut the shape holds: the warmer your cache, the more a mid-session trim costs you. Trimming hurts when: you're in a busy session and replying within the cache lifetime (5 minutes by default on Anthropic, 30 minutes by default on OpenAI's current models); the trimmer removes a bit of old output on every call, like a sliding window; your usage already shows mostly cache reads. Trimming pays when: the cache is cold anyway: you've been idle past its lifetime, or you've switched model or effort, which starts a new cache; you start fresh with /clear and a handoff file, so there's nothing cached to lose (I covered this in Your Handoff Doc Isn't Working); you remove a big block once, and the session runs for many more calls on the new, shorter prefix; your logs show the cache misses most of the time anyway. Neither paper says \"never trim\".\n\nStudy 2 ends by asking for \"less frequent history rewriting\", with one caveat: \"Batched rewriting still invalidates the prefix, so its net savings require evaluation\". So here's what I'd do, from what the docs offer today. Same session, three trimming schedules. Each cache break bills everything after it at full price again.\n\n1. Keep the session append-only between trims. Don't let plugins rewrite earlier turns or put a timestamp at the top of the system prompt. Don't change tool definitions or MCP servers mid-session.\n\nOn Anthropic, changing tool choice, thinking settings, effort or images also breaks the cache. If you want to switch model or effort, do it when you start a new session. 2. Trim in batches, at compaction points.\n\nIf you trim, take out a lot at once, and then leave the history alone for many calls. One rewrite every 20 calls is one cache break. A sliding window of five is a break on every call. 3.\n\nOn the API, make each clear worth it. Anthropic's context editing (in beta) has a tool-result clearing strategy (clear_tool_uses_20250919) that keeps the last three tool uses by default once input passes 100,000 tokens. Its docs warn that clearing \"invalidates cached prompt prefixes\", and tell you to use clear_at_least so each clear removes enough tokens to be worth the cache break. That parameter has no default, so set it.\n\nSomething like: \"context_management\": { \"edits\": [{ \"type\": \"clear_tool_uses_20250919\", \"trigger\": {\"type\": \"input_tokens\", \"value\": 100000}, \"keep\": {\"type\": \"tool_uses\", \"value\": 3}, \"clear_at_least\": {\"type\": \"input_tokens\", \"value\": 40000} }] } The 40,000 is my starting point, not a measured optimum. Your break-even depends on how many calls the session still has to go. 4. Summarise once, then start fresh.\n\nCompaction is a batch rewrite too. Do it at a natural break, on a warm cache if you can (I wrote about why in Compact, Clear or Hand Off?). Or skip compaction, write a handoff, /clear and start clean. 5.\n\nFix the cause before trimming the symptom. Study 1's Eager policy saved work by handing the agent the right files up front. It made fewer calls instead of shorter ones. A huge tool output is often a command that should have been | tail -50 or a file read that should have been a line range.\n\nTotal tokens can't tell you whether trimming helps. You need to see how the input splits into cache reads, cache writes and uncached input, per call. Anthropic API: each response's usage has cache_read_input_tokens, cache_creation_input_tokens and input_tokens. Here input_tokens is only what comes after the last cache breakpoint, so total input is all three added together.\n\nIn a healthy long session, cache reads are most of the input on each call. A trimmer that breaks the cache every call shows big cache writes on every call. OpenAI API: usage.input_tokens_details.cached_tokens tells you how much was read from cache, and on newer models cache_write_tokens tells you what was written. Note that here input_tokens already includes the cached part.\n\nOpenAI's docs add the important caveat: \"fewer input tokens can still save money even when the cache-hit rate falls.\" So compare total cost before and after, not the hit rate alone. Claude Code: /cost lists cache reads and cache writes separately (\"940.0k cache read, 50.0k cache write\" in the docs' example), and the OpenTelemetry api_request event splits cache writes into 5-minute and 1-hour. On a subscription you don't see dollars, but the split still shows whether the cache is doing its job. A quick test I'd run before installing any trimmer: do the same task twice, once with the trimmer and once without, and add up read × 0.1 + write × 1.25 + uncached × 1 (adjust for your model's prices) across all calls.\n\nIf the trimmed run isn't clearly lower, the trimmer is moving cost around, not removing it. It's also worth knowing what share of your bill is cache reads. One r/ClaudeCode poster went through four weeks of their transcripts on 2 Nov: cache reads were 53% of spend. That's the share a careless trimmer turns into full-price input.\n\nNeither is a cloud bill. Study 1 measures fresh prefill, time and energy on one local GPU box serving one request at a time. Its authors say batching, cache pressure and cloud GPUs weren't tested. Study 2's dollar figures come from a price model, not invoices.\n\nThe 608% is a token count, not a price. It's the rise in tokens the server had to compute fresh. It doesn't translate one-to-one into money. One agent family, open models.\n\nBoth used SWE-bench-style tasks, mini-SWE-agent (plus Trae Agent in Study 2), and open-weights models. Claude Code and Codex manage context in their own ways, and neither paper tested them. Both are preprints, posted 26 Oct and not yet peer reviewed. Token savings still matter at the limit.\n\nIf you're near the context window, trimming may be what keeps a session running. The papers are about cost, not room. r/opencodeCLI: \"Lots of times opencode sends almost the whole conversation as input tokens\", cache misses within a minute of the last reply. r/ClaudeCode: \"Spotify's shunt plugin cuts big file reads by 90%.\n\nOn my transcripts it would save under 1%\", where cache reads were 53% of spend. r/ClaudeAI: \"Automating /clear session\", about when to clear: 150k or 500k. Yu, Ma, Yu, Yuan, Lin, Zhang, Zhao, \"The Physical Cost of Context: Quality-Constrained Resource Accounting for Coding Agents\", arXiv 2610.34763, 26 Oct 2026 (read 2 Nov 2026; preprint) Vallecillos-Ruiz, Bouzenia, Pradel, Moonen, \"The Hidden Costs of Forgetting: A Study of Trajectory Deletion in Software Engineering Agents\", arXiv 2610.35496, 26 Oct 2026 (read 2 Nov 2026; preprint) Anthropic docs, Prompt caching and Context editing (read 2 Nov 2026) OpenAI docs, Prompt caching (read 2 Nov 2026) Claude Code docs, Costs and CHANGELOG (read 2 Nov 2026) Previous articles: Compact, Clear or Hand Off?, Your Handoff Doc Isn't Working, Where Did My Claude Usage Go? If this was useful, subscribe to Agents in Practice for one practical note on AI coding agents a week.\n\nHelping your team adopt Claude Code or Codex safely and without the surprise bill? I consult on exactly that: andre-burgstahler-9n5.pages.dev. This article was written with AI assistance and reviewed by the author.\n\n## Key Takeaways\n\n- •Prompt caching is what keeps that cheap: an unchanged prefix is billed at about a tenth of the normal input price.\n- •This story was reported by **Dev.to** , covering developments in the**dev** space.\n- •AI advancements continue to reshape industries — read the full article on Dev.to for complete coverage.\n\n📖 Continue reading the full article:\n\n[Read Full Article on Dev.to →](https://dev.to/r3dtuxedo/trimming-context-can-make-your-agent-more-expensive-5gib)", "url": "https://wpnews.pro/news/trimming-context-can-make-your-agent-more-expensive", "canonical_source": "https://ainexusdaily.vercel.app/article/2026-11-04-trimming-context-can-make-your-agent-more-expensive", "published_at": "2026-11-04 13:41:38+00:00", "updated_at": "2026-11-04 14:50:28.576300+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "large-language-models", "ai-tools"], "entities": ["Anthropic", "OpenAI", "UC San Diego", "UMass Amherst", "mini-SWE-agent", "SWE-bench Verified", "The Physical Cost of Context", "r/opencodeCLI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/trimming-context-can-make-your-agent-more-expensive", "markdown": "https://wpnews.pro/news/trimming-context-can-make-your-agent-more-expensive.md", "text": "https://wpnews.pro/news/trimming-context-can-make-your-agent-more-expensive.txt", "jsonld": "https://wpnews.pro/news/trimming-context-can-make-your-agent-more-expensive.jsonld"}}