{"slug": "what-i-learned-cutting-claude-code-s-token-bill-by-77", "title": "What I Learned Cutting Claude Code's Token Bill by 77%", "summary": "A developer built the AI Agent Profiler, a transparent proxy that records every byte of traffic between AI coding agents and language models. Profiling Claude Code sessions revealed that agents re-send the entire conversation history on every turn, causing massive token waste. By pruning stale context and unused tool schemas, the developer cut token costs by 77% without changing any results.", "body_md": "**What building a profiler for AI coding agents taught me about the hidden river of data flowing to Claude on every turn, and how to cut most of it without changing a single result.**\n\n[Dashboard live demo](https://rguiu.github.io/ai-agent-profiler/) · [Source](https://github.com/rguiu/ai-agent-profiler)\n\n*(The live dashboard profiles a real DeepSeek session — the plumbing is identical for Claude Code.)*\n\n⚠️\n\nEarly results.Single-sample runs on one fixture (Claude Code + Opus 4.6 on Bedrock). The direction is strong and reproducible; treat individual percentages as ballpark. Caveats are at the end, that honesty is part of the point.\n\nAn AI coding agent feels like magic: ask it to fix a bug, it reads files, writes code, runs tests, reports back. Underneath, it's just making HTTP requests to a model, over and over — traffic you never see. I built the **AI Agent Profiler** (`aap`\n\n): a transparent proxy that records every request byte-for-byte and passes it through untouched. The first thing that jumps out is the **volume**.\n\nEvery call to the model is **stateless** — it remembers nothing between turns. So the agent re-sends the **entire history, from the top, on every turn**. Turn 30 doesn't send turn 30; it sends turns 1–30. Again. It's not a chat, it's a snowball:\n\n```\n[ System:  \"You are Claude Code. Here are your 15 tools…\" ]  ~5,000 tok — re-sent every turn\n[ Tools:   full JSON schemas ]                               ~7,000 tok — re-sent every turn\n[ User:    \"fix the failing tests\" ]\n[ Tool:    <full 2 KB of scheduler.js> ]                     read once — re-sent forever\n[ Tool:    <200 lines of test output> ]\n   … and on, and on …\n```\n\nMost of it is **dead weight**: a file you read and edited ten turns ago is still mailed on every later turn; 15 tool schemas ride along when the agent uses three. That's why long sessions get expensive, not the model \"thinking hard,\" but re-mailing stale paperwork thousands of times. One 70-request session I measured pushed **~3.9M input tokens** while the model wrote back a few thousand words.\n\nAnthropic softens this with **prompt caching** — and it's an **explicit** model you control. You mark where a cacheable prefix ends with a `cache_control`\n\nbreakpoint:\n\n```\n{ \"type\": \"text\", \"text\": \"You are Claude Code…\", \"cache_control\": { \"type\": \"ephemeral\" } }\n```\n\nWhat matters:\n\nPricing (Opus 4.6 / Bedrock; *verified against anthropic.com/pricing on 12 Jul 2026 — rates change*): cache read **$0.50/MTok** vs input **$5.00** vs output **$25.00**.\n\nClaude Code already places its 4 markers well and hits ~99–100% cache. So the real question isn't \"turn caching on\" — it's **how much of that cached snowball can I stop carrying without breaking the prefix?**\n\nBecause the profiler sits in the middle of every request, it can do more than watch. With the optimize layer on, it tidies the outgoing pile just before each request leaves, never touching the reply, never changing what the agent does next. Biggest levers first:\n\n`pruneStale`\n\n(~57% of the savings).`[read scheduler.js earlier — 47 lines]`\n\n. The model already acted on it.`pruneUnusedTools`\n\n(~23%).`insertBreakpoints`\n\n.(A fourth lever, `collapseSystem`\n\n, stubs the repeated system prompt — see the caveats; I'm deliberately not leaning on it.)\n\nTo get real numbers I used a fixed **benchmark fixture**: a small JavaScript project with 9 planted bugs and 3 stubbed-out methods, ~54 visible tests, plus a set of hidden edge-case tests the agent never sees (used to grade quality afterwards). The task handed to Claude Code was blunt — fix everything and make the tests pass. I ran that identical task through the profiler three ways:\n\n```\nRun                    Reqs   Input tok   Cache hit   Cost    vs baseline\n──────────────────────────────────────────────────────────────────────────\nNo optimization          70   3,917,198     100%     $2.04       —\nFull optimization        94     776,366     100%     $0.48     −77%\n\"Cache-protective\"*     118   4,978,066     100%     $2.67     +31% WORSE\n```\n\n* don't-prune-anything, to \"protect\" the cache — the **worst** result (more below).\n\nQuality was **identical** across runs: 54/54 fixture tests, same 54/57 edge tests. The optimizer changed the *cost*, not the *outcome*.\n\nThe naive fear — \"editing old messages shatters the prefix and costs more\", is backwards here, and *not* because misses were outweighed by volume. The winning run held **100% hit with only 92 uncached tokens**: pruning kept the hit rate **and** shrank the volume, because `insertBreakpoints`\n\nre-anchors the markers after the edit. Cached tokens carried per request fell from ~4.3M to ~776K.\n\nHonest note, simulation vs reality.My offline simulator (a strict byte-prefix model) predicted pruning would tank the hit rate to ~67%. The live Bedrock runs held ~100%. Anthropic's real cache tolerates in-region edits far better than a naive model assumes; where they disagreed, I trusted the live runs.\n\nSo \"protect the cache by not pruning\" is exactly wrong on Anthropic-format traffic: carrying 4.3M cached tokens every turn costs more than carrying 776K — even though both are \"cached.\"\n\nI tried the same layer on DeepSeek (OpenAI chat format) and got the opposite result — the identical `pruneStale`\n\nedit **shattered** its automatic prefix cache and blew cost up. Different cache, different rules: mutating old context is safe-ish on Anthropic's explicit breakpoints and toxic on a pure prefix cache. I built a separate **cache-safe** DeepSeek profile instead. Full cross-provider story and data are in [the repo](https://github.com/rguiu/ai-agent-profiler). The lesson worth keeping: **there's no universal \"make it cheaper\" button — the right one is provider-aware.**\n\n`collapseSystem`\n\nis unverified.Open source (MIT). In read-only mode it just shows you the firehose — how big the pile is, which files get re-read, where the money goes:\n\n```\naap serve             # the meter on the pipe\naap run claude        # launch the agent through it\naap serve --optimize  # …and trim the pile (measure on your own provider first!)\n```\n\nThe interesting question turned out not to be *\"how much can I save?\"* — it's *\"what is my agent actually sending, and why is it sending it a thousand times?\"*", "url": "https://wpnews.pro/news/what-i-learned-cutting-claude-code-s-token-bill-by-77", "canonical_source": "https://dev.to/rguiu/what-i-learned-cutting-claude-codes-token-bill-by-77-3ef", "published_at": "2026-07-12 00:43:04+00:00", "updated_at": "2026-07-12 01:14:10.100417+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "large-language-models", "ai-infrastructure", "ai-research"], "entities": ["Claude Code", "Anthropic", "Bedrock", "Opus 4.6", "AI Agent Profiler", "DeepSeek"], "alternates": {"html": "https://wpnews.pro/news/what-i-learned-cutting-claude-code-s-token-bill-by-77", "markdown": "https://wpnews.pro/news/what-i-learned-cutting-claude-code-s-token-bill-by-77.md", "text": "https://wpnews.pro/news/what-i-learned-cutting-claude-code-s-token-bill-by-77.txt", "jsonld": "https://wpnews.pro/news/what-i-learned-cutting-claude-code-s-token-bill-by-77.jsonld"}}