What building a profiler for AI coding agents taught me about the hidden river of data flowing to Claude on every turn, and how to cut most of it without changing a single result.
(The live dashboard profiles a real DeepSeek session — the plumbing is identical for Claude Code.)
⚠️
Early results.Single-sample runs on one fixture (Claude Code + Opus 4.6 on Bedrock). The direction is strong and reproducible; treat individual percentages as ballpark. Caveats are at the end, that honesty is part of the point.
An AI coding agent feels like magic: ask it to fix a bug, it reads files, writes code, runs tests, reports back. Underneath, it's just making HTTP requests to a model, over and over — traffic you never see. I built the AI Agent Profiler (aap
): a transparent proxy that records every request byte-for-byte and passes it through untouched. The first thing that jumps out is the volume.
Every call to the model is stateless — it remembers nothing between turns. So the agent re-sends the entire history, from the top, on every turn. Turn 30 doesn't send turn 30; it sends turns 1–30. Again. It's not a chat, it's a snowball:
[ System: "You are Claude Code. Here are your 15 tools…" ] ~5,000 tok — re-sent every turn
[ Tools: full JSON schemas ] ~7,000 tok — re-sent every turn
[ User: "fix the failing tests" ]
[ Tool: <full 2 KB of scheduler.js> ] read once — re-sent forever
[ Tool: <200 lines of test output> ]
… and on, and on …
Most of it is dead weight: a file you read and edited ten turns ago is still mailed on every later turn; 15 tool schemas ride along when the agent uses three. That's why long sessions get expensive, not the model "thinking hard," but re-mailing stale paperwork thousands of times. One 70-request session I measured pushed ~3.9M input tokens while the model wrote back a few thousand words.
Anthropic softens this with prompt caching — and it's an explicit model you control. You mark where a cacheable prefix ends with a cache_control
breakpoint:
{ "type": "text", "text": "You are Claude Code…", "cache_control": { "type": "ephemeral" } }
What matters:
Pricing (Opus 4.6 / Bedrock; verified against anthropic.com/pricing on 12 Jul 2026 — rates change): cache read $0.50/MTok vs input $5.00 vs output $25.00.
Claude Code already places its 4 markers well and hits ~99–100% cache. So the real question isn't "turn caching on" — it's how much of that cached snowball can I stop carrying without breaking the prefix?
Because the profiler sits in the middle of every request, it can do more than watch. With the optimize layer on, it tidies the outgoing pile just before each request leaves, never touching the reply, never changing what the agent does next. Biggest levers first:
pruneStale
(~57% of the savings).[read scheduler.js earlier — 47 lines]
. The model already acted on it.pruneUnusedTools
(~23%).insertBreakpoints
.(A fourth lever, collapseSystem
, stubs the repeated system prompt — see the caveats; I'm deliberately not leaning on it.)
To get real numbers I used a fixed benchmark fixture: a small JavaScript project with 9 planted bugs and 3 stubbed-out methods, ~54 visible tests, plus a set of hidden edge-case tests the agent never sees (used to grade quality afterwards). The task handed to Claude Code was blunt — fix everything and make the tests pass. I ran that identical task through the profiler three ways:
Run Reqs Input tok Cache hit Cost vs baseline
──────────────────────────────────────────────────────────────────────────
No optimization 70 3,917,198 100% $2.04 —
Full optimization 94 776,366 100% $0.48 −77%
"Cache-protective"* 118 4,978,066 100% $2.67 +31% WORSE
- don't-prune-anything, to "protect" the cache — the worst result (more below).
Quality was identical across runs: 54/54 fixture tests, same 54/57 edge tests. The optimizer changed the cost, not the outcome.
The naive fear — "editing old messages shatters the prefix and costs more", is backwards here, and not because misses were outweighed by volume. The winning run held 100% hit with only 92 uncached tokens: pruning kept the hit rate and shrank the volume, because insertBreakpoints
re-anchors the markers after the edit. Cached tokens carried per request fell from ~4.3M to ~776K.
Honest note, simulation vs reality.My offline simulator (a strict byte-prefix model) predicted pruning would tank the hit rate to ~67%. The live Bedrock runs held ~100%. Anthropic's real cache tolerates in-region edits far better than a naive model assumes; where they disagreed, I trusted the live runs.
So "protect the cache by not pruning" is exactly wrong on Anthropic-format traffic: carrying 4.3M cached tokens every turn costs more than carrying 776K — even though both are "cached."
I tried the same layer on DeepSeek (OpenAI chat format) and got the opposite result — the identical pruneStale
edit shattered its automatic prefix cache and blew cost up. Different cache, different rules: mutating old context is safe-ish on Anthropic's explicit breakpoints and toxic on a pure prefix cache. I built a separate cache-safe DeepSeek profile instead. Full cross-provider story and data are in the repo. The lesson worth keeping: there's no universal "make it cheaper" button — the right one is provider-aware.
collapseSystem
is unverified.Open source (MIT). In read-only mode it just shows you the firehose — how big the pile is, which files get re-read, where the money goes:
aap serve # the meter on the pipe
aap run claude # launch the agent through it
aap serve --optimize # …and trim the pile (measure on your own provider first!)
The interesting question turned out not to be "how much can I save?" — it's "what is my agent actually sending, and why is it sending it a thousand times?"