{"slug": "ai-agent-profiler-measure-agent-cost-cache-waste-and-context-bloat", "title": "AI Agent Profiler — Measure agent cost, cache waste, and context bloat", "summary": "A developer built a local-first profiler that acts as a transparent reverse proxy between coding agents and LLM providers, revealing that only ~60% of API costs go to actual prompts while the rest is agent overhead. The tool classifies requests into 11 types and tracks cache waste, showing that a 5-minute cache TTL penalty can cause expensive cold regenerations on long sessions.", "body_md": "I built a local-first profiler that sits as a transparent reverse proxy between your coding agent (Claude Code, OpenCode) and the LLM provider, recording every request without adding latency. It's like `perf`\n\nfor your agent — showing you exactly where your tokens go.\n\n[Live demo (read-only dashboard with sample data).](https://rguiu.github.io/ai-agent-profiler/)\n\n**Only ~60% of my API cost was my actual prompts.** The rest was agent overhead I never saw — subagent exploration (`search`\n\n), context compaction, catch-up summaries when I stepped away, and session-title generation. The profiler classifies every request into 11 kinds (`main`\n\n, `search`\n\n, `compact`\n\n, `recap`\n\n, `title`\n\n, `subagent`\n\n, `webfetch`\n\n, `quota`\n\n, `tool_result`\n\n, `guide`\n\n, `unknown`\n\n) and shows a \"Cost by kind\" table. The `search`\n\nsubagent alone ate ~25-30% of tokens when I told the agent \"explore thoroughly.\"\n\n**The 5-minute cache TTL penalty is real and expensive.** Claude Code places `cache_control: {\"type\": \"ephemeral\"}`\n\nmarkers without an explicit TTL — so your cached prefix (system prompt, tools, message history) expires after ~5 minutes of inactivity. Step away to read docs or grab coffee, come back, and the next request pays a full cache write on potentially 200K+ tokens. On Opus models, a cache write is a massive multiplier on your first request back. The profiler detects cold regenerations, marks them with severity badges, and shows exactly what each idle gap cost you.\n\n**Long sessions compound it.** Every turn grows the cached prefix, so cold-refresh costs grow linearly. A 3-hour session can re-write 200K+ tokens on each expiry. The profiler tracks context growth over time and flags when tool definitions alone account for 10-15K tokens re-sent on every call.\n\n**I tried to optimize and mostly failed.** I built a full optimization layer (pruning stale tool results, collapsing system prompts, removing unused tools) and benchmarked it. Early numbers looked spectacular — but they were wrong. Cross-session cache warming inflated baselines, and the cost model initially ignored cache-write tokens. Once both were fixed, savings vanished. Editing the cached prefix turns cheap reads into expensive writes, and the client re-sends the pristine full history every turn anyway. I wrote up the autopsy in the repo under docs/optimization/FINDINGS.md.\n\n```\nnpm install -g ai-agent-profiler\naap install              # seeds config\naap serve                # terminal 1: proxy + dashboard at :8080/ui\naap run claude           # terminal 2: launch Claude Code through the proxy\n```\n\nSupports Anthropic, OpenAI, DeepSeek, AWS Bedrock (SigV4), and Ollama. Zero telemetry, secrets redacted. All metrics rebuildable from raw traces.\n\nRepo: [https://github.com/rguiu/ai-agent-profiler](https://github.com/rguiu/ai-agent-profiler)", "url": "https://wpnews.pro/news/ai-agent-profiler-measure-agent-cost-cache-waste-and-context-bloat", "canonical_source": "https://dev.to/rguiu/ai-agent-profiler-measure-agent-cost-cache-waste-and-context-bloat-p86", "published_at": "2026-07-21 22:14:18+00:00", "updated_at": "2026-07-21 22:29:35.208100+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "large-language-models", "ai-infrastructure"], "entities": ["Claude Code", "OpenCode", "Anthropic", "OpenAI", "DeepSeek", "AWS Bedrock", "Ollama", "ai-agent-profiler"], "alternates": {"html": "https://wpnews.pro/news/ai-agent-profiler-measure-agent-cost-cache-waste-and-context-bloat", "markdown": "https://wpnews.pro/news/ai-agent-profiler-measure-agent-cost-cache-waste-and-context-bloat.md", "text": "https://wpnews.pro/news/ai-agent-profiler-measure-agent-cost-cache-waste-and-context-bloat.txt", "jsonld": "https://wpnews.pro/news/ai-agent-profiler-measure-agent-cost-cache-waste-and-context-bloat.jsonld"}}