Kvdiff – Local proxy showing the byte breaking your LLM prompt cache A developer released kvdiff v0.1.0, a zero-code local proxy that identifies the exact UTF-8 byte offset breaking an LLM prompt cache across Anthropic, OpenAI, Google Gemini/Antigravity, llama.cpp and vLLM, and tallies the wasted spend. In its bundled demo, a Claude Code session with 15 MCP tools runs at 94% cache hit until Turn 14, when mcp-github reconnects and reorders tool schemas; kvdiff flags the break at $.tools[9].name, byte 1,670, calculates $0.38 in wasted cost, and shows Turn 15 returning to 97% cache hit under the opt-in --heal flag. The tool runs via npx @drod915/kvdiff and classifies breaks into six kinds, including TOOL_REORDER, VOLATILE_SYSTEM and TTL_EXPIRED. Zero-code local proxy that pinpoints the exact UTF-8 byte breaking your LLM prompt cache Anthropic, OpenAI, Google Gemini / Antigravity, llama.cpp /vLLM , tallies wasted dollars, and optionally auto-heals prefixes live. Watch the 15-second kvdiff demo MP4 https://github.com/flipcyde101/kvdiff/releases/download/v0.1.0/kvdiff-scoped-demo.mp4 Try the demo: npx @drod915/kvdiff --demo npx @drod915/kvdiff --demo Opens http://127.0.0.1:4318 pre-seeded with two realistic sessions: 1. Claude Code + 15 MCP Tools 16 turns : Turn 1 warms the prompt cache at 0%, and Turns 2-13 run at 94% cache hit. On Turn 14 , mcp-github reconnects and reorders tool schemas: kvdiff highlights the exact red byte offset $.tools 9 .name , byte 1,670 , calculates the wasted cost $0.38 , and shows Turn 15 snapping back to 97% cache hit under --heal . 2. Antigravity / Gemini 2.5 Pro 5 turns : Catches a volatile timestamp injected into $.systemInstruction.parts 0 .text on Turn 4. Record the demo in a terminal with vhs demo/kvdiff.tape . Wrap any coding agent CLI with zero code changes: Claude Code Anthropic Messages API npx @drod915/kvdiff -- claude Codex CLI / OpenCode OpenAI Responses & Chat APIs npx @drod915/kvdiff -- codex Gemini CLI / Antigravity API workflows Google Gemini API npx @drod915/kvdiff -- gemini Opt-in deterministic tool-schema healing npx @drod915/kvdiff --heal -- claude Or run as a persistent local proxy for Antigravity, Cursor, IDE plugins, or local llama.cpp /vLLM: Cloud providers Anthropic, OpenAI, Google Gemini / Antigravity npx @drod915/kvdiff serve --port 4318 Local llama.cpp / vLLM KV-cache prefix reuse debugging npx @drod915/kvdiff serve --port 4318 --upstream-local http://127.0.0.1:8080 Then point your client base URL: - ANTHROPIC BASE URL=http://127.0.0.1:4318 - OPENAI BASE URL=http://127.0.0.1:4318/v1 - GOOGLE GEMINI BASE URL=http://127.0.0.1:4318 | Break Kind | What Happened on the Wire | Fix | |---|---|---| | TOOL REORDER | MCP servers reconnected or registered tools in non-deterministic order | Pass --heal to sort tools and Gemini functionDeclarations deterministically | | TOOL MUTATED | A dynamic tool description or schema field changed mid-session | Pin static tool descriptions outside the cached prefix | | VOLATILE SYSTEM | A timestamp, UUID, or mutable status string landed inside $.system / $.systemInstruction | Move dynamic metadata into the trailing user turn | | TTL EXPIRED | Bytes matched 100%, but 5m elapsed between turns | Keep interactive sessions warm or opt into extended TTL | | MODEL CHANGED | Model name switched mid-session | Keep one model per cache prefix | | PREFIX TOO SHORT | Prompt fell under OpenAI 1,024 or Gemini 1,024 / 2,048 implicit-cache floor | Expected until context grows past threshold | | Feature | kvdiff | ccusage / codexbar | ContextSpy | Langfuse / Phoenix | |---|---|---|---|---| | Install & Infra | 1 cmd npx , built-in node:sqlite | 1 cmd npx /binary | 1 cmd npx | Docker + Postgres/ClickHouse | | Wire-level UTF-8 byte LCP + JSONPath diff | Yes | No reads post-hoc logs | No token category pies | No | | Opt-in live prefix healing --heal | Yes | No | No | No | | Multi-provider Anthropic + OpenAI + Gemini/Antigravity + llama.cpp | Yes | Single-provider focus | Anthropic/OpenAI | Yes | | Zero-mutation default passthrough | Yes OAuth/beta headers intact | N/A offline log reader | Yes | SDK instrumentation | | Where competitors win | - | Zero proxy setup; great historical billing totals across past months | High-level token breakdown visualizations | Production team evals, prompt versioning, multi-user cloud traces | flowchart LR Agent "Coding Agent / IDE\n Claude Code · Codex · Antigravity · Gemini · llama.cpp " -- |"HTTP 127.0.0.1:4318 "| Proxy "kvdiff Proxy\nZero-mutation default\n opt-in --heal " Proxy -- |"Immediate chunked SSE flush"| Upstream "Upstream Provider\nAnthropic · OpenAI · Gemini · Local" Proxy -.- |"Non-blocking SSE tee"| Engine "UTF-8 Byte LCP +\nJSONPath Mapper +\nBreak Classifier" Engine -- DB "~/.kvdiff/kvdiff.db\nNode 24 node:sqlite WAL" DB -- UI "Localhost Waterfall UI\n127.0.0.1:4318" - 100% Local: Binds to 127.0.0.1 by default. Stores traffic only in ~/.kvdiff/kvdiff.db . Zero telemetry, zero cloud accounts. - Strict Zero-Mutation Default: Without --heal , kvdiff forwards your exact raw request bytes and every non-hop-by-hop header Authorization , x-api-key , x-goog-api-key , anthropic-beta , openai-beta unmodified. - Zero-Buffering SSE: Streaming responses flush downstream chunk-by-chunk as they arrive from upstream. - Safe --heal Invariant: --heal sorts tools and tool-definition keys, normalizes JSON formatting, and keeps the trailing Anthropic cache breakpoint on the final tool after sorting. It never reorders messages , contents , parts , content , input , or JSON Schema enum . git clone https://github.com/flipcyde101/kvdiff.git cd kvdiff npm install npm run typecheck npm run build npm test Found a cache-break pattern or provider edge case? Open an issue at github.com/flipcyde101/kvdiff/issues https://github.com/flipcyde101/kvdiff/issues .