cd /news/ai-tools/kvdiff-local-proxy-showing-the-byte-… · home › topics › ai-tools › article
[ARTICLE · art-173973] src=github.com ↗ pub= topic=ai-tools verified=true sentiment=↑ positive

Kvdiff – Local proxy showing the byte breaking your LLM prompt cache

A developer released kvdiff v0.1.0, a zero-code local proxy that identifies the exact UTF-8 byte offset breaking an LLM prompt cache across Anthropic, OpenAI, Google Gemini/Antigravity, llama.cpp and vLLM, and tallies the wasted spend. In its bundled demo, a Claude Code session with 15 MCP tools runs at 94% cache hit until Turn 14, when mcp-github reconnects and reorders tool schemas; kvdiff flags the break at $.tools[9].name, byte 1,670, calculates $0.38 in wasted cost, and shows Turn 15 returning to 97% cache hit under the opt-in --heal flag. The tool runs via npx @drod915/kvdiff and classifies breaks into six kinds, including TOOL_REORDER, VOLATILE_SYSTEM and TTL_EXPIRED.

key takeaway

Run kvdiff as a local proxy to see the exact UTF-8 byte and JSONPath where a prompt prefix diverges, and pass --heal to sort tools[] deterministically so MCP reconnects stop breaking the cache.

read4 min views3 publishedNov 21, 2026
Kvdiff – Local proxy showing the byte breaking your LLM prompt cache
Image: Michielbdejong (auto-discovered)

Zero-code local proxy that pinpoints the exact UTF-8 byte breaking your LLM prompt cache (Anthropic, OpenAI, Google Gemini / Antigravity, llama.cpp/vLLM), tallies wasted dollars, and optionally auto-heals prefixes live.

Watch the 15-second kvdiff demo (MP4)

Try the demo: npx @drod915/kvdiff --demo

npx @drod915/kvdiff --demo

Opens http://127.0.0.1:4318 pre-seeded with two realistic sessions:

  1. Claude Code + 15 MCP Tools (16 turns): Turn 1 warms the prompt cache at 0%, and Turns 2-13 run at 94% cache hit. OnTurn 14 ,mcp-github reconnects and reorders tool schemas:kvdiff highlights the exact red byte offset ($.tools[9].name , byte 1,670), calculates the wasted cost ($0.38), and showsTurn 15 snapping back to97% cache hit under--heal .
  2. Antigravity / Gemini 2.5 Pro (5 turns): Catches a volatile timestamp injected into$.systemInstruction.parts[0].text on Turn 4.

Record the demo in a terminal with vhs demo/kvdiff.tape.

Wrap any coding agent CLI with zero code changes:

npx @drod915/kvdiff -- claude

npx @drod915/kvdiff -- codex

npx @drod915/kvdiff -- gemini

npx @drod915/kvdiff --heal -- claude

Or run as a persistent local proxy for Antigravity, Cursor, IDE plugins, or local llama.cpp/vLLM:

npx @drod915/kvdiff serve --port 4318

npx @drod915/kvdiff serve --port 4318 --upstream-local http://127.0.0.1:8080

Then point your client base URL:

  • ANTHROPIC_BASE_URL=http://127.0.0.1:4318
  • OPENAI_BASE_URL=http://127.0.0.1:4318/v1
  • GOOGLE_GEMINI_BASE_URL=http://127.0.0.1:4318
Break Kind What Happened on the Wire Fix
TOOL_REORDER MCP servers reconnected or registered tools in non-deterministic order Pass --heal to sorttools[] (and GeminifunctionDeclarations[] ) deterministically
TOOL_MUTATED A dynamic tool description or schema field changed mid-session Pin static tool descriptions outside the cached prefix
VOLATILE_SYSTEM A timestamp, UUID, or mutable status string landed inside $.system /$.systemInstruction Move dynamic metadata into the trailing user turn
TTL_EXPIRED Bytes matched 100%, but >5m elapsed between turns Keep interactive sessions warm or opt into extended TTL
MODEL_CHANGED Model name switched mid-session Keep one model per cache prefix
PREFIX_TOO_SHORT Prompt fell under OpenAI ( 1,024 ) or Gemini (1,024 /2,048 ) implicit-cache floor Expected until context grows past threshold
Feature kvdiff ccusage /codexbar ContextSpy Langfuse / Phoenix
Install & Infra 1 cmd ( npx ), built-innode:sqlite 1 cmd ( npx /binary) 1 cmd ( npx ) Docker + Postgres/ClickHouse
Wire-level UTF-8 byte LCP + JSONPath diff Yes No (reads post-hoc logs) No (token category pies) No
Opt-in live prefix healing (--heal) Yes No No No
Multi-provider (Anthropic + OpenAI + Gemini/Antigravity + llama.cpp) Yes Single-provider focus Anthropic/OpenAI Yes
Zero-mutation default passthrough Yes (OAuth/beta headers intact) N/A (offline log reader) Yes SDK instrumentation
Where competitors win - Zero proxy setup; great historical billing totals across past months High-level token breakdown visualizations Production team evals, prompt versioning, multi-user cloud traces
flowchart LR
    Agent["Coding Agent / IDE\n(Claude Code · Codex · Antigravity · Gemini · llama.cpp)"] -->|"HTTP (127.0.0.1:4318)"| Proxy["kvdiff Proxy\nZero-mutation default\n(opt-in --heal)"]
    Proxy -->|"Immediate chunked SSE flush"| Upstream["Upstream Provider\nAnthropic · OpenAI · Gemini · Local"]
    Proxy -.->|"Non-blocking SSE tee"| Engine["UTF-8 Byte LCP +\nJSONPath Mapper +\nBreak Classifier"]
    Engine --> DB[("~/.kvdiff/kvdiff.db\nNode 24 node:sqlite WAL")]
    DB --> UI["Localhost Waterfall UI\n127.0.0.1:4318"]
  • 100% Local: Binds to127.0.0.1 by default. Stores traffic only in~/.kvdiff/kvdiff.db . Zero telemetry, zero cloud accounts.
  • Strict Zero-Mutation Default: Without--heal ,kvdiff forwards your exact raw request bytes and every non-hop-by-hop header (Authorization ,x-api-key ,x-goog-api-key ,anthropic-beta ,openai-beta ) unmodified.
  • Zero-Buffering SSE: Streaming responses flush downstream chunk-by-chunk as they arrive from upstream.
  • Safe --heal Invariant:--heal sorts tools and tool-definition keys, normalizes JSON formatting, and keeps the trailing Anthropic cache breakpoint on the final tool after sorting. Itnever reordersmessages[] ,contents[] ,parts[] ,content[] ,input[] , or JSON Schemaenum[] .
git clone https://github.com/flipcyde101/kvdiff.git
cd kvdiff
npm install
npm run typecheck
npm run build
npm test

Found a cache-break pattern or provider edge case? Open an issue at github.com/flipcyde101/kvdiff/issues.

── more in #ai-tools 4 stories · sorted by recency
── more on @kvdiff 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/kvdiff-local-proxy-s…] indexed:0 read:4min 2026-11-21 · —