{"slug": "the-token-compressor-that-made-my-bill-go-up-and-the-proof-it-had-to", "title": "The token compressor that made my bill go up — and the proof it had to", "summary": "A developer discovered that Paritok, a 4B model that compresses AI coding-agent context, can increase token costs by up to 69.2% on turns where the model expands compressed content, despite reporting 64.0% savings. The structural flaw means even perfect compression cannot win on expanding turns, and the expand call itself provides a free supervision signal that is currently ignored.", "body_md": "I went looking for a small improvement to an open-source tool. I found a number\n\nthat pointed the wrong way, and then I found out why it had to.\n\n**Live demo — paste your own file and watch it happen:\nhttps://pin-on-expand.onrender.com**\n\n[Paritok](https://github.com/Paritok-official/paritok-4b-v1) is a 4B model that\n\ncompresses AI coding-agent context. It sits between your agent and Anthropic or\n\nOpenAI, squeezes the file reads and tool output, and tells you what it saved.\n\nIt's genuinely good work. Trained on 45,000 real agent trajectories, so it knows\n\na function signature matters more than a debug line. Apache 2.0. Runs on a\n\nconsumer GPU. Their benchmark numbers hold up.\n\nI wanted to build a policy improvement on top of it. To prove my improvement\n\nhelped, I first had to measure what stock Paritok cost.\n\nThat measurement is the whole story.\n\nOne coding-agent session. One 20,005-token file in context.\n\n**Paritok's own /stats endpoint: 64.0% of input tokens saved.**\n\n**What the provider was actually POSTed: 69.2% more than sending the file with no\ncompression at all.**\n\nSame session. Same file. Both numbers correct.\n\nParitok is non-destructive by design, which is the good part. Compressed content\n\ngets tagged `[REF:id]`\n\n, and when the model needs the exact original it calls an\n\ninjected `expand_context`\n\ntool to pull it back. Lossy on the wire, recoverable\n\nwhen it counts.\n\nThe proxy answers that call *itself*. It appends the full original to a\n\nproxy-local thread and POSTs that thread upstream a second time.\n\nAnd `stats`\n\nis computed once, in `process_request`\n\n— **before** that loop runs.\n\n```\npost 0:  6,919 tokens   compressed request          ← counted by /stats\npost 1: 26,924 tokens   carries the full original   ← never counted\n                        ─────────\nbilled:  33,843\n```\n\nThen it compounds. The proxy conceals the virtual exchange from the client, so\n\nyour agent never sees it. Next turn the agent re-sends the original file, Paritok\n\nre-compresses it to the same reference, and the model expands it again.\n\nEvery turn. Forever.\n\nIn fairness: Paritok's README says `/stats`\n\nis \"scoped to what Paritok actually\n\nintervenes in.\" Nothing is hidden. But the excluded traffic is *generated by the\ngateway itself* — which makes it the one category you'd most expect to see\n\nI wrote the cost model out to sanity-check my harness. It says something stronger\n\nthan \"there's a bug.\"\n\nLet\nT\nbe the tokens in the file the agent\n\ncarries, and\nc\nthe compression ratio — compressed\n\nover original,\n0<c<1\n, smaller is better.\n\nMeasured on the hosted 4B model:\nc≈0.36\n.\n\nA turn where the model *doesn't* expand costs exactly what compression promises:\n\nA turn where it *does* expand costs the compressed request, plus a second POST\n\ncarrying that same context **and** the restored original:\n\nNow compare that to just sending the file, which costs\n\nT\n:\n\n**On any turn where the model expands, compression cannot win — no matter how\ngood the compressor is.**\n\nSit with that for a second. A *perfect* compressor — one that squeezed your\n\ncontext down to a single token — would still lose on an expanding turn. The\n\nexpansion re-sends the original anyway, and the compressed copy rides along as\n\npure overhead.\n\nThe loss is structural. It lives in the gateway's control flow, not in the\n\nmodel's weights. No amount of better compression fixes it.\n\nAnd here's the neat, horrible part: both numbers a user sees fall out of the same\n\nc\n.\n\nMeasured: 64.0% reported, 69.2% overspent. The gap is message-structure overhead\n\nthe model ignores.\n\nTwo true statements about one session, computed from one variable, pointing in\n\nopposite directions.\n\nWhen the model calls `expand_context`\n\n, it is **telling you that compressing that\ncontent was a mistake.**\n\nNot guessing. Stating it, in a tool call, with the exact reference id. That's a\n\nfree supervision signal, produced by the system itself, on every single failure.\n\nNothing consumes it.\n\nSo honour it: **pin that content and pass it through verbatim from then on.** The\n\nreference never reappears, so there's nothing left to expand, and the turn costs\n\none POST instead of two.\n\nPins key on content hash and source path, so re-reading the same file at a\n\ndifferent offset still hits. Content nobody expands compresses exactly as before.\n\nIt narrows compression *only* where the model has already proven compression\n\nfailed.\n\nOver\nn\nturns where the model needs exact source\n\neach time:\n\nAt\nn=3\n, the model predicts **27.9%**.\n\nMeasured, on two real proxy processes against Paritok's hosted GPU: **29.1%.**\n\n| 3 turns, hosted GPU | POSTs | Billed | vs no proxy |\n|---|---|---|---|\n| stock Paritok | 6 | 104,229 | −73.7% |\n| pin-on-expand | 4 | 73,901 | −23.1% |\n\nAnd the control — model never expands — is a dead heat at 20,787 tokens each.\n\nPinning costs nothing when it never fires.\n\nThe arithmetic and the proxies were built independently, and they agree to about\n\na percentage point. That's the part I'm happiest with.\n\nPinning does not make that session profitable, and I'm not going to pretend it\n\ndoes. It moves a 73.7% overspend to 23.1%. The residual is exactly what the\n\nformula predicts:\n\nThe pin is *learned* from the first expansion, so the first turn is always paid\n\nin full. Pinning converges to break-even from above. It never beats sending the\n\nfile plainly on a session where the model needs the file plainly.\n\nThat's the honest ceiling, and the honest goal is to stop losing badly rather\n\nthan claim a win that isn't there.\n\nI also haven't modelled prompt caching, which would lower the absolute overspend\n\nfigures — though not their direction, since the expanded original is new input\n\nand can't be a cache hit.\n\nWorth saying, because this nearly went out wrong.\n\nMy first fixture had 26 near-identical functions, which Paritok's\n\n`deduplicate_definitions`\n\ncollapsed to almost nothing — a glorious, fake 95%\n\ncompression rate. My SEG-parsing regex matched the literal `[SEG]`\n\nin the\n\nprompt's *instruction text* instead of the actual segment, so every segment\n\ncompressed to `...`\n\n. I used Paritok's own `wrapper.py`\n\nas a fixture without\n\nnoticing it contains the strings `expand_context`\n\nand `[REF:`\n\n, which fired my\n\ndetection logic on the fixture's own text. And the proxy's in-memory cache\n\noutlived a fix, making a working correction look broken.\n\nThree of those four would have shipped a confidently wrong headline.\n\nWorse: my *first* reproduction was right about the wrong thing. I confirmed an\n\nexpand-then-re-collapse loop, then realised it only applies to SDK mode — in\n\nproxy mode the failure is different. Threw the model out and re-derived it from\n\nthe source.\n\nEvery serious bug I hit was in my own measurement, not in Paritok.\n\nWhich is the lesson I'm actually taking from this: **when you're making a claim\nabout someone else's numbers, the infrastructure that proves you're not lying is\nthe deliverable.** The fix is about 120 lines. The harness that earns the right\n\n[https://pin-on-expand.onrender.com](https://pin-on-expand.onrender.com)\n\nPaste any source file. It cold-starts two real `paritok proxy`\n\nprocesses — one\n\nstock, one patched — drives a genuine multi-turn session through each, and shows\n\nyou the reconciliation: every upstream POST, which ones `/stats`\n\ncounted, and\n\nwhat the bill actually was.\n\nNothing on that page is hard-coded. Every run cold-starts its proxies, because\n\nthe compression cache, shadow store and pin set all live for the life of the\n\nprocess — a warm proxy would silently hand back a previous run's answer.\n\nCode, harness, every measurement:\n\n** https://github.com/GauravGupte20/paritok-pin-on-expand** — Apache 2.0, same as\n\nParitok's `level`\n\nparameter — L0 through L3, mapping to real target ratios from\n\n≤0.50 down to ≤0.20 — is fully implemented and wired to the model.\n\nNothing ever selects between values. Tool results always take the default L1.\n\nHistory is hardcoded to L3.\n\nPin-on-expand is the binary case of a policy that should be graded: not *compress\nor don't*, but\n\n`bge-small`\n\nis already a dependency for toolParitok built the dial. It just doesn't have a controller yet.\n\nThat's the thing I originally set out to build, and I still intend to.\n\n*Built with Paritok for their Token-Efficiency Hackathon. #BuiltWithParitok*", "url": "https://wpnews.pro/news/the-token-compressor-that-made-my-bill-go-up-and-the-proof-it-had-to", "canonical_source": "https://dev.to/gaurav_gupte_f260c5ec68e5/the-token-compressor-that-made-my-bill-go-up-and-the-proof-it-had-to-ei", "published_at": "2026-07-30 21:21:42+00:00", "updated_at": "2026-07-30 22:01:06.287801+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-tools", "developer-tools"], "entities": ["Paritok", "Anthropic", "OpenAI"], "alternates": {"html": "https://wpnews.pro/news/the-token-compressor-that-made-my-bill-go-up-and-the-proof-it-had-to", "markdown": "https://wpnews.pro/news/the-token-compressor-that-made-my-bill-go-up-and-the-proof-it-had-to.md", "text": "https://wpnews.pro/news/the-token-compressor-that-made-my-bill-go-up-and-the-proof-it-had-to.txt", "jsonld": "https://wpnews.pro/news/the-token-compressor-that-made-my-bill-go-up-and-the-proof-it-had-to.jsonld"}}