I went looking for a small improvement to an open-source tool. I found a number
that pointed the wrong way, and then I found out why it had to.
Live demo — paste your own file and watch it happen: https://pin-on-expand.onrender.com
Paritok is a 4B model that
compresses AI coding-agent context. It sits between your agent and Anthropic or
OpenAI, squeezes the file reads and tool output, and tells you what it saved.
It's genuinely good work. Trained on 45,000 real agent trajectories, so it knows
a function signature matters more than a debug line. Apache 2.0. Runs on a
consumer GPU. Their benchmark numbers hold up.
I wanted to build a policy improvement on top of it. To prove my improvement
helped, I first had to measure what stock Paritok cost.
That measurement is the whole story.
One coding-agent session. One 20,005-token file in context.
Paritok's own /stats endpoint: 64.0% of input tokens saved.
What the provider was actually POSTed: 69.2% more than sending the file with no compression at all.
Same session. Same file. Both numbers correct.
Paritok is non-destructive by design, which is the good part. Compressed content
gets tagged [REF:id]
, and when the model needs the exact original it calls an
injected expand_context
tool to pull it back. Lossy on the wire, recoverable
when it counts.
The proxy answers that call itself. It appends the full original to a
proxy-local thread and POSTs that thread upstream a second time.
And stats
is computed once, in process_request
— before that loop runs.
post 0: 6,919 tokens compressed request ← counted by /stats
post 1: 26,924 tokens carries the full original ← never counted
─────────
billed: 33,843
Then it compounds. The proxy conceals the virtual exchange from the client, so
your agent never sees it. Next turn the agent re-sends the original file, Paritok
re-compresses it to the same reference, and the model expands it again.
Every turn. Forever.
In fairness: Paritok's README says /stats
is "scoped to what Paritok actually
intervenes in." Nothing is hidden. But the excluded traffic is generated by the gateway itself — which makes it the one category you'd most expect to see
I wrote the cost model out to sanity-check my harness. It says something stronger
than "there's a bug."
Let T be the tokens in the file the agent
carries, and c the compression ratio — compressed
over original, 0<c<1 , smaller is better.
Measured on the hosted 4B model: c≈0.36 .
A turn where the model doesn't expand costs exactly what compression promises:
A turn where it does expand costs the compressed request, plus a second POST
carrying that same context and the restored original:
Now compare that to just sending the file, which costs
T :
On any turn where the model expands, compression cannot win — no matter how good the compressor is.
Sit with that for a second. A perfect compressor — one that squeezed your
context down to a single token — would still lose on an expanding turn. The
expansion re-sends the original anyway, and the compressed copy rides along as
pure overhead.
The loss is structural. It lives in the gateway's control flow, not in the
model's weights. No amount of better compression fixes it.
And here's the neat, horrible part: both numbers a user sees fall out of the same
c .
Measured: 64.0% reported, 69.2% overspent. The gap is message-structure overhead
the model ignores.
Two true statements about one session, computed from one variable, pointing in
opposite directions.
When the model calls expand_context
, it is telling you that compressing that content was a mistake.
Not guessing. Stating it, in a tool call, with the exact reference id. That's a
free supervision signal, produced by the system itself, on every single failure.
Nothing consumes it.
So honour it: pin that content and pass it through verbatim from then on. The
reference never reappears, so there's nothing left to expand, and the turn costs
one POST instead of two.
Pins key on content hash and source path, so re-reading the same file at a
different offset still hits. Content nobody expands compresses exactly as before.
It narrows compression only where the model has already proven compression
failed.
Over n turns where the model needs exact source
each time:
At n=3 , the model predicts 27.9%.
Measured, on two real proxy processes against Paritok's hosted GPU: 29.1%.
| 3 turns, hosted GPU | POSTs | Billed | vs no proxy |
|---|---|---|---|
| stock Paritok | 6 | 104,229 | −73.7% |
| pin-on-expand | 4 | 73,901 | −23.1% |
And the control — model never expands — is a dead heat at 20,787 tokens each.
Pinning costs nothing when it never fires.
The arithmetic and the proxies were built independently, and they agree to about
a percentage point. That's the part I'm happiest with.
Pinning does not make that session profitable, and I'm not going to pretend it
does. It moves a 73.7% overspend to 23.1%. The residual is exactly what the
formula predicts:
The pin is learned from the first expansion, so the first turn is always paid
in full. Pinning converges to break-even from above. It never beats sending the
file plainly on a session where the model needs the file plainly.
That's the honest ceiling, and the honest goal is to stop losing badly rather
than claim a win that isn't there.
I also haven't modelled prompt caching, which would lower the absolute overspend
figures — though not their direction, since the expanded original is new input
and can't be a cache hit.
Worth saying, because this nearly went out wrong.
My first fixture had 26 near-identical functions, which Paritok's
deduplicate_definitions
collapsed to almost nothing — a glorious, fake 95%
compression rate. My SEG-parsing regex matched the literal [SEG]
in the
prompt's instruction text instead of the actual segment, so every segment
compressed to ...
. I used Paritok's own wrapper.py
as a fixture without
noticing it contains the strings expand_context
and [REF:
, which fired my
detection logic on the fixture's own text. And the proxy's in-memory cache
outlived a fix, making a working correction look broken.
Three of those four would have shipped a confidently wrong headline.
Worse: my first reproduction was right about the wrong thing. I confirmed an
expand-then-re-collapse loop, then realised it only applies to SDK mode — in
proxy mode the failure is different. Threw the model out and re-derived it from
the source.
Every serious bug I hit was in my own measurement, not in Paritok.
Which is the lesson I'm actually taking from this: when you're making a claim about someone else's numbers, the infrastructure that proves you're not lying is the deliverable. The fix is about 120 lines. The harness that earns the right
https://pin-on-expand.onrender.com
Paste any source file. It cold-starts two real paritok proxy
processes — one
stock, one patched — drives a genuine multi-turn session through each, and shows
you the reconciliation: every upstream POST, which ones /stats
counted, and
what the bill actually was.
Nothing on that page is hard-coded. Every run cold-starts its proxies, because
the compression cache, shadow store and pin set all live for the life of the
process — a warm proxy would silently hand back a previous run's answer.
Code, harness, every measurement:
** https://github.com/GauravGupte20/paritok-pin-on-expand** — Apache 2.0, same as
Paritok's level
parameter — L0 through L3, mapping to real target ratios from
≤0.50 down to ≤0.20 — is fully implemented and wired to the model.
Nothing ever selects between values. Tool results always take the default L1.
History is hardcoded to L3.
Pin-on-expand is the binary case of a policy that should be graded: not compress or don't, but
bge-small
is already a dependency for toolParitok built the dial. It just doesn't have a controller yet.
That's the thing I originally set out to build, and I still intend to.
Built with Paritok for their Token-Efficiency Hackathon. #BuiltWithParitok