{"slug": "lossless-codec-for-ai-agent-messages-36-fewer-tokens-overhead-counted", "title": "Lossless codec for AI agent messages – 36% fewer tokens, overhead counted", "summary": "A new lossless wire format called a2acompress cuts token usage by 36.6% on real cl100k_base tokens for agent-to-agent handoffs, reducing 140,661 raw tokens to 89,235 full-cost tokens across 196 held-out ToolBench trajectories (3,013 handoffs). The codec, which includes a benchmark harness that counts all overhead, is available via pip and GitHub, and its test suite enforces byte-exact round trips and strict train/test splits.", "body_md": "**A compact, lossless wire format for agent-to-agent handoffs — cuts 36% of real\ncl100k_base tokens on held-out ToolBench trajectories, with every byte of\noverhead counted against it.**\n\nMulti-agent pipelines (planner → builder → reviewer, tool loops, LangGraph / CrewAI / AutoGen-style orchestration) burn most of their context window re-sending structure: JSON keys, quoting, restated identifiers, tool catalogs, and outputs quoted verbatim two handoffs later. If you control both sides of a handoff, that structure is free to compress — as long as the compression is exactly reversible and you are honest about what decoding costs.\n\n`a2acompress`\n\nis that codec, plus the benchmark harness that keeps it honest.\n\n```\nraw canonical JSON   ████████████████████████████████████  140,661 tokens\na2c/1 full cost      ███████████████████████               89,235 tokens   −36.6%\n                     └── payload 79,439 + decode instructions 9,796 + dictionary 0\n```\n\n*Held-out split of 196 real ToolBench trajectories (3,013 handoffs), actual\ncl100k_base token counts, byte-exact round trip on every record. Reproduce\nwith one command below.*\n\nMost compression demos compare payload bytes and skip the part where the\nreceiver needs to be told how to read the payload. Here, **full cost** is the\nonly headline number:\n\n```\nfull cost = compact payload\n          + the reconstruction instructions a model needs to decode it\n          + any learned dictionary shipped alongside\n```\n\nThree more rules, enforced by the test suite and CI:\n\n**Byte-exact or rejected.** Every configuration must decode back to identical canonical JSON for every held-out record, or it cannot win.**Train/test split by complete trajectory.** Anything learned (dictionaries, tuning choices) sees training tasks only; the reported number comes from tasks the optimizer never touched, re-verified across three more split seeds.**No invented tokens.** GPT and Claude APIs cannot consume custom token IDs. Everything here is ordinary text counted with the real`cl100k_base`\n\nencoder. Transport size (gzip) is reported separately and never sold as token savings.\n\n```\npip install -e .\na2acompress optimize --adapter synthetic --limit 40 --output release\n```\n\nThat runs the full bounded optimize loop on a synthetic corpus and writes a\nrelease folder: report, results JSON, the selected protocol, charts, and a\n`practical`\n\n/ `research-only`\n\nverdict.\n\nThe real benchmark on public ToolBench trajectories:\n\n```\na2acompress fetch --dataset toolbench --limit 240 --output data/toolbench\na2acompress optimize --adapter toolbench --source data/toolbench --seed 7 --output release\n```\n\nOr use the codec directly:\n\n``` python\nfrom a2acompress.protocol import ProtocolOptions, encode_session, decode_session\n\noptions = ProtocolOptions(\n    positional_body=True, packed_header=True,\n    minimal_instructions=True, inline_backrefs=True, token_aligned_refs=True,\n)\nencoded = encode_session(records, options)      # one task trajectory in\nrestored = decode_session(encoded.payload, options)\nassert restored == records                      # always, or it's a bug\n```\n\nCLI: `fetch`\n\n· `normalize`\n\n· `train`\n\n· `encode`\n\n· `decode`\n\n· `verify`\n\n·\n`benchmark`\n\n· `optimize`\n\n.\n\nEvery handoff (plan, message, tool call/result, status, artifact, review, error, final) is first normalized to one canonical record shape. Records then encode to one line each:\n\n```\n#task|agents\nc13|tool_catalog|image_search;{\"count\":5,\"q\":\"web design\"};c1\nr31||c1;The search returned 5 results: ${2f:8a} …\nf10||done — ${0:2q:1d4}\n```\n\nThe savings stack, all reversible:\n\n| Mechanism | What it does |\n|---|---|\n| compact schema | one-char keys, defaults omitted, positional fields, packed headers |\n| value references | `$3` repeats an earlier value; the first occurrence is the definition |\ninline backreferences |\n`${o:l}` splices a character span from earlier in the session — LZ-style, but the payload stays readable text and no dictionary ships |\n| token-aligned splices | ref boundaries shift a few chars when the measured token cost improves; refs that don't beat their own literal are dropped |\n| adaptive instructions | the decode instructions ship per session, compressed, and only describe features that session actually uses |\n\nThe inline backreference is the workhorse: final answers quote tool outputs verbatim, tool catalogs repeat their own envelope per tool — one 6-token marker replaces spans of any length, and the decoder rebuilds everything by replaying the session. On ToolBench it fully displaced the learned dictionary (the optimizer kept 0 entries: shipping a dictionary per session costs more than it saves — measured, not assumed).\n\nEvery constant in the format — separator characters, marker syntax, span\nthresholds — was chosen by measuring real `cl100k_base`\n\ncosts, not guessed.\nThe alternatives that lost (tab separators, longer n-grams, lazy dictionary\nbootstrap) are documented in the release report with their numbers.\n\n`a2acompress optimize`\n\nis a bounded search, not a demo script:\n\n- evaluate the baseline protocol across a grid of configurations\n- verify losslessness for every configuration; failures cannot win\n- enable one improvement per cycle, chosen by a probe on\n**training** data - stop after two stale cycles or 10 cycles; select the cheapest verified winner\n- re-test the winner on three other split seeds and report the spread\n- write a release folder with a manifest (seed, corpus fingerprint, search space) so any result is reproducible bit-for-bit\n\nVerdict thresholds: `practical`\n\nneeds ≥15% full-cost saving *and* break-even\nwithin 10 handoffs. The current winner saves 36.6% and breaks even on the\n**first** handoff of the median session.\n\n**Good fit:** you run a multi-agent system where the same process (or two\nprocesses you control) encodes and decodes handoffs, you pay per token, and\nyour handoffs carry repeated structure — tool catalogs, quoted outputs,\nschema-shaped messages.\n\n**Not a fit:**\n\n- You want a drop-in proxy that shrinks arbitrary prompts. This is a protocol; both sides must speak it.\n- Your receiving model must act on the compact form directly with zero decode errors at any stakes. A model reads the format from the shipped instructions; that path is honest about its token cost but is still a model reading a spec — test on your own traffic before trusting it.\n- Your messages are mostly unique prose. The ~63% that survives compression\nhere\n*is*the unique content; a corpus without structural repetition will save less.\n\n**Related work:** [TOON](https://github.com/toon-format/toon) shrinks tabular\nJSON for prompts by declaring shape once (~40% on uniform arrays);\nLLMLingua-style prompt compression is lossy. `a2acompress`\n\ndiffers by being\nlossless on heterogeneous agent trajectories, adding session-replay references\nand LZ-style splices, and charging itself for every byte of decode overhead.\n\n| Held-out metric (seed 7) | Value |\n|---|---|\n| Raw canonical JSON | 140,661 tokens |\n| Compact payload | 79,439 |\n| Reconstruction instructions | 9,796 |\n| Learned dictionary | 0 |\nFull cost |\n89,235 (−36.6%) |\n| Payload-only (not the claim) | −43.5% |\n| Median break-even | 1 handoff |\n| Sessions never breaking even | 0 of 39 |\n| Stability on seeds 11/23/47 | −36.1% / −36.0% / −35.2% |\n| Round trip | byte-exact, every record |\n\nFull detail: [release/optimization_report.md](/reh8n/a2acompress/blob/main/release/optimization_report.md),\nevery evaluated configuration in\n[release/optimization_results.json](/reh8n/a2acompress/blob/main/release/optimization_results.json).\n\nBenchmarked on one public corpus (ToolBench) so far. The `jsonl`\n\nadapter takes\nyour own traces — a `role`\n\n/`content`\n\nJSONL is enough — and the loop will tell\nyou *your* number:\n\n```\na2acompress optimize --adapter jsonl --source your_traces.jsonl --output my_release\n```\n\nIf you run it on your own corpus, an issue with your numbers (good or bad) is the most useful contribution you can make.\n\n```\npip install -e \".[dev]\"\npython -m pytest tests -q\n```\n\nCI runs the unit tests, lossless round-trip fixtures against adversarial inputs, a synthetic benchmark, the full optimize loop, and a release checker that fails the build if a report's arithmetic, verdict, or stability re-tests don't hold up.\n\nA TypeScript port of the codec (byte-parity with Python, verified by cross-tests) powers a browser lab that encodes, decodes, and verifies the round trip live; it lives in a companion project and will be published separately.\n\nMIT — see [LICENSE](/reh8n/a2acompress/blob/main/LICENSE).", "url": "https://wpnews.pro/news/lossless-codec-for-ai-agent-messages-36-fewer-tokens-overhead-counted", "canonical_source": "https://github.com/reh8n/a2acompress", "published_at": "2026-08-13 04:33:28+00:00", "updated_at": "2026-08-13 04:40:42.279266+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-infrastructure", "developer-tools"], "entities": ["a2acompress", "ToolBench", "LangGraph", "CrewAI", "AutoGen", "cl100k_base"], "alternates": {"html": "https://wpnews.pro/news/lossless-codec-for-ai-agent-messages-36-fewer-tokens-overhead-counted", "markdown": "https://wpnews.pro/news/lossless-codec-for-ai-agent-messages-36-fewer-tokens-overhead-counted.md", "text": "https://wpnews.pro/news/lossless-codec-for-ai-agent-messages-36-fewer-tokens-overhead-counted.txt", "jsonld": "https://wpnews.pro/news/lossless-codec-for-ai-agent-messages-36-fewer-tokens-overhead-counted.jsonld"}}