{"slug": "deterministic-serialization-for-multi-agent-llm-sessions-3-45x-fewer-tokens-than", "title": "Deterministic serialization for multi-agent LLM sessions - 3.45x fewer tokens than JSON, up to 9.9x for non-English content", "summary": "A developer built a serialization protocol called SCP that reduces token usage in multi-agent LLM sessions by up to 3.45x compared to JSON, and up to 9.9x for non-English content. The protocol replaces natural language or JSON messages with short positional ASCII identifiers resolved against an external dictionary, eliminating hallucination risk on decode. Benchmarks show significant savings, especially for non-English languages, though caching discounts from providers like Anthropic and OpenAI may affect real-world cost benefits.", "body_md": "Multi-agent LLM systems -” several models exchanging messages within\n\none session -” pay for context, not intelligence. Every round trip in\n\nnatural language or verbose JSON burns tokens re-stating structured\n\ncontext that a fixed, external schema could carry in a fraction of\n\nthe size.\n\nI got tired of watching this happen in my own pipelines, so I built a\n\nsmall serialization protocol to fix it. Sharing it here in case it's\n\nuseful to others hitting the same wall.\n\nMove inter-agent messages from natural language / JSON to short,\n\npositional ASCII identifiers (`P1:A2:X0:V4`\n\n), resolved against an\n\nexternal, versioned `dictionary.json`\n\n. A deterministic Python layer\n\nhandles encode/decode -” no model involved in reconstructing meaning,\n\nso there's no hallucination risk on the decode side.\n\n``` php\ndef encode(payload: dict, schema: dict) -> str:\n    parts = []\n    for field_name, field_id in schema[\"fields\"].items():\n        if field_name not in payload:\n            continue\n        value = str(payload[field_name])\n        value_id = schema[\"values\"][field_name][value]\n        parts.append(f\"{field_id}{value_id}\")\n    return \":\".join(parts)\n```\n\nUnknown fields or values raise an explicit error instead of guessing -”\n\nthe whole point of an external schema is that the model never has to\n\nimprovise meaning on decode.\n\nConceptually this is closer to **Protocol Buffers** than to prompt\n\nengineering: a fixed contract, not a clever prompt.\n\nMeasured on `cl100k_base`\n\n(industry-standard reference tokenizer):\n\n| Format | Tokens |\n|---|---|\n| Natural language (RU) | 49 |\n| Standard JSON | 38 |\n| SCP ASCII ID-stack | 11 |\n\n**3.45x fewer tokens than JSON.** Full reproducible benchmark script\n\nis in the repo -” run it yourself against your own tokenizer before\n\ntrusting these numbers for a cost projection.\n\nTokenizer vocabularies are trained predominantly on English text, so\n\nnon-Latin scripts pay a real, measurable tax. Same sentence, same\n\nmeaning, measured multiplier vs. the SCP ID-stack:\n\n| Language | Multiplier vs. SCP |\n|---|---|\n| English | 1.89x |\n| Russian | 5.11x |\n| Arabic | 5.56x |\n| Japanese | 4.22x |\n| Hindi | 9.89x |\n\nBecause the ID-stack costs the same regardless of source language (9\n\ntokens either way -” it's just ASCII after encoding), SCP's savings\n\nscale disproportionately for non-English multi-agent deployments.\n\nThat's not a marketing angle, it's just what the tokenizer does.\n\n`cl100k_base`\n\nas a common reference point. If you're\ndeploying against a different model family, re-run the benchmark\nscript against that tokenizer before relying on these numbers.Anthropic and OpenAI both offer ~90% discounts on cached input tokens.\n\nThree conditions determine whether SCP's savings actually materialize\n\nin a caching setup:\n\n```\npython mvp/encoder_decoder.py encode '{\"system\": \"Quantumoan\", \"version\": \"4\", \"action\": \"paradigm_shift\", \"target\": \"cognitive_profiles_alignment\"}'\n# -> P1:V4:A2:X0\n\npython mvp/encoder_decoder.py decode \"P1:V4:A2:X0\"\n# -> {\"system\": \"Quantumoan\", \"version\": \"4\", \"action\": \"paradigm_shift\", \"target\": \"cognitive_profiles_alignment\"}\n```\n\nRepo (AGPLv3): [https://github.com/andrey-architect/scp-protocol](https://github.com/andrey-architect/scp-protocol)\n\nWould genuinely like to know where this breaks -” issues and PRs welcome.", "url": "https://wpnews.pro/news/deterministic-serialization-for-multi-agent-llm-sessions-3-45x-fewer-tokens-than", "canonical_source": "https://dev.to/andreyarchitect/deterministic-serialization-for-multi-agent-llm-sessions-345x-fewer-tokens-than-json-up-to-99x-3pl2", "published_at": "2026-07-18 19:57:43+00:00", "updated_at": "2026-07-18 20:28:16.013982+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "developer-tools"], "entities": ["Anthropic", "OpenAI", "SCP", "cl100k_base"], "alternates": {"html": "https://wpnews.pro/news/deterministic-serialization-for-multi-agent-llm-sessions-3-45x-fewer-tokens-than", "markdown": "https://wpnews.pro/news/deterministic-serialization-for-multi-agent-llm-sessions-3-45x-fewer-tokens-than.md", "text": "https://wpnews.pro/news/deterministic-serialization-for-multi-agent-llm-sessions-3-45x-fewer-tokens-than.txt", "jsonld": "https://wpnews.pro/news/deterministic-serialization-for-multi-agent-llm-sessions-3-45x-fewer-tokens-than.jsonld"}}