{"slug": "toolgz-cut-llm-tool-definition-tokens-80-without-hurting-accuracy", "title": "Toolgz – cut LLM tool-definition tokens ~80% without hurting accuracy", "summary": "Toolgz, a new open-source library, cuts LLM tool-definition tokens by approximately 80% without hurting accuracy, reclaiming up to 40k tokens of context per request. In a 420-run cross-provider sweep across four frontier models (Claude Opus 5, Grok 4.5, Gemini 3.1 Pro Preview, GPT 5.6 Sol), Toolgz achieved 60/60 tasks completed with zero hallucinated tool names and zero malformed arguments, while reducing prompt tokens by 71–85% and lowering costs by 7–78%.", "body_md": "**Your agent spends 30–50k tokens of context on tool definitions before the user types a word. toolgz gets ~80% of it back.**\n\n[420-run cross-provider sweep](#measured-results) ·\n4 frontier models ·\nzero runtime dependencies ·\n[generated before/after](/dperussina/toolgz/blob/main/docs/BEFORE-AFTER.md)\n\n```\nnpm install toolgz\n```\n\nYou connect a few MCP servers. Each ships 20–50 tools. Every tool is a JSON Schema with a\nsentence of prose per parameter. That block renders at the **front of every single request**.\n\nA realistic tool definition is ~420 tokens, and roughly 400 of them are prose the model doesn't need in order to pick correctly. Fifty tools is 20k tokens. A hundred is 40k.\n\nPrompt caching makes those tokens *cheap*. It does not make them take up less *room*.\nReclaiming the room is what this does.\n\n``` js\nimport { compress, forAnthropic } from \"toolgz\";\n\nconst c = compress(myTools);                  // your existing MCP/SDK tool array\nconst { tools, system } = forAnthropic(c);    // send these instead\n```\n\nThen translate the model's call back before you dispatch:\n\n``` js\nconst r = c.resolve(block.name, block.input);\nif (r.kind === \"call\") await myDispatch(r.name, r.args);   // real name, real args\n```\n\n`myDispatch`\n\ngets exactly what it got before. **Nothing downstream changes.**\n\nFour frontier models, seven strategies, five tool-selection tasks, 3 reps —\n**420 runs** on the current sweep, 1,200+ across all rounds. Every raw per-run record is\ncommitted in [ bench/results/](/dperussina/toolgz/blob/main/bench/results); recompute any figure with\n\n`npx tsx bench/analyze-multi.ts`\n\n.| Provider | Model | Tool block | Prompt tokens | Cost | Latency | Tasks |\n|---|---|---|---|---|---|---|\n| Anthropic | `claude-opus-5` |\n9,242 → 1,284 |\n30,817 → 4,628 (−85%) |\n−78% |\n15.0s → 12.1s |\n15/15 |\n| xAI | `grok-4.5` |\n6,421 → 775 |\n17,522 → 2,663 (−85%) |\n−70% |\n6.1s → 4.6s |\n15/15 |\n`gemini-3.1-pro-preview` |\n5,264 → 732 |\n10,948 → 2,302 (−79%) |\n−62% |\n5.6s → 5.5s |\n15/15 | |\n| OpenAI | `gpt-5.6-sol` |\n2,752 → 573 |\n7,694 → 2,196 (−71%) |\n−7% |\n6.8s → 5.6s |\n15/15 |\n\nReasoning is enabled on all four at high effort, so this is a like-for-like frontier\ncomparison. **60/60 tasks completed, zero hallucinated tool names, zero malformed\narguments** — and it is faster than uncompressed on every provider.\n\nThat was the thing to disprove, and we tried hard to. The task suite is built from\ndeliberately confusable tool clusters — `search_issues`\n\nvs `list_issues`\n\n, comment-vs-update,\napprove-vs-merge, the same three products side by side — where the correct choice turns on\nthe tool name that compression takes away.\n\nThe model doesn't lose the ability to choose — it converts a recall problem into a retrieval\nproblem and looks up what it needs. The default map style exists because of the red cell:\nbare tool names failed on `grok-4.5`\n\n**deterministically**, 3 of 3 attempts on one scenario,\nanswering with zero tool calls and no error raised. Naming the required arguments fixed it.\n\nThe first cross-provider sweep found cost going **up 15% on OpenAI** even while context fell\n69%. The dispatcher was spending extra turns, and on a reasoning model every turn pays for a\nfresh round of thinking.\n\nSo we captured the calls that were being rejected instead of guessing, and found three bugs\nin *this library*: models pass `query`\n\nto a parameter named `q`\n\n(14 of 18 rejections), they\nsometimes call the map code as the tool name, and they sometimes pass arguments flat instead\nof nested. Fixing all three took OpenAI from **+15% to −7%** and drove malformed arguments to\n**zero on every provider**.\n\nOpenAI's −7% is still the smallest saving, and honestly so: reasoning output dominates its\nbill, so a smaller prompt moves the total less. **Context-window occupancy remains the\nprimary claim** — cost follows from it, by an amount that depends on your reasoning settings.\n\nAsk the library. It returns 1 or 3, never 2, and explains itself:\n\n``` js\nimport { recommendLevel } from \"toolgz\";\nconst { level, reason } = recommendLevel(myTools);\n```\n\n| Level | Sends | Real names | Provider schema enforcement | Use when |\n|---|---|---|---|---|\n1 |\none native tool each, signature-line descriptions | yes | yes |\ndefault. Small or wide-and-sparse tool sets. Zero measured downside. |\n| 2 | one compound tool per namespace | yes | no | you need readable op names on the wire. Otherwise skip. |\n3 |\none dispatcher + one lookup tool | codes | no | large, deep tool sets. The 80% number above. |\n\n*(Level 0 is a passthrough, for A/B testing inside your own app.)*\n\n**Level 1 is free** — measured: fewer tokens, zero malformed arguments, zero extra turns,\nlatency no worse. **Level 2 is dominated by level 3** on every axis, including producing more\nmalformed arguments; it is not a stepping stone.\n\nTwo tools on the wire regardless of how many you start with, and a map in the system prompt behind a cache breakpoint:\n\n```\n<toolmap>\na0 github_create_issue owner,repo,title\na1 github_search_issues q\nb0 slack_post_message channel,text\n</toolmap>\n```\n\nThe model calls `t(f=\"a0\", a={…})`\n\n, and `q(c=\"a0\")`\n\nexpands a code to its full signature when\nit needs the optional parameters.\n\nIf you want to remove those lookups entirely, put the whole signature in the map:\n\n```\ncompress(myTools, { level: 3, mapStyle: \"signature\" });\n// a0 github_create_issue(owner,repo,title,body?,labels?)\n```\n\nMeasured: lookups drop to zero and it was the **fastest and cheapest** arm on OpenAI (4.0s,\n−17% cost). It is slightly larger, and on xAI it was worse than the default, so it is an\noption rather than the default.\n\n**The trade:** at levels 2–3 the model fills a generic argument object, so the provider's\nsampler no longer enforces your schema. toolgz validates against your *original* schema and\nreturns a model-readable error instead. That is why `validate`\n\ndefaults to on — leave it on.\n\n**Every artifact above is generated by running the library** — see\n** docs/BEFORE-AFTER.md** for the full tools array and system prompt,\nbefore and after, at every level, with real token counts and a live encode → decode round\ntrip. A test asserts that file matches the code, so it cannot drift.\n\n|\nInstall → working agent loop. Per-provider setup for all four, prompt caching, MCP aggregation, troubleshooting. Start here. |\n|\nGenerated, not illustrated. Both artifacts toolgz modifies, at every level. |\n|\nEvery number, the methodology, and what it does not establish. |\n|\nReady-to-post write-ups for HN and LinkedIn, plus claims not to make. |\n|\nPublishing to npm over OIDC, with no long-lived token. |\n\n``` js\nimport { forAnthropic, forOpenAI, forOpenAIResponses, forGemini } from \"toolgz\";\n```\n\nPure functions; they never mutate what you pass them.\n\n| Adapter | Endpoint | Handles |\n|---|---|---|\n`forAnthropic` |\nMessages API | Places one `cache_control` breakpoint; skips deferred tools, which the API rejects |\n`forOpenAIResponses` |\n`/v1/responses` |\nFlat tool shape. Required if you want tools and reasoning |\n`forOpenAI` |\n`/v1/chat/completions` |\nNested tool shape |\n`forGemini` |\n`generateContent` |\nOne `functionDeclarations` array |\n\nxAI is OpenAI-compatible — use `forOpenAI`\n\nwith `baseURL: \"https://api.x.ai/v1\"`\n\n.\n\n**The size of the cost saving is not the size of the token saving.** Measured 62–78% cheaper on three providers but only 7% on OpenAI, where reasoning output dominates the bill. The claim is context-window occupancy; cost follows, by a variable amount.**It does not beat Anthropic's native tool search on tool-block size.** It composes with it, works where there is no equivalent, and is more reliable below the frontier tier —`defer_loading`\n\ncompleted only 6/30 tasks on Haiku 4.5, silently, because it lets the model*choose*whether to discover tools. A dispatcher makes discovery the entry point.**It has not been measured on a non-frontier model at level 3.** On Haiku 4.5, argument errors rose sharply (17 of 30 runs) — all caught and retried, no task lost, but that is the known edge.**It is not magic on ten tools.** Under ~15 tools there is little to reclaim;`recommendLevel()`\n\nwill tell you so.\n\n`compress()`\n\nis referentially transparent: same tools in, byte-identical payload out. Tools\nare sorted, never left in iteration order.\n\nThis is a correctness property, not tidiness — prompt caching is a prefix match, so one reordered tool silently re-bills your whole prompt. There is a test asserting byte-stability, and it does not get deleted.\n\n```\nnpm test        # 131 tests, offline, no cost\nnpm run build   # tsc → dist/ with .d.ts\n\nnpx tsx bench/harness/run-multi.ts --provider=all --reps=3 --variants   # costs money\nnpx tsx bench/analyze-multi.ts\nnpx tsx docs/generate-examples.ts\n```\n\nMethodology and repo conventions: [AGENTS.md](/dperussina/toolgz/blob/main/AGENTS.md).\nPrinciples specs are checked against: [docs/CONSTITUTION.md](/dperussina/toolgz/blob/main/docs/CONSTITUTION.md).\n\nApache-2.0 — see [LICENSE](/dperussina/toolgz/blob/main/LICENSE) and [NOTICE](/dperussina/toolgz/blob/main/NOTICE).\n\nApache-2.0 rather than MIT deliberately: it carries an express patent grant and a patent-retaliation clause, which matters for a library implementing a technique rather than just glue code, and it is the license most enterprises prefer in a dependency.", "url": "https://wpnews.pro/news/toolgz-cut-llm-tool-definition-tokens-80-without-hurting-accuracy", "canonical_source": "https://github.com/dperussina/toolgz", "published_at": "2026-07-25 21:05:43+00:00", "updated_at": "2026-07-25 21:22:24.400640+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools", "ai-tools"], "entities": ["Toolgz", "Anthropic", "xAI", "OpenAI", "Claude Opus 5", "Grok 4.5", "Gemini 3.1 Pro Preview", "GPT 5.6 Sol"], "alternates": {"html": "https://wpnews.pro/news/toolgz-cut-llm-tool-definition-tokens-80-without-hurting-accuracy", "markdown": "https://wpnews.pro/news/toolgz-cut-llm-tool-definition-tokens-80-without-hurting-accuracy.md", "text": "https://wpnews.pro/news/toolgz-cut-llm-tool-definition-tokens-80-without-hurting-accuracy.txt", "jsonld": "https://wpnews.pro/news/toolgz-cut-llm-tool-definition-tokens-80-without-hurting-accuracy.jsonld"}}