{"slug": "show-hn-turo-an-aggressive-token-saving-proxy-for-cli-ai-agents", "title": "Show HN: Turo – An Aggressive Token-Saving Proxy for CLI AI Agents", "summary": "A developer released Turo, an open-source token-saving proxy for CLI AI agents that reduces prompt token counts by up to 70% by stripping articles, prepositions, and repeated words while preserving code and identifiers. The tool, installable via Homebrew, Go, or shell script, supports multiple compression levels and works with agents like Claude Code, Codex, and Gemini.", "body_md": "**Point more. Token less.**\n\nA real instruction block — **138 tokens** (cl100k):\n\n```\nWhen you are reviewing a pull request, please make sure that you carefully\nexamine each of the changed files and verify that the new code does not\nintroduce any regressions in the existing behavior. It is really important that\nyou check whether the author has added appropriate tests for the new\nfunctionality, because untested code is very likely to break in subtle ways that\nare difficult to debug later. You should also confirm that the documentation has\nbeen updated to reflect the changes, and that the commit messages clearly\nexplain what was changed and why. If you notice any potential security\nvulnerabilities, such as unsanitized user input or hardcoded credentials, you\nmust flag them immediately and request changes before the pull request can be\nmerged.\n```\n\nbecomes **54 tokens — 61% fewer** (`turo`\n\n, meaning intact):\n\n```\nReviewing pull request make examine changed files verify new code introduce\nregressions existing behavior important check author added appropriate tests\nfunctionality untested break subtle ways difficult debug later also confirm\ndocumentation updated reflect changes commit messages explain notice potential\nsecurity vulnerabilities unsanitized user input hardcoded credentials must flag\nmerged\n```\n\nor **41 tokens — 70% fewer** at `--level ultra`\n\n(deduped by lemma):\n\n```\nReview pull request make examine change file verify code introduce regression\nexist behavior important check author add test untested break debug later\nconfirm documentation updated reflect commit message notice security\nvulnerability unsanitized user input hardcoded must flag merge\n```\n\nOr **69 characters** with `--level wenyan`\n\n, which additionally swaps each\nword for a single Classical Chinese character (for CJK-tokenizer models — see\n[wenyan](#wenyan-cjk-tokenizer-models-only)):\n\n```\n閱引請作察變檔驗碼引退存為要查者增試 untested 破診後證文更映交訊覺安隙 unsanitized 戶入 hardcoded 須標併\n```\n\n766 -> 69 characters (35 Han chars). Token counts for that output:\n\n| tokenizer | tokens |\n|---|---|\n| Qwen / DeepSeek / GLM (~1 per Han char) | ~42 |\n| OpenAI cl100k (2-3 per Han char) | 67 |\n\nSo wenyan **wins on CJK-tokenizer models** (~42 vs plain `ultra`\n\n's ~71) but\nloses on OpenAI (67 vs 41) — use it only with CJK models. See below.\n\nNo articles. No prepositions. No adverbs. No repeated words. Only the content words that carry meaning, deduplicated, in reading order. Every prompt, every turn — the savings compound. If a reduction is not smaller than the input, turo passes the original through unchanged.\n\nInstall turo once and any coding agent that can shell out to a binary — Claude Code, Codex, Gemini, Cursor, Windsurf, Cline, Copilot, and 20+ more — pipes its context through the same reducer. Code, paths, and identifiers pass through untouched.\n\n| Method | Command |\n|---|---|\nHomebrew |\n`brew install kdeps/tap/turo` |\nGo |\n`go install github.com/kdeps/turo@latest` |\nShell |\n`curl -fsSL https://raw.githubusercontent.com/kdeps/turo/main/install.sh | sh` |\nManual |\nDownload from\n|\n\n``` php\ncat CLAUDE.md | turo              # text -> deduped content words\necho \"fox jumps over dog\" | turo  # pipe mode\nturo --preamble                   # wrap for system prompt injection\nturo -passes 1                    # single pass (default runs to convergence)\nturo -filler=false                # skip filler deletion\nturo -synonyms=false              # skip the synonym pass (keep words verbatim)\nturo -gloss=false                 # skip the defining-word swap (less lossy)\nturo -arrows                      # replace connective phrases with -> (opt-in)\nturo --version                    # print version\n```\n\n`-gloss`\n\n(on by default) replaces each word with the shortest\nsame-part-of-speech word from its own dictionary definition (`approach`\n\n->\n`come`\n\n). Definitions are prose, not synonyms, so it is the lossiest stage —\ndisable it with `-gloss=false`\n\n/ `TURO_GLOSS=off`\n\nwhen you need words closer to\nthe original.\n\n`-arrows`\n\n(off by default) replaces multi-word causal/sequential connectives\n(`leads to`\n\n, `results in`\n\n, `gives rise to`\n\n, `which produces`\n\n) with a single `->`\n\ntoken. Only multi-word phrases qualify, so the swap always saves at least one\ntoken; single-token connectives (`then`\n\n, `becomes`\n\n, `thus`\n\n) are left alone\nbecause `->`\n\ncosts the same. Enable with `-arrows`\n\n/ `TURO_ARROWS=on`\n\n.\n\n```\nA cache miss leads to a slow query which produces a timeout\n                     |  -arrows\nCache miss -> slow query -> timeout\n```\n\nEvery run is four stages, each on by default:\n\n``` php\ntext -> [1] delete filler -> [2] swap cheaper synonyms -> [3] swap defining words -> [4] reduce to content words\n```\n\n**Filler deletion** removes pleasantries, hedges, and leaders that survive word-level stopword lists (`please`\n\n,`I think`\n\n,`of course`\n\n,`let me`\n\n), while protecting code, paths, URLs, and identifiers verbatim. Disable with`-filler=false`\n\n/`TURO_FILLER=off`\n\n.**Synonym swap** replaces words with a fewer-token synonym (see below). Disable with`-synonyms=false`\n\n/`TURO_SYNONYMS=off`\n\n.**Gloss swap** replaces words with the shortest defining word from their dictionary definition — the lossiest stage. Disable with`-gloss=false`\n\n/`TURO_GLOSS=off`\n\n.**Reduction** drops the remaining stopwords, keeps content words by part of speech, deduplicates, and (ultra) collapses inflections by lemma.\n\n**Arrows** (opt-in, runs before reduction) rewrites multi-word connective\nphrases to `->`\n\n, which the reducer keeps verbatim between the surviving content\nwords. Off by default; enable with `-arrows`\n\n/ `TURO_ARROWS=on`\n\n.\n\nThe whole pipeline repeats until the output stops changing (`-passes 0`\n\n, the\ndefault; a positive `-passes N`\n\ncaps the count). The first pass keeps document\nstructure (headings, per-section bodies); later passes flatten that and dedupe\nacross it, so large structured docs keep shrinking before converging — this\nREADME goes 522 (1 pass) -> 376 tokens (converged, ~29 passes). Set `-passes 1`\n\nto keep it single-shot.\n\nturo never emits output larger than the input: if a stage does not save tokens, the text passes through unchanged.\n\nturo runs a first pass that replaces each word with a fewer-token synonym before\nreducing (`utilize`\n\n-> `use`\n\n, `demonstrate`\n\n-> `show`\n\n). The table is built from\n**WordNet synsets** (real synonyms) and frequency-filtered so swaps land on\ncommon words, then gated to same-part-of-speech words and validated to cost\nstrictly fewer tokens. turo still passes the original through if the result is\nnot smaller.\n\nDisable it with `-synonyms=false`\n\nor `TURO_SYNONYMS=off`\n\nwhen you need words\nverbatim. The gain is usually small — modern tokenizers\nalready encode most words as a single token. WordNet polysemy also leaves some\nnoise (`leverage`\n\n-> `purchase`\n\n), so keep it opt-in for prose, not code. The\ntable is generated by `tools/gensyn.py`\n\n(WordNet + wordfreq + the cl100k\ntokenizer), so token counts are measured for cl100k, not the target model's\ntokenizer.\n\n| Level | What it keeps | Reduction |\n|---|---|---|\nlite |\nAdjectives, nouns, verbs, and leftover adverbs/prepositions | ~65% |\nfull |\nAdjectives, nouns, verbs | ~70% |\nultra (default) |\nNouns and verbs only, deduplicated by lemma (base form) | ~70%+ |\nwenyan |\nultra, then swap surviving words for a single 文言 (Classical Chinese) character | CJK models only |\n\n```\necho \"the quick brown fox jumps over the lazy dog\" | turo --level lite   # quick brown fox jumps over lazy dog\necho \"the quick brown fox jumps over the lazy dog\" | turo --level full   # quick brown fox jumps lazy dog\necho \"the quick brown fox jumps over the lazy dog\" | turo --level ultra  # fox jump dog\necho \"the wise king studies the old book\" | turo --level wenyan    # 智王學舊書\n```\n\n`wenyan`\n\nreduces at ultra, then swaps each surviving English content word\nfor one Classical Chinese character (`water`\n\n-> `水`\n\n, `king`\n\n-> `王`\n\n, `verify`\n\n->\n`驗`\n\n) from a ~380-entry hand-curated lexicon. One char per concept, no spaces\n(Classical Chinese has none).\n\nTwo examples, measured:\n\n| input | ultra | wenyan | chars | cl100k | CJK-model (~1/char) |\n|---|---|---|---|---|---|\n`The wise king uses water and fire...` (80 ch) |\n`Wise king use water fire person see mountain old tree` |\n`智王用水火人見山舊樹` |\n10 |\n15 | ~10 |\n| the PR-review paragraph (766 ch) | 283 ch / 41 tok | `閱引請作察變檔驗碼引退存為要查者增試 untested 破診後證文更映交訊覺安隙 unsanitized 戶入 hardcoded 須標併` |\n69 |\n67 | ~42 |\n\nIt collapses to the fewest **characters** (766 -> 69 on the paragraph). A CJK\ncharacter is 2-3 tokens on OpenAI's cl100k, so `wenyan`\n\nis *larger* there\n(67 > 41). **It only wins on CJK-optimized tokenizers** (Qwen, DeepSeek, GLM),\nwhere a common character is ~1 token — then those 69 chars are ~42 tokens vs\nplain ultra's 71. Don't use it with OpenAI models.\n\nturo's own token estimator counts CJK as 1 rune = 1 token (matching those\nmodels), so it treats `wenyan`\n\nas a reduction and never rejects it. Words\noutside the lexicon stay English (`untested`\n\n, `unsanitized`\n\n, `hardcoded`\n\nabove); code/paths/URLs are preserved verbatim. Extend `wenyanMap`\n\nfor more\ncoverage.\n\nIn **ultra**, inflections of the same word collapse to one token by their\ndictionary base form: `goes`\n\n, `went`\n\n, `going`\n\n-> `go`\n\n; `children`\n\n-> `child`\n\n;\n`servers`\n\n-> `server`\n\n. A reduction is only applied when it lands on a real\ndictionary word, so no mangled non-words are ever emitted.\n\nSet default via `TURO_LEVEL`\n\nenv var.\n\nTo compress **all** input for an agent that turo can't reach from the inside\n(Claude Code, Codex, ...), route the agent's requests through turo.\n\n```\nturo run claude          # every claude request reduced, base URL wired for you\nturo run codex           # OPENAI_BASE_URL wired instead\nturo run                 # list supported agents\n```\n\n`turo run`\n\nstarts an in-process proxy on a free port, points the agent's\nbase-URL env var at it (`ANTHROPIC_BASE_URL`\n\nfor claude, `OPENAI_BASE_URL`\n\nfor\nOpenAI-compatible agents), execs the agent, and stops the proxy when it exits.\nOne command, no exports, no `/turo`\n\ninside the agent. Supported:\n`claude`\n\n, `codex`\n\n, `opencode`\n\n, `qwen`\n\n, `aider`\n\n, `crush`\n\n, `goose`\n\n, `amp`\n\n.\n\n```\nturo -proxy -upstream https://api.openai.com   # listens on 127.0.0.1:8787\nexport OPENAI_BASE_URL=http://127.0.0.1:8787/v1\n```\n\nEvery `/chat/completions`\n\n(and Anthropic `/messages`\n\n) request has its message\ncontent reduced before it reaches the real endpoint; the response streams back\nuntouched. By default only `user`\n\nand `tool`\n\ncontent is reduced (system and\nassistant history are left verbatim, since they are lossier to touch) — pass\n`-proxy-all`\n\nto reduce every role. Auth headers pass through; non-chat paths are\nforwarded unchanged.\n\nkdeps does not need this: in agent mode it already pipes the preamble, input, tool results, and history through turo before every call.\n\n`npx turo`\n\ninstalls the binary **and** registers the turo skill + `/turo`\n\ncommand with every coding agent it finds on your machine — Claude Code, Gemini\nCLI, opencode, Codex, Cursor, Windsurf, Cline, Copilot, and 20+ more. Install\nonce; every agent gets the same reducer.\n\n```\nnpx turo                 # binary + register with detected agents\nnpx turo --list          # show every supported agent and its status\nnpx turo --only claude   # register with one agent\nnpx turo --all           # register with every supported agent\nnpx turo --no-binary     # register agents only (binary already installed)\nnpx turo --uninstall     # remove binary + registered skills\n```\n\nUnder the hood each agent gets one of:\n\n**Claude Code / opencode**— the skill and`/turo`\n\ncommand are copied into the agent's config dir**Gemini CLI**—`gemini extensions install`\n\n**everything else**—`npx skills add kdeps/turo --skill turo -a <profile>`\n\nOnce turo is on PATH, any agent can also pipe context through it directly:\n\n```\ncat CLAUDE.md | turo --preamble    # compact system prompt\ncat error.log | turo               # reduce log output\n```\n\nSet `TURO_LEVEL=ultra`\n\nfor maximum compression. `KDEPS_TURO=off`\n\nor `TURO_DISABLED=1`\n\nto disable.\n\n- Code blocks and inline code — passed through unchanged\n- URLs, file paths, version numbers — verbatim\n- Technical terms (API names, CLI commands, error strings) — exact\n\n- Embedded English dictionary (120k words, 14MB) classifies every word\n- Strips articles, prepositions, conjunctions, pronouns (~70 stop words)\n- Keeps the content words for the level (nouns, verbs, adjectives)\n- Deduplicates and emits them in reading order — then keeps the result only if it is actually smaller than the input\n\n[caveman](https://github.com/JuliusBrussee/caveman) is a sibling idea with the\nopposite dial. caveman deletes filler with regex (articles, pleasantries,\nhedges) and **keeps readable prose**. turo runs that same filler pass, then\nkeeps going — POS-classifying, deduplicating, lemmatizing, and swapping words\nfor shorter synonyms and glosses — trading readability for a much smaller token\ncount.\n\nSame input, measured with the cl100k tokenizer:\n\n| output | tokens | |\n|---|---|---|\n| input | `Please, I think you should really just utilize this approach to demonstrate the functionality of the component.` |\n19 |\n| caveman | `You should utilize this approach to demonstrate functionality of component.` |\n11 |\n| turo full | `Use come show functionality component` |\n5 |\n| turo ultra | `Use come show component` |\n4 |\n\n| caveman | turo | |\n|---|---|---|\n| method | regex filler removal | dict POS + dedup + lemma + synonyms + gloss |\n| output | readable prose | keyword stream |\n| dictionary / WordNet | no | yes |\n| synonym / gloss swaps | no | yes (on by default) |\n| best when | you still need to read it | you only feed it to an LLM |\n\nUse caveman when a human reads the result; use turo when only a model does.\n\nSystem prompts are 50-200k tokens. Most of those words carry grammar, not meaning. Turo points at what matters and drops the rest.\n\nPoint more. Token less.", "url": "https://wpnews.pro/news/show-hn-turo-an-aggressive-token-saving-proxy-for-cli-ai-agents", "canonical_source": "https://github.com/kdeps/turo", "published_at": "2026-07-24 00:32:12+00:00", "updated_at": "2026-07-24 00:52:24.962484+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "artificial-intelligence", "large-language-models"], "entities": ["Turo", "Claude Code", "Codex", "Gemini", "Cursor", "Windsurf", "Cline", "Copilot"], "alternates": {"html": "https://wpnews.pro/news/show-hn-turo-an-aggressive-token-saving-proxy-for-cli-ai-agents", "markdown": "https://wpnews.pro/news/show-hn-turo-an-aggressive-token-saving-proxy-for-cli-ai-agents.md", "text": "https://wpnews.pro/news/show-hn-turo-an-aggressive-token-saving-proxy-for-cli-ai-agents.txt", "jsonld": "https://wpnews.pro/news/show-hn-turo-an-aggressive-token-saving-proxy-for-cli-ai-agents.jsonld"}}