{"slug": "tokencompress-a-sub-2ms-go-cli-and-mcp-sidecar-that-prunes-ai-agent-tool-context", "title": "Tokencompress A sub-2ms Go CLI and MCP sidecar that prunes AI agent tool context", "summary": "Tokencompress, a zero-dependency Go CLI and MCP sidecar, cuts AI agent tool context token consumption by 60% to 80% without a second LLM summarization turn, according to its developer. The tool prunes raw tool outputs such as JSON, terminal logs, and HTML before they enter an agent's context window, with benchmarks showing reductions of 93.2% for a 500-item JSON array, 87.1% for a Python stack trace, and 88.3% for an HTML web scrape. It is available under the MIT License and can be integrated with Claude Desktop via MCP.", "body_md": "A zero-dependency, sub-millisecond Go CLI and MCP sidecar that prunes raw tool outputs (JSON, terminal logs, HTML) before they enter your AI agent's context window.\n\nCut LLM context token consumption by **60% to 80%** without using a second LLM summarization turn.\n\nWhen AI agents execute tools (calling APIs, running terminal commands, or scraping web pages), they receive thousands of lines of raw, unparsed data:\n\n**Massive JSON Payloads:** A 1,000-item array floods the prompt with 15,000+ tokens.**Verbose Stack Traces:** Framework noise and node_modules paths drown out the root exception.**Raw HTML:** CSS, scripts, and navigation menus bloat the context window.\n\nThis leads to high API costs, slower response times, and **context rot**—where agents hallucinate or repeat tool calls mid-task.\n\nacts as a high-speed, deterministic filter between your tools and your model:\n\n**JSON Truncation:** Keeps representative schema examples and replaces redundant array items with metadata (: N).**Log Pruning:** Strips internal framework paths and returns only the root error message, user file paths, and execution lines.**HTML Cleaning:** Strips scripts, styles, and navigation elements, converting content to readable text/markdown.**Duplicate Loop Detection:** Hashes tool outputs per session and prepends a warning header if an agent receives duplicate tool results twice in a row.\n\nmake build sudo mv tokencompress /usr/local/bin/\n\ncat large_response.json | tokencompress --mode json\n\ncat app.log | tokencompress --mode log --log-internal-marker \"mycompany/internal\"\n\ncurl -s [https://example.com](https://example.com) | tokencompress --mode html\n\nAdd directly to your Claude Desktop config ():\n\n{ \"mcpServers\": { \"tokencompress\": { \"command\": \"/usr/local/bin/tokencompress\", \"args\": [\"--mode\", \"mcp\"] } } }\n\n| Input Payload | Raw Token Count | Compressed Token Count | Token Reduction | Execution Time |\n|---|---|---|---|---|\nJSON Array (500 items) |\n~12,500 tokens | ~850 tokens | 93.2% |\n|\nPython Stack Trace |\n~3,200 tokens | ~410 tokens | 87.1% |\n|\nHTML Web Scrape |\n~18,000 tokens | ~2,100 tokens | 88.3% |\n\nNeed a custom high-performance MCP proxy, specialized parsers for enterprise tools, or custom token-optimization setups for your agent infrastructure?\n\n**Custom Setup Gigs:** 50 - 00 per custom integration setup.**Custom Log/AST Parsers:** 50 per custom domain parser.**Contact:** Open an issue on this repository or contact directly via GitHub profile.\n\nMIT License. Free to use in open-source and commercial agent setups.", "url": "https://wpnews.pro/news/tokencompress-a-sub-2ms-go-cli-and-mcp-sidecar-that-prunes-ai-agent-tool-context", "canonical_source": "https://github.com/dburnett11155-rgb/Tokencompress", "published_at": "2026-08-16 05:55:12+00:00", "updated_at": "2026-08-16 06:10:39.337826+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "ai-tools", "natural-language-processing"], "entities": ["Tokencompress", "Claude Desktop", "MIT License"], "alternates": {"html": "https://wpnews.pro/news/tokencompress-a-sub-2ms-go-cli-and-mcp-sidecar-that-prunes-ai-agent-tool-context", "markdown": "https://wpnews.pro/news/tokencompress-a-sub-2ms-go-cli-and-mcp-sidecar-that-prunes-ai-agent-tool-context.md", "text": "https://wpnews.pro/news/tokencompress-a-sub-2ms-go-cli-and-mcp-sidecar-that-prunes-ai-agent-tool-context.txt", "jsonld": "https://wpnews.pro/news/tokencompress-a-sub-2ms-go-cli-and-mcp-sidecar-that-prunes-ai-agent-tool-context.jsonld"}}