An LSP layer for 70%+ token reduction in multi-turn agents

A compiled Rust CLI binary called unlimited-bench, built on a Tree-sitter AST engine, cut input token spend by 75.2% — from 1,450,000 to 280,000 tokens — in a simulated 25-step agentic refactoring loop over a 150-file TypeScript/Python codebase, according to the repository's published benchmark. The same simulation reported P95 latency per step falling from 1,240ms to 115ms (10.7x faster) and estimated cost per 1,000 runs dropping from $21.75 to $4.20, an 80.6% margin expansion. The in-container LSP daemon runs over stdio and JSON-RPC 2.0, routing sub-tasks with a Cognitive Dependency Index below 0.60 to a local SHA-256 symbol-masking engine while sending higher-CDI work to cloud models such as Claude 3.5 Sonnet and GPT-4o.

A compiled CLI binary and architectural specification for reducing AI agent token overhead via structural AST context pruning & network. Autonomous AI coding agents Replit, Claude Code, Devin, Cursor, custom internal agent loops are facing an existential margin bottleneck: Context Accumulation Overhead "Context Rot" . When an agent executes multi-step refactoring loops, standard IDE integration re-submits entire repository trees and verbose boilerplate back to frontier cloud models $15–$30/M tokens just to run basic syntax checks, import resolutions, and linter passes. This repository outlines the specification for an In-Container LSP Daemon : a lightweight Rust binary running over stdio that intercepts JSON-RPC streams, performs zero-knowledge AST pruning, and routes non-cognitive background tasks through an optimized execution engine. Simulated 25-step agentic refactoring loop on a 150-file TypeScript/Python codebase: | Metric | Traditional Cloud Agent Loop | In-Container LSP Daemon Engine | Improvement | |---|---|---|---| | Input Token Spend | 1,450,000 tokens | 280,000 tokens | 75.2% Token Reduction | | P95 Latency / Step | 1,240ms Cloud Roundtrip | 115ms Local AST Resolution | 10.7x Faster Execution | | Est. Cost / 1k Runs | $21.75 | $4.20 | 80.6% Margin Expansion | | Data Privacy | Raw code sent to cloud endpoints | Zero-Knowledge Symbol Masking | 100% SOC2 Compliant | IDE / Agent Loop │ ▼ stdio / JSON-RPC 2.0 ┌─────────────────────────────────────────────────────────────┐ │ In-Container Daemon / LSP Interceptor │ │ - Non-blocking Cognitive Dependency Index CDI Scan │ └────────┬──────────────────────────────────────────┬─────────┘ │ │ CDI = 0.60 CDI < 0.60 │ │ ▼ ▼ Central Cloud LLM TREE-SITTER AST ENGINE High Cognitive Logic SHA-256 Symbol Masking │ ▼ PROPRIETARY EXECUTION GRID Zero-Knowledge Context Offload - Zero Host Installation: Runs natively inside Docker, MicroVMs, or Cloud Devboxes. - SOC2 Compliant Egress: Un-hashed proprietary source code never leaves the workspace perimeter. - Asymmetric Context Optimization: Dynamically filters non-cognitive agent sub-tasks away from expensive cloud models. 1. OVERVIEW This compiled CLI binary unlimited-bench is a standalone execution module engineered to demonstrate the core parsing latency, token reduction metrics, and distributed edge cache behavior of the UnlimitedAI™ AST engine cite: 1, 2 . The proxy operates at the local/edge layer before context payloads are passed to centralized LLM endpoints e.g., Claude 3.5 Sonnet / GPT-4o , dynamically pruning low-density syntax noise while preserving full semantic code graphs cite: 1, 2 . 1. QUICK START TERMINAL EXECUTION Open your terminal, navigate to the directory containing the binary, and execute it against any target code file in your repository: Standard Single-File Benchmark: $ chmod +x unlimited-bench $ ./unlimited-bench --file /path/to/target file.ts Enterprise Edge Cluster Benchmark Simulates Shared State & Latency : $ ./unlimited-bench --file /path/to/target file.ts --distributed Optional Query Flag: $ ./unlimited-bench --file /path/to/target file.ts --query "Refactor handler" 1. UNDERSTANDING BENCHMARK RESULTS & METRICS When running the benchmark against internal codebases, note the following operational behaviors: A. Local Edge Parse Time <50ms SLA : - The Rust/Tree-sitter AST engine parses file structures deterministically cite: 1, 2 . - Sub-10ms execution across standard files, easily maintaining the <50ms SLA on large enterprise codebases up to 50,000 LOC cite: 2 . B. Token Reduction Variance 15% to 88% Range : - Dense Logic Files e.g., single core algorithms, pure functions : Expect ~15% - 30% reduction. Pure functional logic contains high node density and minimal structural noise. - Production / Enterprise Files e.g., React components, API routes, heavily-typed SDKs, files with extensive import headers/comments : Expect ~75% - 87.6% reduction as import bloat, docstrings, and non-state AST nodes are systematically pruned cite: 1, 2 . C. Unit Economics $CDI Metric : - Token estimates are modeled against standard Claude 3.5 Sonnet pricing $3.00 / 1M input tokens cite: 2 . - In multi-turn agent execution loops e.g., 20–50 calls per user task , micro-savings compound directly into overall gross margin expansion cite: 1, 2 . 1. ENTERPRISE DISTRIBUTED CLUSTER ARCHITECTURE UnlimitedAI™ operates as an in-VPC edge acceleration layer across multi-tenant agent fleets cite: 1, 2 : - Distributed AST State Caching: Nodes synchronize AST signatures across local agent clusters cite: 1, 2 . When Agent A parses a repository file, downstream agent workers retrieve pre-pruned context states instantly <3.5ms without duplicate AST parsing cite: 1, 2 . - Reduced Network Egress: Eliminates centralized choke points and gateway load by pruning context payloads directly at the container/pod boundary before API dispatch cite: 1, 2 . - Isolated Edge Execution: Maintains localized cache state and AST reduction within your existing cloud VPC boundary, guaranteeing zero external data egress cite: 1, 2 . 1. PRODUCTION CONTAINER INTEGRATION This CLI utility evaluates AST parsing and edge state mechanics cite: 1, 2 . The full production license UnlimitedAI™ Enterprise Container runs as an in-container LSP proxy sitting directly inside devbox container runtimes e.g., Replit Devboxes over stdio JSON-RPC streams, offering zero-installation integration across web IDEs cite: 1, 2 . About the Author Head of Product & AI Product Strategy Specializing in AI Product Architecture, Multi-Modal Systems, Venture Strategy, and 0-to-1 Commercialization. I architect high-impact AI product frameworks, system designs, and go-to-market strategies for emerging tech companies and venture studios. Currently open to select Head of Product / VP of Product / Chief Product Officer roles offering cash compensation + significant equity. - Email: mgmtparker@gmail.com mailto:mgmtparker@gmail.com Commercial Implementation & Hiring Enquiries: The underlying prompt mechanics, failure taxonomies, and execution algorithms are proprietary. To discuss commercial licensing, advisory roles, or hiring opportunities, please contact the author directly at mgmtparker@gmail.com mailto:mgmtparker@gmail.com