Stop Wasting LLM Tokens! I Built a Rust CLI to Prune JS/TS Codebases by 80% 🦀🚀

A developer built urai-ecma, a multi-threaded Rust CLI that uses SWC to parse JavaScript and TypeScript into ASTs and semantically prune codebases before feeding them to LLMs. The tool reportedly compresses a 209,757-token codebase to roughly 36,000 tokens, an 82.7% reduction, addressing attention degradation, KV-cache prefill lag, and rate-limit throttling in agentic coding workflows.

┌─────────────────────────────────────────────────────────────────────────────┐ │ THE "INFINITE CONTEXT" TRAP │ ├──────────────────────────────┬──────────────────────────────────────────────┤ │ 1. Attention Degradation │ Lost-in-the-Middle: critical interfaces get │ │ │ buried under repetitive DOM noise and loops. │ ├──────────────────────────────┼──────────────────────────────────────────────┤ │ 2. KV-Cache Prefill Lag │ Time-to-First-Token TTFT scales with prompt│ │ │ size; 150k+ raw tokens stall your agent. │ ├──────────────────────────────┼──────────────────────────────────────────────┤ │ 3. The "Tailwind Tax" │ Paying frontier API rates to ingest 80-char │ │ │ strings like "flex items-center justify-..." │ ├──────────────────────────────┼──────────────────────────────────────────────┤ │ 4. Rate-Limit Throttling │ Bloated prompts quickly exhaust TPM Tokens │ │ │ Per Minute quotas in CI/CD pipelines. │ └──────────────────────────────┴──────────────────────────────────────────────┘ Have you ever dumped an entire React or Next.js repository into Claude 3.5 Sonnet, GPT-4o, or a local Ollama model to ask: "How does authentication state flow through my UI, and what endpoints handle it?" If you inspect the prompt you sent, over 70% of the tokens are dead weight : className="flex flex-col items-center justify-between p-8 bg-white dark:bg-zinc-950 rounded-2xl shadow-xl..." . While building an agentic Chrome extension powered by local LLMs, my context window collapsed: 209,757 tokens per scan . Responses took forever, local inference crawled, and the model routinely hallucinated core functions because key architectural interfaces were buried under syntactic noise. I built urai-ecma https://sanjaiyan-dev.github.io/urai-ecma : a multi-threaded CLI tool written in Rust that uses SWC Speedy Web Compiler to parse JavaScript and TypeScript into Abstract Syntax Trees AST . Instead of blindly concatenating files together like a text scraper, it acts as a semantic compiler for prompt engineering—compressing that same 209k token codebase down to 36k tokens an 82.7% reduction in milliseconds . Here is how it works, how it is architected under the hood, real benchmarks, and the engineering trade-offs you should know before using it. In classical Tamil literary heritage, monumental masterworks like the Thirukkuṛaḷ திருக்குறள் and Tolkāppiyam தொல்காப்பியம் contained dense, multi-layered philosophical thought. To make these works practical without destroying their architectural depth, classical scholars practiced உரை எழுதுதல் Urai Ezhuthudhal . Master commentators Uraiyāsiriyars like Parimelazhagar and Ilampuranar did not just copy or mechanically summarize texts. They performed structural distillation : Modern enterprise JavaScript and TypeScript codebases are the epic literatures of software engineering. When asking an LLM to reason about your code, it doesn't need raw syntactic exhaustion—it needs the structural anatomy, API contracts, state flows, and component signatures. urai-ecma acts as a modern Uraiyāsiriyar for your codebase. Tools like repomix , gitingest , and code2prompt are file dumpers. They walk your directory, wrap raw text in XML/Markdown fences, and pass every single line of styling directly into your model's context. urai-ecma is an AST-aware compiler engine . Rather than treating code as raw strings, it parses your source into concrete syntax trees using ByteDance/Vercel’s swc ecma engine and applies deterministic, semantic transformations: ┌─────────────────────────────────────────────────────────────────────────┐ │ URAI COMPILER PIPELINE │ └─────────────────────────────────────────────────────────────────────────┘ Enterprise Monorepo .ts, .tsx, .js, .mjs, .json │ ▼ ignore::WalkBuilder Rust Honor .gitignore, prune node modules & dist │ ▼ Rayon Parallel Work-Stealing Multi-threaded AST parsing across all CPU cores │ ┌───────────────┴───────────────┐ ▼ ▼ swc ecma parser swc ecma parser Worker Thread A Worker Thread B │ │ ├─► RouteVisitor ├─► RouteVisitor │ Next.js/Express/NestJS │ Next.js/Express/NestJS │ │ ├─► ReactComponentAnalyzer ├─► ReactComponentAnalyzer │ Props, State, Hooks, JSX │ Props, State, Hooks, JSX │ │ ├─► ReactJsxPruner ├─► ReactJsxPruner │ Tailwind static class strip │ Tailwind static class strip │ │ └─► FunctionSummarizerVisitor └─► FunctionSummarizerVisitor Preserve structural stubs Preserve structural stubs │ ▼ Foyer Hybrid Cache Disk + RAM Sha512 256 + Zstd compression │ ▼ swc ecma codegen + Tiktoken Engine Emits high-density Markdown prompt + BPE o200k report is structural stub stmt Traditional minification forces a bad compromise: either include full function bodies wasting thousands of tokens on loops and math or strip functions down to empty signatures which deletes hooks, event listeners, and JSX layouts . urai-ecma solves this through Structural Stubbing . It inspects AST statements and retains only nodes critical to architectural comprehension: // Only statements defining component anatomy are preserved: fn is structural stub stmt stmt: &Stmt - bool { match stmt { Stmt::Decl Decl::Fn = true, // Nested helper declarations Stmt::Decl Decl::Var var decl = var decl.decls.iter .any |decl| { if let Some init = &decl.init { matches init, Expr::Arrow | Expr::Fn } else { false } } , Stmt::Expr expr stmt = { if let Expr::Call call expr = & expr stmt.expr && let Callee::Expr callee expr = &call expr.callee && let Expr::Ident ident = & callee expr { let name = ident.sym.as ref ; // Preserves React Hooks, lifecycle timers, and global listeners: return name.starts with "use" || name == "setTimeout" || name == "setInterval" || name.contains "addEventListener" || name.contains "requestIdleCallback" ; } false } Stmt::Return ret stmt = { // Preserves JSX layout hierarchies: if let Some arg = &ret stmt.arg { matches & arg, Expr::JSXElement | Expr::JSXFragment | Expr::Paren } else { false } } = false, // Computational loops, arithmetic, & validations are pruned } } useEffect = { ... }, dep remains intact, signaling side-effects to the LLM. Modern utility CSS accounts for massive token bloat. urai-ecma provides 4 modes remove , remove aggr , summarize , preserve : className={clsx "btn", isActive && "btn-active" } or ternary conditions, / UI: Frosted glass card with dark mode / . Summarizing every single function with an LLM is slow. urai-ecma uses a two-tier resolution strategy : @description , @param , @return already exist. It even includes a proximity-scan fallback within a 300-byte span to associate detached comments. This takes gemma4 , llama3.2 . foyer crate 64MB direct RAM buffer + 128MB Zstd-compressed disk storage with Sha512 256 keys . Look at what happens to a bloated React component when passed through urai-ecma : js const ErrorUI = { headerDescTxt = "The real-time telemetry pipeline requires runtime binding. Ensure this window resides in a Chrome extension popup configured with permission parameters.", copyTextCommand = 'OLLAMA ORIGINS=" " ollama serve', copyTagTxt = "MV3", copyHeaderTxt = "Manifest Interface Schema", copiedButtonTxt = "Copied Configuration", copyButtonTxt = "Copy Permission Manifest", } = { const copyState, setCopyState = useState false ; const handleCopyManifest = = { navigator.clipboard.writeText copyTextCommand ; setCopyState true ; setTimeout = setCopyState false , 2000 ; }; const copyButtonTxtNode = copyState ? copiedButtonTxt : copyButtonTxt; return <div className="relative min-h-screen w-full bg- 05050A border border-white/10 overflow-hidden p-6 text- F8FAFC flex flex-col justify-between" <div className="absolute top- -10% left- -10% w-45 h-45 rounded-full bg- FF2E63 opacity-20 blur- 64px pointer-events-none" / <div className="space-y-6" <div className="flex items-center space-x-3" <div className="w-2.5 h-2.5 rounded-full bg- FF2E63 animate-pulse shadow- 0 0 8px FF2E63 " / <span className="text- 10px font-mono tracking-widest text- FF2E63 uppercase font-bold" Diagnostics Status: Telemetry Offline </span </div <div className="bg-white/ 0.03 border border-white/ 0.08 rounded-2xl p-4 space-y-3" <button className={ text- 9px font-mono px-2 py-1 rounded-md transition-all bg- 8B5CF6 text-white } {copyTagTxt} </button <motion.button onClick={handleCopyManifest} className="w-full py-2 bg-white/ 0.06 hover:bg-white/ 0.1 border border-white/10 text-xs font-mono font-medium rounded-xl flex items-center justify-center space-x-2 text-white" <span {copyButtonTxtNode}</span </motion.button </div </div </div ; }; urai-ecma After React Component Breakdown: <ErrorUI - Props : - headerDescTxt type: any optional - copyTextCommand type: any optional - copyTagTxt type: any optional - copyHeaderTxt type: any optional - copiedButtonTxt type: any optional - copyButtonTxt type: any optional - State Management : - Manages state copyState via setter setCopyState . - Hooks : Uses useState Total Side-Effects: 0 . - Rendered JSX Tree : <div , <span , <h1 , <p , <button , <motion.button js const ErrorUI = { headerDescTxt = "...", copyTextCommand = "...", copyTagTxt = "MV3", copyHeaderTxt = "...", copiedButtonTxt = "...", copyButtonTxt = "..." } = { const handleCopyManifest = = { setTimeout = setCopyState false , 2000 ; '/ "Copies the manifesto text to the clipboard and sets a temporary success state for two seconds." /'; }; return <div className="/ UI: Full-screen dark mode layout with border and text overflow management /" <div className="/ UI: Absolute background blur element positioned outside the main container /"/ <div <span Diagnostics Status: Telemetry Offline</span <div <button className={ text- 9px font-mono px-2 py-1 rounded-md transition-all bg- 8B5CF6 text-white } {copyTagTxt} </button <motion.button onClick={handleCopyManifest} <span {copyButtonTxtNode}</span </motion.button </div </div </div ; '/ "Presents a user interface displaying diagnostic status and provides a copy function for a necessary runtime binding command." /'; }; Notice what happened: text- 9px ... ${...} were left untouched. We benchmarked urai-ecma on an Apple Silicon machine across different workloads using OpenAI’s native o200k base BPE tokenizer: | Metric | Raw Project TS/TSX | urai-ecma Output | Total Reduction | |---|---|---|---| | Token Volume o200k base | 209,757 tokens | 36,153 tokens | -82.76% 🚀 | | Downstream LLM Context | Exceeds local 64k limits | Fits easily in local Ollama | Usable on 8GB VRAM | | KV-Cache TTFT Time-to-First-Token | ~18.4 seconds | ~1.9 seconds | ~9.6x Faster | plugin-api-docgen — Cold vs. Warm Performance Here are the terminal runs comparing an initial cold run querying local Ollama against subsequent warm runs hitting the foyer hybrid cache : bash RUN 1: Cold Execution AST Parsing + Local Ollama Inference $ time urai-ecma 🚀 urai-ecma Starting AST Analysis on project: ./src 🔍 Found 6 source file s for analysis. ✅ urai-ecma Prompt successfully generated at: ./output.md 📊 urai-ecma Estimated Tokens in ./output.md: 1317 tokens ============================================================ 📊 TOKEN SAVINGS & OPTIMIZATION REPORT ============================================================ 📁 Raw Source Code All JS/TS : 3023 tokens ⚡ Optimized Output output.md : 1317 tokens ------------------------------------------------------------ 🎉 Reduction: -56.43% tokens saved Saved ~1706 tokens ============================================================ urai-ecma 0.06s user 0.04s system 0% cpu 21.541 total bash RUN 2: Warm Execution AST Parsing + Foyer Hybrid Cache Hits $ time urai-ecma 🚀 urai-ecma Starting AST Analysis on project: ./src 🔍 Found 6 source file s for analysis. ✅ urai-ecma Prompt successfully generated at: ./output.md 📊 urai-ecma Estimated Tokens in ./output.md: 1299 tokens ============================================================ 📊 TOKEN SAVINGS & OPTIMIZATION REPORT ============================================================ 📁 Raw Source Code All JS/TS : 3023 tokens ⚡ Optimized Output output.md : 1299 tokens ------------------------------------------------------------ 🎉 Reduction: -57.03% tokens saved Saved ~1724 tokens ============================================================ urai-ecma 0.03s user 0.01s system 92% cpu 0.049 total Execution Latency Comparison plugin-api-docgen ────────────────────────────────────────────────────────────────── Cold Run Ollama Local Inference : ████████████████████ 21.541s Warm Run Foyer Zstd Cache : ▍ 0.049s 49ms ────────────────────────────────────────────────────────────────── Speedup Factor: ~439x Faster on Cache Hit | Feature | repomix / gitingest | code2prompt | urai-ecma | |---|---|---|---| | Engine | Node.js / Python | Rust | Rust SWC + Rayon | | Parsing Strategy | Naive String Concatenation | Handlebars Templates | True AST Traversal | | Tailwind Handling | Preserves all noise 0% saved | Preserves all noise 0% saved | 4-Mode AST Pruning | | Structural Stubbing | ❌ No | ❌ No | ✅ Yes is structural stub | | API Route Tables | ❌ No | ❌ No | ✅ Auto-extracted Next/Nest/Express | | Component Analysis | ❌ No | ❌ No | ✅ Props, State, Hooks, JSX tree | | Privacy / Offline | Dependent on API | Offline string copy | 100% Offline Local Ollama/JSDoc | | Token Savings | 0% Expands token size | 0% | Up to 82.7% reduction | No tool is a silver bullet. Because urai-ecma prunes function internals into architectural stubs, you need to understand when to use it and when to skip it: urai-ecma is distributed as a single static binary with zero runtime dependencies. curl --proto '=https' --tlsv1.2 -LsSf https://github.com/sanjaiyan-dev/urai-ecma/releases/download/v0.1.1/urai-ecma-installer.sh | sh powershell -ExecutionPolicy Bypass -c "irm https://github.com/sanjaiyan-dev/urai-ecma/releases/download/v0.1.1/urai-ecma-installer.ps1 | iex" npm install -g urai-ecma or: pnpm add -g urai-ecma | bun add -g urai-ecma | deno add -g npm:urai-ecma cargo install urai-ecma Initialize a documented configuration file in your project root: urai-ecma create This creates urai.config.jsonc . Because urai-ecma is registered globally with SchemaStore , you get instant auto-complete and documentation in VS Code, WebStorm, IntelliJ, and Visual Studio : { "$schema": "https://www.schemastore.org/urai-ecma.json", // Source directory or file to analyze "input project": "./src", // Target markdown prompt output "output file": "./output.md", // Local Ollama instance Optional "ollama endpoint": "http://localhost:11434", "ollama modelname": "gemma4", // Tailwind CSS mode: "remove" | "remove aggr" | "summarize" | "preserve" "tailwind mode": "remove", "tailwind threshold": 96, // Summarize function bodies via JSDoc or Ollama "summarize functions": true, "summarize functions threshold": 5, // Extract Express / Fastify / Next.js / NestJS routes "generate route table": true, // React component introspection Props, State, Hooks "analyze react components": true, // Generate ASCII tree & Mermaid ESM dependency graph "generate file graph": true } Run analysis anywhere: Run with configuration file urai-ecma Or run ad-hoc via CLI flags urai-ecma -i ./src -o prompt.md --tailwind-mode remove llms.txt Transform bloated JavaScript & TypeScript repositories into hyper-dense, token-optimized context prompts. In classical Tamil literary heritage, monumental epics and ancient treatises—such as the Thirukkuṛaḷ , Tolkāppiyam , and Cilappatikāram —span vast volumes of dense, poetic, and complex thought. To make these monumental texts intelligible without losing their depth, classical scholars Uraiyāsiriyars practiced உரை எழுதுதல் Urai Ezhuthudhal : the disciplined art of writing a lucid, structured, and insightful commentary that distills the core essence, syntax, and architectural meaning of vast literature. Today, enterprise JavaScript and TypeScript codebases are the epic literatures of modern software . Spanning thousands of files across Next.js, React, Node.js, and TypeScript, they are laden with boilerplate, repetitive utility classes, and nested syntax. When feeding these systems to Large Language Models: If you're sick of burning through API credits and watching your AI coding agents drown in static CSS strings, give urai-ecma a spin on your project. Drop your before-and-after token savings in the comments below