{"slug": "how-we-cut-repo-wide-symbol-indexing-for-llm-agents-from-30s-to-98ms", "title": "How we cut repo-wide symbol indexing for LLM agents from 30s to 98ms", "summary": "LiuHe, a code-operation toolchain designed for LLM agents, has cut repo-wide symbol indexing from tens of seconds to 98ms. The project, built by developer wulun811, uses a Rust tree-sitter parse daemon, SQLite symbol index, and transactional journal-backed writes to address the unique constraints of agents that lack hands, eyes, and memory. The toolkit includes features like atomic edits, undo journals, and executable error recovery actions to prevent silent data loss.", "body_md": "If your coding agent has ever stalled for tens of seconds on \"what's in this repo?\" — or burned hundreds of tokens re-reading a file after a failed edit — this is the story of why that happens and how we fixed it.\n\n**TL;DR** — we rebuilt code tooling for agents that have no hands, no eyes, and no memory:\n\n`repo_map`\n\nin `kill -9`\n\n— no more silently lost work**LiuHe** ([https://github.com/wulun811/LiuHe](https://github.com/wulun811/LiuHe)) is a code-operation toolchain designed for LLMs rather than humans: Node orchestration, a Rust tree-sitter parse daemon, a SQLite symbol index, and transactional journal-backed writes. MIT, v0.4.6, zero-build deploy (no cargo, no `npm install`\n\n).\n\nThis post is the architecture story: what was slow, what we changed, and the numbers we measured while doing it.\n\nSix families, all deterministic, all reproducible from `benchmarks/`\n\n:\n\n`read_symbol`\n\n(version-anchored), `symbol_search`\n\n, `code_search`\n\n, `repo_map`\n\n(98ms, paginated skeleton), `reindex`\n\n, `dep_graph`\n\n, `references`\n\n`impact_analysis`\n\n, `call_chain`\n\n, `trace_symbol`\n\n(constant tracing), `inspect`\n\n, `sweep_dead_code`\n\n, `config_drift`\n\n`edit_batch`\n\n(4-level tolerant matching), `edit_transaction`\n\n(atomic + undo journal), `rename_symbol`\n\n, `git_worktree`\n\n, `edit_sandbox`\n\n, `edit_collision_guard`\n\n, `diff_facts`\n\n`code_review`\n\n, `security_review`\n\n, `code_quality`\n\n, `style_sniffer`\n\n, `guard_patterns`\n\n, `naming_consistency`\n\n, `dependency_gatekeeper`\n\n, `fix_imports`\n\n, `mock_sync`\n\n— zero LLM calls`test_bridge`\n\n, `find_tests`\n\n, `verify_pipeline`\n\n, `debug_runner`\n\n, `tsc_check`\n\n, `patch_parser`\n\n, `spec_gen`\n\n`health`\n\n(self-healing), `gc`\n\n, `feedback`\n\nIf you've watched an agent burn thousands of tokens re-reading a file because a `sed`\n\ndidn't match, or lose work to a silently botched write — these are the failure modes this toolkit exists for. Everything below is about making those tools fast, safe, and cheap.\n\nOur first version answered \"what symbols exist in this repo?\" by walking the tree and parsing every file on every request. On a 347-file project that was **tens of seconds**. On a real Ansible repo of 1,482 files it was worse — and agents ask for repo maps *constantly* (every tool-call needs file → symbol → reference context).\n\nThe fix came in three layers:\n\n**Result: full index of 1,482 files in 9.7s (153 files/s); repo map afterwards: 98ms.**\n\nThe original tree-sitter binding inside Node had two failure modes: a parse exception killed the entire MCP process (on average every 2–4 hours of use), and GC pauses + per-node JS↔C crossings made batch indexing stall.\n\nThe Rust daemon fixes both:\n\n`catch_unwind`\n\n`PARSE_PANIC`\n\nerror code; the MCP server keeps running. Users forgive slow, never dead.`worker_threads`\n\nstartup costs.Human tools assume you have hands, eyes, and memory. An LLM has none. Three compensations:\n\n`edit_transaction`\n\nis all-or-nothing; every write produces an undo journal. We tested `kill -9`\n\nmid-write: the half-written transaction rolls back, source files untouched.`suggestion`\n\n`next_action`\n\n— an executable recovery call the model reissues verbatim instead of guessing.`workspace_dir`\n\n; writes are version-anchored (optimistic concurrency), so even if the model forgets the version it read, the write On the \"silent corruption\" point: while building with a default agent tool stack, one overwrite write silently lost 400+ lines — surfaced ~40 turns later, by luck. We stopped betting on \"models will get better\" and moved the safety into the tool layer.\n\nErrors aren't a dead end — they're an interface. Every failure carries a stable code, a human-readable suggestion, and a `next_action`\n\nthat is executable, not advice:\n\n```\n{ \"error\": { \"code\": \"VERSION_CONFLICT\", \"message\": \"base_version mismatch: FILE_CHANGED\", \"suggestion\": \"Re-read the file and regenerate the batch.\", \"next_action\": { \"tool\": \"read_symbol\", \"params\": { \"locator\": { \"file_path\": \"src/api.js\" } } } } }\n```\n\nThe model doesn't parse the suggestion and decide what to do — it reissues `next_action`\n\nverbatim and recovers. Successful calls carry a `next_step`\n\nthe same way. Errors become signposts with navigation instead of dead ends.\n\n`edit_batch`\n\n: tolerant matching, paranoid writing\nLLMs generate `old_string`\n\nanchors with mistakes humans rarely make: collapsed double spaces, truncated line ends, curly quotes where the code has straight ones. A bare `no_match`\n\nsends the model off to re-read the whole file — thousands of tokens per failed match.\n\nMatching degrades in four stages: exact → trailing-whitespace-stripped → edit-distance candidates (similarity ≥ 0.5) → diagnostics (whitespace visualized, 17 Unicode confusable pairs listed). A typical failure reads: *\"candidate at line 42, similarity 0.87 — you used curly quotes, the code has straight ones.\"* Usually one retry fixes it.\n\nThe write side is paranoid: symlink guards, unique temp file names, TOCTOU check between match and commit, rename retries for Windows AV file locks, post-write syntax check (`node --check`\n\n/ `py_compile`\n\n/ `JSON.parse`\n\n).\n\n`security_review`\n\n, `code_review`\n\n, `sweep_dead_code`\n\nare pure regex/AST — **zero LLM calls**. Same input, same output; CI-safe and auditable.\n\nWe state the boundary explicitly: these tools do not cover control flow, data flow, or cross-module semantics. Zero findings ≠ safe; a high score ≠ healthy. A deterministic pattern scanner that admits its scope beats a \"comprehensive security\" claim every time.\n\n30+ rounds of \"LiuHe reviews LiuHe\" — every bug found becomes a regression test. Real fixes from those rounds: a scope filter scanning outside its target directory, dead-code false positives on registration patterns, constant tracing missing read sites, SQL parameterization cleanup. Assertions grew every round: **2,013 JS + 92 Rust**, full chain green.\n\nLiuHe (六合, \"six harmonies\") names the six design constraints applied to every tool in the toolkit:\n\nThe three compensations earlier (hands / eyes / memory) are the user-facing summary; these six are the per-tool checklist behind them. The AST layer that enforces them is called Malong.\n\nAll under a real `docker --memory=512m`\n\ncgroup:\n\n| Metric | Value |\n|---|---|\n`repo_map` |\n98ms (was tens of seconds) |\n| Full index | 1,482 files in 9.7s |\n| Concurrency | 128 concurrent / 256 in-flight, zero OOM\n|\n| Peak RSS |\n134MB (~26% of limit) |\n| Throughput | ~588 calls/s (60–600× realistic agent demand) |\n| Hot-file storm | 32-way read/write mix, zero torn writes, `integrity_check` PASS, 95/95 conflicts rejected as `FILE_LOCKED`\n|\n| Token savings | ↓65.3% (7,673 → 2,662 est. on same task) |\n\n**Honest boundary:** throughput doesn't scale with concurrency — `better-sqlite3`\n\n's synchronous queries serialize on the Node event loop. We evaluated `worker_threads`\n\nand decided the risk wasn't worth the gain. 588 calls/s is already overkill.\n\n**Where the token savings come from:** tiered tool-description compression (44 tools ≈ 1.33k tokens — core tools keep full descriptions, low-frequency ones shrink to ≤70 chars, verbose ones ≤230, with the detail deferred to `next_step`\n\nhints), incremental returns with explicit pagination instead of dumping everything, and batch endpoints (`read_symbols`\n\n, `write_symbols`\n\n) that cut round-trips. Same task: 7,673 → 2,662 estimated tokens (↓65.3%) and 6 calls → 3 (↓50%).\n\nAll benchmarks are reproducible from `benchmarks/`\n\nand `tests/`\n\nin the repo (concurrency correctness: `tests/test-mvp-concurrency.js`\n\n).\n\nWe also shipped first-day support for DeepSeek Harness (dsh web) — one line to register, all 44 tools exposed as `malong__*`\n\nwith `workspace_dir`\n\nauto-filled from the conversation's workspace:\n\n```\ndsh plugin --profile web add @jieai/dsh-malong-bridge\n```\n\nFull guide in the repo: `malong/dsh/DSH-INTEGRATION.md`\n\n.\n\nZero-build deploy (no cargo, no `npm install`\n\n):\n\n```\ngit clone https://github.com/wulun811/LiuHe liuhe && cd liuhe/malong\nmkdir -p ~/.local/bin\ntar -xzf ../releases/malong-liuhe-0.4.6-linux-x86_64.tar.gz\ncp malong-parse/target/release/malong-parse ~/.local/bin\nmalong-parse &   # start the parse daemon\nnode --max-old-space-size=512 --expose-gc mcp-server.js --workspace /path/to/project\n```\n\nIf `better-sqlite3`\n\nis unavailable, it falls back to vendored `sql.js`\n\nWASM — no install, no compile, no network.\n\nSkepticism welcome — all numbers above are self-measured and reproducible. If this resonates with an agent failure you've had, try it, break it, and tell us where we're wrong — the repo is [wulun811/LiuHe](https://github.com/wulun811/LiuHe).", "url": "https://wpnews.pro/news/how-we-cut-repo-wide-symbol-indexing-for-llm-agents-from-30s-to-98ms", "canonical_source": "https://dev.to/wulun811/how-we-cut-repo-wide-symbol-indexing-for-llm-agents-from-30s-to-98ms-1mn2", "published_at": "2026-08-20 18:51:52+00:00", "updated_at": "2026-08-20 19:14:12.232038+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence", "large-language-models", "ai-agents"], "entities": ["LiuHe", "wulun811", "Rust", "tree-sitter", "SQLite", "MCP"], "alternates": {"html": "https://wpnews.pro/news/how-we-cut-repo-wide-symbol-indexing-for-llm-agents-from-30s-to-98ms", "markdown": "https://wpnews.pro/news/how-we-cut-repo-wide-symbol-indexing-for-llm-agents-from-30s-to-98ms.md", "text": "https://wpnews.pro/news/how-we-cut-repo-wide-symbol-indexing-for-llm-agents-from-30s-to-98ms.txt", "jsonld": "https://wpnews.pro/news/how-we-cut-repo-wide-symbol-indexing-for-llm-agents-from-30s-to-98ms.jsonld"}}