{"slug": "unumsearch-a-trigram-index-so-coding-agents-stop-re-reading-258000-files", "title": "unumsearch: a trigram index so coding agents stop re-reading 258,000 files", "summary": "CorpUnum Labs released unumsearch, an open-source (Apache-2.0) Rust search CLI that maintains a trigram index so coding agents can answer code-search queries without re-reading every file. Over a 247,535-file tree, the unumsearch daemon answered queries in a median 64 ms versus 347 ms for ripgrep 15.1, and across roughly 180 worktrees (258k files) it completed 12 agent-issued patterns in 217 ms versus 4.03 s for ripgrep, an 18.6x speedup with identical results on 12 of 12 patterns. The project notes the gain depends on running it as a persistent daemon, MCP server or rpc process; the one-shot CLI took 12 ms on a 3.7k-file repo, slower than ripgrep's 9.8 ms.", "body_md": "An AI coding agent asks the same sort of question hundreds of times a day: where is this function called, which files mention that setting. A plain `rg` answers each one by reading every file again. That is fine on one repository. It gets slow when you run many checkouts on one machine, which is our situation.\n\nSo we built unumsearch, a search CLI for code and files that keeps an index and answers like ripgrep, usually in milliseconds. It is open source (Apache-2.0), written in Rust, and has no GPU, model or network dependency. This post is what it does, what we measured, and where it loses.\n\nThe index is a list of every three-letter piece of text and the files it appears in. A query is cut into those pieces, the index names the few files that contain all of them, and only those files are opened and checked line by line against their real contents. The index only narrows what to read; the answer always comes from the file.\n\nTwo properties matter for agents:\n\n`fresh`, `backend`, `covered`, `truncated` and `complete` fields so a client knows when to scan for itself.`.ssh/`, `.aws/`, `.env*`, `*.key`, `*.pem` and similar are never indexed, whatever the configuration says.\nThe test set is 20 regexes mined from a week of real coding-agent tool calls, on the author's machine, dated 2026-10-09.\n\nOver the whole tree (247,535 files, warm page cache, median of three passes), ripgrep 15.1 took 347 ms per query. The unumsearch daemon took 64 ms and the one-shot CLI took 91 ms.\n\nIn a separate race of 12 patterns our agents actually issued, over about 180 worktrees (258k files), ripgrep needed 4.03 s in total and the daemon needed 217 ms, 18.6x faster, with the same files as rg on 12 of 12.\n\nOn answers: 19 of 20 queries returned exactly rg's files; the 20th differs only by `.env.example` files, which unumsearch never indexes on purpose. During `git checkout` of a branch that rewrites 44,686 Linux kernel files, the index caught up 6.4 to 7.5 s after the checkout began. Until then answers said `fresh: false`, and 0 of 64,454 fresh answers differed from ripgrep.\n\nOn a single repository of 3.7k files, ripgrep takes a median 9.8 ms. The daemon takes 2.3 ms, but the one-shot CLI, which starts a new process per query, takes 12 ms. That is slower than rg. The speed comes from a process that keeps the index open, so if you run it as a daemon, MCP server or `rpc`, you get the gain; if you shell out once per query on a small repo, you do not.\n\nThe benchmark file lists further weak spots, including that a busier machine made every engine about 1.7x slower in one run, so read the absolute times with that in mind.\n\n```\ncurl -fsSL https://github.com/corpunum/unumsearch/releases/latest/download/install.sh | sh\nunumsearch --root ~/src/myrepo index\nunumsearch search 'fn\\s+main' ~/src/myrepo --text\n```\n\nFor agents, `unumsearch mcp` runs it as a stdio MCP server, and the README lists setup for Claude Code, Codex CLI, Gemini CLI and others. The latest release at the time of writing is v0.1.12, which fixed a case where a daemon reported `fresh` before it had looked at its roots.\n\nYou can reproduce the race on your own code with `python3 bench/race.py --root .`; your numbers will differ.\n\nSource and benchmark method: [github.com/corpunum/unumsearch](https://github.com/corpunum/unumsearch).\n\nThis article was drafted by an AI model (Claude, from Anthropic) working for CorpUnum Labs, from the project's README, benchmark notes and published project facts. Its figures were checked against those files before posting. The numbers come from those files; none were measured by the model.\n\nWe are CorpUnum Labs, in Athens, building OpenUnum, a local-first runtime for user-owned agents. If you want to see what our agents do on that one mini PC, there is a playable lab here: [corpunum.com/play](https://corpunum.com/play?utm_source=devto&utm_medium=article&utm_campaign=unumsearch).", "url": "https://wpnews.pro/news/unumsearch-a-trigram-index-so-coding-agents-stop-re-reading-258000-files", "canonical_source": "https://dev.to/corpunum/unumsearch-a-trigram-index-so-coding-agents-stop-re-reading-258000-files-4mik", "published_at": "2026-11-15 22:39:51+00:00", "updated_at": "2026-11-15 22:50:56.650414+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-tools", "agent-protocols"], "entities": ["CorpUnum Labs", "unumsearch", "ripgrep", "Rust", "Claude Code", "Codex CLI", "Gemini CLI", "OpenUnum"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/unumsearch-a-trigram-index-so-coding-agents-stop-re-reading-258000-files", "markdown": "https://wpnews.pro/news/unumsearch-a-trigram-index-so-coding-agents-stop-re-reading-258000-files.md", "text": "https://wpnews.pro/news/unumsearch-a-trigram-index-so-coding-agents-stop-re-reading-258000-files.txt", "jsonld": "https://wpnews.pro/news/unumsearch-a-trigram-index-so-coding-agents-stop-re-reading-258000-files.jsonld"}}