cd /news/ai-agents/i-indexed-a-96k-file-linux-kernel-tr… · home › topics › ai-agents › article
[ARTICLE · art-156934] src=dev.to ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

I indexed a 96k-file Linux kernel tree in 2m51s on a laptop — benchmarking two code-graph MCP servers

A developer released SMP (Structural Memory Protocol), an MIT-licensed MCP server that maintains a live structural code graph using tree-sitter parsing stored in a single memory-mapped file with a journaled write-ahead log, avoiding Postgres, Neo4j, and embeddings. On an 8-thread Intel i7-1165G7 laptop, SMP fully indexed a 95,937-file Linux kernel snapshot in 2 minutes 51 seconds versus 13 minutes 14 seconds for codebase-memory-mcp, a 4.6x speedup on identical hardware and input. A store revision counter and engine-side corpus cache cut the locate tool from 1,493 ms to 40-85 ms p50 on a 4,575-node graph, with byte-identical results verified by nine equivalence and invalidation tests.

by read2 min views19 publishedOct 24, 2026

Disclosure up front: I'm the author of SMP (Structural Memory Protocol) — an open-source (MIT) MCP server that keeps a live structural code graph. This post is the full methodology and numbers; everything below is reproducible.

Coding agents can read files. What they mostly can't do is answer structural questions: who calls this, what breaks if I change it, which tests cover it. Grep gives you matches, not a map. Embedding-based RAG finds "similar" code, rarely "connected" code. So SMP maintains a graph: tree-sitter parses each file into nodes (functions, classes, modules) and edges (calls, contains, imports), stored in a single memory-mapped file with a journaled write-ahead log — no Postgres, no Neo4j, no embeddings.

Every number below was measured on the same laptop: an 8-thread 11th-gen Intel i7-1165G7, 14 GB RAM, NVMe, Linux, no GPU. Both tools did a fresh full index of the identical directory tree. No vendor numbers are mixed in.

Corpus SMP codebase-memory-mcp (fast mode)
Linux kernel snapshot — 95,937 files (72,171 parsed) 2 min 51 s — 839,516 nodes, 4,066,943 edges, 2.2 GB graph, 6.5 GB peak RAM 13 min 14 s — 4.8 GB index

4.6× faster on the same hardware, same tree. One caveat: codebase-memory-mcp excluded 36 directories by design (gitignored/vendor paths), so node counts describe different graph models — wall-clock on the same input is the comparison.

Corpus SMP codebase-memory-mcp
208-file Python repo 0.99 s — 4,575 nodes / 18,862 edges 5.27 s — 4,346 nodes / 22,002 edges

Micro-benchmark (benchmarks/benchmark_scale.py --nodes 60000 --depth 3), synthetic graph, no MCP overhead:

This is my favorite part. locate used to re-load every node and re-tokenize the entire BM25 corpus on every query — 1,493 ms p50 on a 4,575-node graph. On the 839k-node kernel graph it OOM-killed my desktop twice (systemd-oomd took out the whole session).

The fix: a monotonic store revision counter plus an engine-side corpus cache (≤250k nodes; larger graphs fall back to the original path). Results are byte-identical to the pre-fix code, covered by 9 new equivalence/invalidation tests — and locate dropped to 40–85 ms p50 (~15–37× faster).

Other MCP tools on a warm 4,575-node graph: smp_context p50 29.9 ms, smp_file_outline p50 9.3 ms.

One line wires the MCP server + agent skill into Claude Code, OpenCode, Codex or Cursor:

curl -fsSL https://raw.githubusercontent.com/offx-zinth/SMP/main/install.sh | bash

Repo (MIT, benchmark scripts included so you can re-run every table above on your own hardware): https://github.com/offx-zinth/SMP

Incremental re-index on partial edits at >500k nodes; monorepo-scale retrieval without vectors; Apple Silicon timing runs. If you run agents on a large repo, I genuinely want to know where it breaks for you.

── more in #ai-agents 4 stories · sorted by recency
── more on @smp (structural memory protocol) 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/i-indexed-a-96k-file…] indexed:0 read:2min 2026-10-24 · —