Poler-harvest: A Content-Addressed Search Tool in Rust with 2ms Warm Cache for AI Agents A developer built poler-harvest, a Rust-based content-addressed search tool for AI coding agents that adds persistent L1–L3 caching on top of multi-threaded search using rayon, aho-corasick, and regex. Benchmarked with hyperfine on Linux x86_64 with NVMe, warm repeated searches dropped to 2 ms versus 633–760 ms cold, a claimed 364× speedup over GNU grep and ripgrep, which the developer notes have no memory between runs. Cache invalidation uses size, mtime, and content hash with selective re-scanning of changed files, validated by 7/7 regression tests, and the tool is released under a custom source-available license. I love ripgrep and grep — they are legendary for one-off manual terminal lookups. But while designing local AI coding agents and automated developer loops, I hit a fundamental bottleneck: traditional search tools have zero memory . When an autonomous coding agent or CI test runner performs dozens of searches across the exact same 100,000-file codebase within a few minutes, re-traversing the directory tree and re-scanning gigabytes of files from disk on every single turn wastes seconds of user latency. To solve this, I built poler-harvest https://github.com/poler-engine-org/poler-harvest in Rust. poler-harvest combines high-throughput multi-threaded search with content-addressed L1–L3 persistent caching: Cold Run Zero Cache : rayon , aho-corasick , and regex . .t5a / .tar.poler . Warm Run Instant Lookups : Honest Invalidation 7/7 Regression Tests : size , mtime , and content hash, selectively re-scanning only modified, added, or deleted files in sub-milliseconds. | Tool | Cold Search | Warm Search Repeated | Speedup | |---|---|---|---| | GNU grep 3.8 grep -r | 407–451 ms | 407–451 ms | 1× | | ripgrep 13.0 rg | 728–760 ms | 728–760 ms | 1× | | poler-harvest | 633–760 ms | 2 ms | 364× faster 🚀 | Measurements taken with hyperfine on Linux 6.x, x86 64, NVMe . curl -sSL https://raw.githubusercontent.com/poler-engine-org/poler-harvest/main/scripts/install.sh | bash sudo pacman -U https://github.com/poler-engine-org/poler-harvest/releases/download/v0.96.1/poler-harvest-0.96.1-1-x86 64.pkg.tar.zst curl -LO https://github.com/poler-engine-org/poler-harvest/releases/download/v0.96.1/poler-harvest 0.96.1 amd64.deb sudo dpkg -i poler-harvest 0.96.1 amd64.deb pip install https://github.com/poler-engine-org/poler-harvest/releases/download/v0.96.1/poler harvest-0.96.1-py3-none-manylinux 2 34 x86 64.whl Exact search with 2ms warm lookups: poler-harvest . "TODO: fix this bug" address Contextual markdown search for documentation: poler-harvest src/ "network timeout" markdown -c 2 Regex search with machine-readable JSONL stream: poler-harvest src/ "struct. Error" -r -f json One-off scan without touching persistent cache: poler-harvest /var/log "kernel error" --no-cache The project is published under the POLER Custom Source-Available License v1.0 — completely free for personal use, security audits, academic research, and community developer tooling: ⭐ GitHub Repository: https://github.com/poler-engine-org/poler-harvest https://github.com/poler-engine-org/poler-harvest I would love to hear feedback, architectural suggestions, or questions about the FastCDC chunking pipeline from the community