{"slug": "show-hn-visiongrep-local-semantic-image-search-in-rust", "title": "Show HN: Visiongrep – local semantic image search in Rust", "summary": "VisionGrep, a Rust command-line tool by developer udayj, performs local semantic image search by running CLIP inference on-device and caching embeddings in SQLite for repeat queries. The tool searches JPEG, PNG, WebP and BMP files recursively, uses an INT8-quantized DataComp CLIP vision model of about 119 MB (down from 352 MB) plus roughly 257 MB for the FP32 text model and tokenizer, and defaults to five results at a minimum cosine similarity of 0.25. VisionGrep ships prebuilt binaries for Apple Silicon macOS and Linux x86_64/arm64 (glibc 2.39 or newer), requires Rust 1.89 or newer to build, and emits stable text, JSON or NUL-separated output with exit codes 0 for matches, 1 for no matches and 2 for errors so scripts and AI coding agents can call it.", "body_md": "Search local images with a natural language description or a reference image. VisionGrep is a Rust CLI that runs CLIP inference locally and caches embeddings in SQLite for repeated searches. It recursively searches JPEG, PNG, WebP, and BMP files without following symbolic links.\n\nIt is built to be called by scripts and AI coding agents as well as people: results go to stdout\nin stable text, JSON or NUL-separated formats, diagnostics go to stderr, and exit codes separate\n\"no match\" from errors. See [Use with AI agents](#use-with-ai-agents).\n\nPrebuilt binaries are available for Apple Silicon macOS and Linux (x86_64 and arm64, glibc 2.39\nor newer, such as Ubuntu 24.04). Intel Macs are not supported because ONNX Runtime publishes no\nprebuilt CPU binaries for them. Install the latest release into `~/.cargo/bin`:\n\n```\ncurl --proto '=https' --tlsv1.2 -LsSf https://github.com/udayj/visiongrep/releases/latest/download/visiongrep-installer.sh | sh\nvisiongrep \"a dog on a beach\" ./photos\nvisiongrep --image ./reference.jpg ./photos\n```\n\nTo build from a checkout instead, run `cargo build --release` and use\n`./target/release/visiongrep`. Building needs Rust 1.89 or newer and a C compiler; the optimized\nbuild takes several minutes.\n\nRequired model artifacts are downloaded automatically on first use. They are stored in\n`$XDG_CACHE_HOME/visiongrep/models/`, or `~/.cache/visiongrep/models/` when `XDG_CACHE_HOME` is unset\nor empty. Once the required models are cached, searches can run offline. An image query excludes\nthe query image itself from results.\n\nThe DataComp CLIP vision model uses dynamic INT8 quantization, with the final projection and first transformer block kept in FP32. The vision model is about 119 MB (down from 352 MB); the FP32 text model and tokenizer add about 257 MB. Changing the vision model causes existing indexes to re-embed images once on their next use.\n\nOn first use, VisionGrep also saves an optimized ONNX graph under `models/optimized/` for each\nmodel it loads. Later processes reuse compatible graphs to reduce model startup work. The vision\nand text graphs are cached separately; the tokenizer is unchanged. The first use still pays the\noptimization and save cost. On Apple Silicon, the optimized vision and text graphs measured about\n119 MB and 254 MB respectively, adding about 373 MB of disk usage; sizes can vary by platform.\nAlready-running services reuse their loaded sessions.\n\nOptimized graphs are tied to the source model, ONNX Runtime build and local hardware configuration.\nIf an optimized graph cannot be used or saved, VisionGrep falls back to the original model.\nRemoving `models/optimized/` causes the graphs to be rebuilt on subsequent use.\n`--no-cache` bypasses the SQLite index, while model artifacts and optimized graphs remain cached.\n\n```\nvisiongrep \"sunset\" ./photos --top 10 --threshold 0.3\nvisiongrep \"sunset\" ./photos --json\nvisiongrep \"sunset\" ./photos --paths-only\nvisiongrep \"sunset\" ./photos --index-path ./photos.db\nvisiongrep \"sunset\" ./photos --no-cache\nvisiongrep --help\n```\n\nThe default is up to five results with a minimum cosine similarity of `0.25`. Scores are\nsimilarities, not probabilities. Results go to stdout; progress and diagnostics go to stderr.\nUse `--null` for exact paths separated by NUL bytes, or `--quiet` to suppress progress.\n\nExit codes are `0` for matches, `1` for no matches, and `2` for an operational or argument error.\n\n```\nvisiongrep index ./photos\n```\n\n`index` downloads any missing models, embeds new or changed images, and prepares both models'\noptimized graphs, without running a query. Afterwards the first search only needs to embed its\nquery. It accepts `--index-path PATH` and `--quiet`, exits `0` on success and `2` on error, and\nwrites nothing to stdout. Run it again at any time; unchanged images are skipped.\n\nIndexing saves its progress every 256 images, so an interrupted `index` or search resumes where\nit stopped when run again. An interrupted model download restarts that file. When stderr is not a\nterminal, progress is reported as `Indexed N/TOTAL images` lines instead of a progress bar.\n\nCoding agents with a shell, such as Claude Code or Codex, can call `visiongrep` directly. The\nfirst use of a folder is the slow part: it downloads about 375 MB of models and embeds every image,\nwhich can take minutes for a few thousand photos and exceed an agent's command timeout. Run\n`visiongrep index DIR` once yourself before handing the folder to an agent, or let the agent run\nit with a long timeout. After that, a new query takes well under a second and a repeated query\ntakes a few milliseconds. If an agent's command is cut off while indexing, running it again\ncontinues from the saved progress.\n\nFor many queries against one folder, start `visiongrep serve --socket DIR` in the background.\nOrdinary searches then use the loaded models automatically, with no change to the command.\n\nOutput for agents:\n\n- `--json` prints an array of`{\"path\": ..., \"score\": ...}` objects, highest score first.\n- `--paths-only` prints one path per line;`-0` separates exact paths with NUL bytes.\n- Exit code `1` means the search worked and nothing passed the threshold. The default threshold is\ndeliberately conservative, so an empty result is more likely than an unrelated guess. Use`--threshold -1` to always get the top results.\n- Progress, warnings and errors go to stderr only; `--quiet` suppresses progress.\n\nConcurrent searches can share an index. Do not run `--reindex` while other searches use the same\nindex (see the known limitation below).\n\nA snippet to adapt for `AGENTS.md` or `CLAUDE.md`:\n\n```\nTo find images by content, use `visiongrep \"<description>\" <dir> --json --top 10`, or\n`visiongrep --image <file> <dir> --json` for similar images. Exit code 1 means no image matched;\nretry with a different description or `--threshold 0.2` before concluding nothing matches.\nIf that retry returns only unrelated images, the image is not in the folder. Say so; do not open\nor scan images one by one to double-check.\nScores are raw cosine similarities (around 0.25–0.35 for good matches), not probabilities.\nNever pass `--reindex`.\n```\n\nStart an opt-in server for a search directory to reuse its loaded models across ordinary CLI searches:\n\n```\nvisiongrep serve --socket ./photos &\nvisiongrep \"a dog on a beach\" ./photos\n```\n\nThe server runs in the foreground until stopped. Searches use it automatically when the search\ndirectory and resolved index path match; otherwise they run as normal one-shot searches. Pass the\nsame `--index-path` to the server and searches when using a custom index. The server loads models\nas needed and keeps them in memory (about 400–600 MiB with both models loaded). `--no-cache`,\n`--reindex`, `--verify-models`, and `--timing` always run one-shot searches. `--reindex` refuses to\nrun while a socket server is listening for the same root and index; stop the server first.\n\n```\nvisiongrep serve --stdio --index-path photos.db ./photos\n```\n\nKeep the process open and send one JSON object per line on stdin:\n\n```\n{\"id\":1,\"query\":\"a dog on the beach\",\"top\":3}\n{\"id\":2,\"image\":\"./reference.jpg\",\"threshold\":0.3}\n```\n\nEach request receives one flushed JSON line on stdout, in request order:\n\n```\n{\"id\":1,\"results\":[{\"score\":0.31,\"path\":\"./photos/beach-dog.jpg\"}]}\n{\"id\":2,\"results\":[]}\n```\n\nProvide exactly one non-blank `query` or non-empty `image` path. `top` defaults to 5 and must\nbe positive; `threshold` defaults to 0.25 and accepts -1 through 1. The optional `id` is a\nstring or unsigned 64-bit integer and is echoed back (omitted/null IDs return null). Unknown\nfields are rejected. Paths follow the usual CLI rules: relative paths use the process's working\ndirectory, and JSON requires UTF-8 paths. To search for the literal word `serve` or `index` in\none-shot mode, place a search option first: `visiongrep --top 5 serve ./photos`.\n\nBefore each search, the service scans for additions, modifications and deletions and updates the index. SQLite stays open; text and vision models load only when needed and remain resident until exit. Image vectors are read from SQLite for each request. Keeping both models resident uses more memory than one-shot mode. The search directory is resolved at startup.\n\nRequest failures return `{\"id\":1,\"error\":\"...\"}` and the service continues. Malformed JSON or\nschema errors return a null ID. Progress and warnings go to stderr. Close stdin to shut down;\nEOF and a closed output pipe exit with code 0, even if individual requests failed or had no matches.\nStartup/I/O failures and request lines exceeding 1 MiB (including the newline) exit with code 2.\nStop the service before replacing or rebuilding its index with `--reindex`.\n\nThe default index is `.visiongrep.db` inside the searched directory. Repeated searches reuse\nunchanged image embeddings; modification time and file size determine which images need updating.\nUse `--index-path PATH` to choose another location, `--no-cache` to bypass the index, or `--reindex`\nto rebuild it before returning search results. A custom index belongs to one search root.\n\n**Known limitation: running `--reindex` concurrently with a cached search or another `--reindex`\nusing the same index can corrupt the SQLite database.** Reindexing replaces the database file\nwithout coordinating other processes that may still have it open. Atomic file replacement does\nnot make concurrent database access safe. Wait for all users of that index to finish before\nstarting `--reindex`, and let it finish before accessing the index again. This also applies to\nindexes selected with `--index-path`; concurrency protection is not implemented yet.\n\n```\ncargo fmt --all --check\ncargo check --workspace --all-targets\ncargo clippy --workspace --all-targets -- -D warnings\ncargo test --workspace -- --test-threads=1\n```\n\nThe full test suite needs cached model artifacts. See [test setup and coverage](https://github.com/udayj/visiongrep/blob/main/tests/README.md)\nfor preparation instructions and [benchmarks](https://github.com/udayj/visiongrep/blob/main/benchmarks/README.md) for performance measurements.\n\nReleases are built by [dist](https://opensource.axo.dev/cargo-dist/) from\n`dist-workspace.toml`. Bump the version in `Cargo.toml`, update `CHANGELOG.md`, merge to `main`,\nthen push a matching tag such as `v0.4.1`. The release workflow builds the three target archives,\nthe shell installer and checksums, and publishes them as a GitHub release. Run `dist generate`\nafter changing the dist configuration or upgrading dist. The crate is not published to crates.io.", "url": "https://wpnews.pro/news/show-hn-visiongrep-local-semantic-image-search-in-rust", "canonical_source": "https://github.com/udayj/visiongrep", "published_at": "2026-11-13 10:10:39+00:00", "updated_at": "2026-11-13 10:21:00.990481+00:00", "lang": "en", "topics": ["ai-tools", "computer-vision", "developer-tools", "ai-agents"], "entities": ["VisionGrep", "udayj", "Rust", "CLIP", "SQLite", "ONNX Runtime", "DataComp", "GitHub"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-visiongrep-local-semantic-image-search-in-rust", "markdown": "https://wpnews.pro/news/show-hn-visiongrep-local-semantic-image-search-in-rust.md", "text": "https://wpnews.pro/news/show-hn-visiongrep-local-semantic-image-search-in-rust.txt", "jsonld": "https://wpnews.pro/news/show-hn-visiongrep-local-semantic-image-search-in-rust.jsonld"}}