I Built the SQLite of Vector Search in ~120KB of Pure C & SIMD: 3,000x Faster Cold Starts and Zero Dependencies for AI Agents

A developer built NanoVector, a dependency-free vector search library written in roughly 120KB of pure C with handcrafted AVX2 and NEON SIMD kernels, packaged as a 38KB Python wheel. Benchmarks against ChromaDB and FAISS on 384-dimensional embeddings show a 0.6ms cold import (about 3,000x faster than ChromaDB), 0.13ms search latency at N=2,000, and 1.41 million vectors/sec batch ingestion, with persistence in a single atomic .nvec file. The project targets local RAG and LLM agent workloads under 50,000 vectors, where the developer argues exact SIMD streaming beats approximate methods like HNSW.

If you're building LLM agents, local RAG systems, or CLI tools in Python today, you've likely faced the vector database dependency nightmare . To store a few thousand embeddings from a conversation history or document chunks, standard tutorials tell you to pip install chromadb or install FAISS. Here is what happens under the hood: pydantic , onnxruntime , tokenizers , fastapi , duckdb , uvicorn , grpcio . import chromadb takes I asked myself: Why isn't there an SQLite equivalent for vector search? A single, self-contained binary file. Zero external dependencies. Sub-millisecond import time. Single-file persistence .nvec . So I built NanoVector https://github.com/eminsk/nanovector . Benchmarked on an Intel/AMD x86-64 CPU AVX2+FMA with standard 384-dimensional embeddings all-MiniLM-L6-v2 / sentence-transformers : | Metric / Feature | NanoVector ⚡ | ChromaDB 🐢 | FAISS ⚖️ | |---|---|---|---| | Wheel Download Size | 38 KB ~120 KB unpacked | ~120 MB+ | ~50 MB+ | | External Dependencies | 0 Zero | 35+ packages | OpenMP, BLAS | | Python Cold Import Time | 0.6 ms 🚀 3,000x faster | 1,850 ms | ~120 ms | | Search Latency $N=2,000$, 384D | 0.13 ms 7,478 QPS | 8.2 ms | 0.22 ms | | Batch Ingestion Throughput | 1,414,000 vectors/sec | ~25,000 vectors/sec | ~400,000 vectors/sec | | Persistence Model | Single .nvec binary file | Multi-dir SQLite + DuckDB | Custom binary | | Zero-Copy NumPy Buffer | Yes Python Buffer Protocol | No copies memory | Partial | | GIL Released During Search | Yes Py BEGIN ALLOW THREADS | Partial | Partial | Instead of relying on heavy linear algebra libraries OpenBLAS, MKL that incur function call dispatch overhead, NanoVector uses handcrafted SIMD kernels: float32 elements per vector register cycle with 4-way loop unrolling 32 floats per iteration directly in CPU L1/L2 cache. float32x4 t registers with fused multiply-accumulates vmlaq f32 . Query Vector Q 1 x D Database Vector Matrix N x D q0 q1 q2 q3 q4 q5 q6 q7 d0 d1 d2 d3 d4 d5 d6 d7 - Vector 0 x d0 d1 d2 d3 d4 d5 d6 d7 - Vector 1 . . . . . . . . . . . . d0 d1 d2 d3 d4 d5 d6 d7 - Vector N │ │ └─────────────────┬─────────────────┘ ▼ AVX2 Dot Product Accumulator ymm0-ymm3 Exact Cosine / L2 / IP Score At scale $N < 50,000$ , modern CPUs with 256-bit SIMD can compute dot products across the entire database in less than 0.2 milliseconds . Approximate Nearest Neighbor ANN algorithms like HNSW or IVF trade off accuracy for speed, but at $N < 50k$, the graph traversal overhead and random pointer jumps actually make HNSW slower than sequential SIMD streaming from L2 cache NanoVector provides 100% exact, deterministic recall with zero approximation errors. .nvec Single-File Storage Format Like SQLite's single .db file, NanoVector serializes the vector matrix, vector IDs, and optional JSON metadata strings into a compact, atomic .nvec file: php Header: 32 bytes - Magic 'NVEC', Version, Metric, Dim, Count Vectors: N D 4 - Contiguous 32-byte aligned IEEE-754 floats String Offsets - ID & Metadata index table Strings Data - Packed UTF-8 strings Saving and reloading takes under 1 millisecond . Install via pip: pip install nanovector python import nanovector import numpy as np Initialize index 384D for sentence-transformers, 768D for BERT, 1536D for OpenAI index = nanovector.Index dim=384, metric="cosine" Add embeddings with metadata vec = np.random.randn 384 .astype np.float32 index.add "doc 1", vec, metadata='{"title": "NanoVector Launch", "author": "eminsk"}' Search top-k query = np.random.randn 384 .astype np.float32 results = index.search query, top k=5 for r in results: print f"ID: {r.id} | Score: {r.score:.4f} | Meta: {r.metadata}" Save to a single atomic file index.save "memory.nvec" Reload instantly loaded = nanovector.load "memory.nvec" print f"Loaded {len loaded } vectors in {loaded.dim}D " Here is how you give an LLM agent persistent memory without external database infrastructure: python import os import json import nanovector import numpy as np class AgentMemory: def init self, filepath="agent brain.nvec", dim=384 : self.filepath = filepath self.index = nanovector.load filepath if os.path.exists filepath else nanovector.Index dim=dim, metric="cosine" def remember self, turn id: str, embedding: np.ndarray, user prompt: str, assistant reply: str : meta = json.dumps {"prompt": user prompt, "reply": assistant reply} self.index.add turn id, embedding, metadata=meta self.index.save self.filepath def recall self, query embedding: np.ndarray, top k=3 : return self.index.search query embedding, top k=top k Usage in your agent loop brain = AgentMemory Recalls relevant past experiences in 0.15 milliseconds memories = brain.recall current task embedding, top k=3 If you're tired of 200MB Docker images and 2-second cold imports for simple vector operations, give NanoVector a spin. ⭐ Star the project on GitHub if you believe in lightweight, bare-metal software