{"slug": "i-built-the-sqlite-of-vector-search-in-120kb-of-pure-c-simd-3000x-faster-cold-ai", "title": "I Built the SQLite of Vector Search in ~120KB of Pure C & SIMD: 3,000x Faster Cold Starts and Zero Dependencies for AI Agents", "summary": "A developer built NanoVector, a dependency-free vector search library written in roughly 120KB of pure C with handcrafted AVX2 and NEON SIMD kernels, packaged as a 38KB Python wheel. Benchmarks against ChromaDB and FAISS on 384-dimensional embeddings show a 0.6ms cold import (about 3,000x faster than ChromaDB), 0.13ms search latency at N=2,000, and 1.41 million vectors/sec batch ingestion, with persistence in a single atomic .nvec file. The project targets local RAG and LLM agent workloads under 50,000 vectors, where the developer argues exact SIMD streaming beats approximate methods like HNSW.", "body_md": "If you're building LLM agents, local RAG systems, or CLI tools in Python today, you've likely faced the **vector database dependency nightmare**.\n\nTo store a few thousand embeddings from a conversation history or document chunks, standard tutorials tell you to `pip install chromadb` or install FAISS.\n\nHere is what happens under the hood:\n\n`pydantic`, `onnxruntime`, `tokenizers`, `fastapi`, `duckdb`, `uvicorn`, `grpcio`).` import chromadb` takes I asked myself: **Why isn't there an SQLite equivalent for vector search?**\n\nA single, self-contained binary file. Zero external dependencies. Sub-millisecond import time. Single-file persistence (`.nvec`). \n\nSo I built [**NanoVector**](https://github.com/eminsk/nanovector).\n\nBenchmarked on an **Intel/AMD x86-64 CPU (AVX2+FMA)** with standard 384-dimensional embeddings (`all-MiniLM-L6-v2` / sentence-transformers):\n\n| Metric / Feature | **NanoVector** ⚡ | **ChromaDB** 🐢 | **FAISS** ⚖️ | \n|---|---|---|---|\n| **Wheel Download Size** | **38 KB** (~120 KB unpacked) | ~120 MB+ | ~50 MB+ | \n| **External Dependencies** | **0 (Zero)** | 35+ packages | OpenMP, BLAS | \n| **Python Cold Import Time** | **0.6 ms** (🚀**3,000x faster** ) | 1,850 ms | ~120 ms | \n| **Search Latency ($N=2,000$, 384D)** | **0.13 ms** (7,478 QPS) | 8.2 ms | 0.22 ms | \n| **Batch Ingestion Throughput** | **1,414,000 vectors/sec** | ~25,000 vectors/sec | ~400,000 vectors/sec | \n| **Persistence Model** | **Single `.nvec` binary file** | Multi-dir SQLite + DuckDB | Custom binary | \n| **Zero-Copy NumPy Buffer** | **Yes (Python Buffer Protocol)** | No (copies memory) | Partial | \n| **GIL Released During Search** | **Yes (`Py_BEGIN_ALLOW_THREADS`)** | Partial | Partial | \n\nInstead of relying on heavy linear algebra libraries (OpenBLAS, MKL) that incur function call dispatch overhead, NanoVector uses handcrafted SIMD kernels:\n\n`float32` elements per vector register cycle with 4-way loop unrolling (32 floats per iteration) directly in CPU L1/L2 cache.`float32x4_t` registers with fused multiply-accumulates (`vmlaq_f32`).\n\n```\n       Query Vector Q (1 x D)              Database Vector Matrix (N x D)\n     [ q0 q1 q2 q3 q4 q5 q6 q7 ]           [ d0 d1 d2 d3 d4 d5 d6 d7 ] -> Vector 0\n                                     x     [ d0 d1 d2 d3 d4 d5 d6 d7 ] -> Vector 1\n                                           [ . . . . . . . . . . . . ]\n                                           [ d0 d1 d2 d3 d4 d5 d6 d7 ] -> Vector N\n                      │                                   │\n                      └─────────────────┬─────────────────┘\n                                        ▼\n                   AVX2 Dot Product Accumulator (ymm0-ymm3)\n                         Exact Cosine / L2 / IP Score\n```\n\nAt scale ($N < 50,000$), modern CPUs with 256-bit SIMD can compute dot products across the entire database in less than **0.2 milliseconds**. \n\nApproximate Nearest Neighbor (ANN) algorithms like HNSW or IVF trade off accuracy for speed, but at $N < 50k$, the graph traversal overhead and random pointer jumps actually make HNSW **slower** than sequential SIMD streaming from L2 cache!\n\nNanoVector provides **100% exact, deterministic recall** with zero approximation errors.\n\n`.nvec` Single-File Storage Format\nLike SQLite's single `.db` file, NanoVector serializes the vector matrix, vector IDs, and optional JSON metadata strings into a compact, atomic `.nvec` file:\n\n``` php\n[Header: 32 bytes]  -> Magic 'NVEC', Version, Metric, Dim, Count\n[Vectors: N * D * 4] -> Contiguous 32-byte aligned IEEE-754 floats\n[String Offsets]    -> ID & Metadata index table\n[Strings Data]      -> Packed UTF-8 strings\n```\n\nSaving and reloading takes **under 1 millisecond**.\n\nInstall via pip:\n\n```\npip install nanovector\npython\nimport nanovector\nimport numpy as np\n\n# Initialize index (384D for sentence-transformers, 768D for BERT, 1536D for OpenAI)\nindex = nanovector.Index(dim=384, metric=\"cosine\")\n\n# Add embeddings with metadata\nvec = np.random.randn(384).astype(np.float32)\nindex.add(\"doc_1\", vec, metadata='{\"title\": \"NanoVector Launch\", \"author\": \"eminsk\"}')\n\n# Search top-k\nquery = np.random.randn(384).astype(np.float32)\nresults = index.search(query, top_k=5)\n\nfor r in results:\n    print(f\"ID: {r.id} | Score: {r.score:.4f} | Meta: {r.metadata}\")\n\n# Save to a single atomic file\nindex.save(\"memory.nvec\")\n\n# Reload instantly\nloaded = nanovector.load(\"memory.nvec\")\nprint(f\"Loaded {len(loaded)} vectors in {loaded.dim}D!\")\n```\n\nHere is how you give an LLM agent persistent memory without external database infrastructure:\n\n``` python\nimport os\nimport json\nimport nanovector\nimport numpy as np\n\nclass AgentMemory:\n    def __init__(self, filepath=\"agent_brain.nvec\", dim=384):\n        self.filepath = filepath\n        self.index = nanovector.load(filepath) if os.path.exists(filepath) else nanovector.Index(dim=dim, metric=\"cosine\")\n\n    def remember(self, turn_id: str, embedding: np.ndarray, user_prompt: str, assistant_reply: str):\n        meta = json.dumps({\"prompt\": user_prompt, \"reply\": assistant_reply})\n        self.index.add(turn_id, embedding, metadata=meta)\n        self.index.save(self.filepath)\n\n    def recall(self, query_embedding: np.ndarray, top_k=3):\n        return self.index.search(query_embedding, top_k=top_k)\n\n# Usage in your agent loop\nbrain = AgentMemory()\n# Recalls relevant past experiences in 0.15 milliseconds!\nmemories = brain.recall(current_task_embedding, top_k=3)\n```\n\nIf you're tired of 200MB Docker images and 2-second cold imports for simple vector operations, give NanoVector a spin. ⭐ Star the project on GitHub if you believe in lightweight, bare-metal software!", "url": "https://wpnews.pro/news/i-built-the-sqlite-of-vector-search-in-120kb-of-pure-c-simd-3000x-faster-cold-ai", "canonical_source": "https://dev.to/eminsk/i-built-the-sqlite-of-vector-search-in-120kb-of-pure-c-simd-3000x-faster-cold-starts-and-zero-5elj", "published_at": "2026-09-11 12:17:42+00:00", "updated_at": "2026-09-11 12:40:48.696354+00:00", "lang": "en", "topics": ["ai-tools", "ai-agents", "ai-infrastructure", "developer-tools"], "entities": ["NanoVector", "ChromaDB", "FAISS", "all-MiniLM-L6-v2", "sentence-transformers", "NumPy", "SQLite", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/i-built-the-sqlite-of-vector-search-in-120kb-of-pure-c-simd-3000x-faster-cold-ai", "markdown": "https://wpnews.pro/news/i-built-the-sqlite-of-vector-search-in-120kb-of-pure-c-simd-3000x-faster-cold-ai.md", "text": "https://wpnews.pro/news/i-built-the-sqlite-of-vector-search-in-120kb-of-pure-c-simd-3000x-faster-cold-ai.txt", "jsonld": "https://wpnews.pro/news/i-built-the-sqlite-of-vector-search-in-120kb-of-pure-c-simd-3000x-faster-cold-ai.jsonld"}}