# I Built the SQLite of Vector Search in ~120KB of Pure C & SIMD: 3,000x Faster Cold Starts and Zero Dependencies for AI Agents

> Source: <https://dev.to/eminsk/i-built-the-sqlite-of-vector-search-in-120kb-of-pure-c-simd-3000x-faster-cold-starts-and-zero-5elj>
> Published: 2026-09-11 12:17:42+00:00

If you're building LLM agents, local RAG systems, or CLI tools in Python today, you've likely faced the **vector database dependency nightmare**.

To store a few thousand embeddings from a conversation history or document chunks, standard tutorials tell you to `pip install chromadb` or install FAISS.

Here is what happens under the hood:

`pydantic`, `onnxruntime`, `tokenizers`, `fastapi`, `duckdb`, `uvicorn`, `grpcio`).` import chromadb` takes I asked myself: **Why isn't there an SQLite equivalent for vector search?**

A single, self-contained binary file. Zero external dependencies. Sub-millisecond import time. Single-file persistence (`.nvec`). 

So I built [**NanoVector**](https://github.com/eminsk/nanovector).

Benchmarked on an **Intel/AMD x86-64 CPU (AVX2+FMA)** with standard 384-dimensional embeddings (`all-MiniLM-L6-v2` / sentence-transformers):

| Metric / Feature | **NanoVector** ⚡ | **ChromaDB** 🐢 | **FAISS** ⚖️ | 
|---|---|---|---|
| **Wheel Download Size** | **38 KB** (~120 KB unpacked) | ~120 MB+ | ~50 MB+ | 
| **External Dependencies** | **0 (Zero)** | 35+ packages | OpenMP, BLAS | 
| **Python Cold Import Time** | **0.6 ms** (🚀**3,000x faster** ) | 1,850 ms | ~120 ms | 
| **Search Latency ($N=2,000$, 384D)** | **0.13 ms** (7,478 QPS) | 8.2 ms | 0.22 ms | 
| **Batch Ingestion Throughput** | **1,414,000 vectors/sec** | ~25,000 vectors/sec | ~400,000 vectors/sec | 
| **Persistence Model** | **Single `.nvec` binary file** | Multi-dir SQLite + DuckDB | Custom binary | 
| **Zero-Copy NumPy Buffer** | **Yes (Python Buffer Protocol)** | No (copies memory) | Partial | 
| **GIL Released During Search** | **Yes (`Py_BEGIN_ALLOW_THREADS`)** | Partial | Partial | 

Instead of relying on heavy linear algebra libraries (OpenBLAS, MKL) that incur function call dispatch overhead, NanoVector uses handcrafted SIMD kernels:

`float32` elements per vector register cycle with 4-way loop unrolling (32 floats per iteration) directly in CPU L1/L2 cache.`float32x4_t` registers with fused multiply-accumulates (`vmlaq_f32`).

```
       Query Vector Q (1 x D)              Database Vector Matrix (N x D)
     [ q0 q1 q2 q3 q4 q5 q6 q7 ]           [ d0 d1 d2 d3 d4 d5 d6 d7 ] -> Vector 0
                                     x     [ d0 d1 d2 d3 d4 d5 d6 d7 ] -> Vector 1
                                           [ . . . . . . . . . . . . ]
                                           [ d0 d1 d2 d3 d4 d5 d6 d7 ] -> Vector N
                      │                                   │
                      └─────────────────┬─────────────────┘
                                        ▼
                   AVX2 Dot Product Accumulator (ymm0-ymm3)
                         Exact Cosine / L2 / IP Score
```

At scale ($N < 50,000$), modern CPUs with 256-bit SIMD can compute dot products across the entire database in less than **0.2 milliseconds**. 

Approximate Nearest Neighbor (ANN) algorithms like HNSW or IVF trade off accuracy for speed, but at $N < 50k$, the graph traversal overhead and random pointer jumps actually make HNSW **slower** than sequential SIMD streaming from L2 cache!

NanoVector provides **100% exact, deterministic recall** with zero approximation errors.

`.nvec` Single-File Storage Format
Like SQLite's single `.db` file, NanoVector serializes the vector matrix, vector IDs, and optional JSON metadata strings into a compact, atomic `.nvec` file:

``` php
[Header: 32 bytes]  -> Magic 'NVEC', Version, Metric, Dim, Count
[Vectors: N * D * 4] -> Contiguous 32-byte aligned IEEE-754 floats
[String Offsets]    -> ID & Metadata index table
[Strings Data]      -> Packed UTF-8 strings
```

Saving and reloading takes **under 1 millisecond**.

Install via pip:

```
pip install nanovector
python
import nanovector
import numpy as np

# Initialize index (384D for sentence-transformers, 768D for BERT, 1536D for OpenAI)
index = nanovector.Index(dim=384, metric="cosine")

# Add embeddings with metadata
vec = np.random.randn(384).astype(np.float32)
index.add("doc_1", vec, metadata='{"title": "NanoVector Launch", "author": "eminsk"}')

# Search top-k
query = np.random.randn(384).astype(np.float32)
results = index.search(query, top_k=5)

for r in results:
    print(f"ID: {r.id} | Score: {r.score:.4f} | Meta: {r.metadata}")

# Save to a single atomic file
index.save("memory.nvec")

# Reload instantly
loaded = nanovector.load("memory.nvec")
print(f"Loaded {len(loaded)} vectors in {loaded.dim}D!")
```

Here is how you give an LLM agent persistent memory without external database infrastructure:

``` python
import os
import json
import nanovector
import numpy as np

class AgentMemory:
    def __init__(self, filepath="agent_brain.nvec", dim=384):
        self.filepath = filepath
        self.index = nanovector.load(filepath) if os.path.exists(filepath) else nanovector.Index(dim=dim, metric="cosine")

    def remember(self, turn_id: str, embedding: np.ndarray, user_prompt: str, assistant_reply: str):
        meta = json.dumps({"prompt": user_prompt, "reply": assistant_reply})
        self.index.add(turn_id, embedding, metadata=meta)
        self.index.save(self.filepath)

    def recall(self, query_embedding: np.ndarray, top_k=3):
        return self.index.search(query_embedding, top_k=top_k)

# Usage in your agent loop
brain = AgentMemory()
# Recalls relevant past experiences in 0.15 milliseconds!
memories = brain.recall(current_task_embedding, top_k=3)
```

If you're tired of 200MB Docker images and 2-second cold imports for simple vector operations, give NanoVector a spin. ⭐ Star the project on GitHub if you believe in lightweight, bare-metal software!