If you're building LLM agents, local RAG systems, or CLI tools in Python today, you've likely faced the vector database dependency nightmare.
To store a few thousand embeddings from a conversation history or document chunks, standard tutorials tell you to pip install chromadb or install FAISS.
Here is what happens under the hood:
pydantic, onnxruntime, tokenizers, fastapi, duckdb, uvicorn, grpcio). import chromadb takes I asked myself: Why isn't there an SQLite equivalent for vector search?
A single, self-contained binary file. Zero external dependencies. Sub-millisecond import time. Single-file persistence (.nvec).
So I built NanoVector.
Benchmarked on an Intel/AMD x86-64 CPU (AVX2+FMA) with standard 384-dimensional embeddings (all-MiniLM-L6-v2 / sentence-transformers):
| Metric / Feature | NanoVector ⚡ | ChromaDB 🐢 | FAISS ⚖️ |
|---|---|---|---|
| Wheel Download Size | 38 KB (~120 KB unpacked) | ~120 MB+ | ~50 MB+ |
| External Dependencies | 0 (Zero) | 35+ packages | OpenMP, BLAS |
| Python Cold Import Time | 0.6 ms (🚀3,000x faster ) | 1,850 ms | ~120 ms |
| Search Latency ($N=2,000$, 384D) | 0.13 ms (7,478 QPS) | 8.2 ms | 0.22 ms |
| Batch Ingestion Throughput | 1,414,000 vectors/sec | ~25,000 vectors/sec | ~400,000 vectors/sec |
| Persistence Model | Single .nvec binary file |
Multi-dir SQLite + DuckDB | Custom binary |
| Zero-Copy NumPy Buffer | Yes (Python Buffer Protocol) | No (copies memory) | Partial |
| GIL Released During Search | Yes (Py_BEGIN_ALLOW_THREADS) |
Partial | Partial |
Instead of relying on heavy linear algebra libraries (OpenBLAS, MKL) that incur function call dispatch overhead, NanoVector uses handcrafted SIMD kernels:
float32 elements per vector register cycle with 4-way loop unrolling (32 floats per iteration) directly in CPU L1/L2 cache.float32x4_t registers with fused multiply-accumulates (vmlaq_f32).
Query Vector Q (1 x D) Database Vector Matrix (N x D)
[ q0 q1 q2 q3 q4 q5 q6 q7 ] [ d0 d1 d2 d3 d4 d5 d6 d7 ] -> Vector 0
x [ d0 d1 d2 d3 d4 d5 d6 d7 ] -> Vector 1
[ . . . . . . . . . . . . ]
[ d0 d1 d2 d3 d4 d5 d6 d7 ] -> Vector N
│ │
└─────────────────┬─────────────────┘
▼
AVX2 Dot Product Accumulator (ymm0-ymm3)
Exact Cosine / L2 / IP Score
At scale ($N < 50,000$), modern CPUs with 256-bit SIMD can compute dot products across the entire database in less than 0.2 milliseconds.
Approximate Nearest Neighbor (ANN) algorithms like HNSW or IVF trade off accuracy for speed, but at $N < 50k$, the graph traversal overhead and random pointer jumps actually make HNSW slower than sequential SIMD streaming from L2 cache!
NanoVector provides 100% exact, deterministic recall with zero approximation errors.
.nvec Single-File Storage Format
Like SQLite's single .db file, NanoVector serializes the vector matrix, vector IDs, and optional JSON metadata strings into a compact, atomic .nvec file:
[Header: 32 bytes] -> Magic 'NVEC', Version, Metric, Dim, Count
[Vectors: N * D * 4] -> Contiguous 32-byte aligned IEEE-754 floats
[String Offsets] -> ID & Metadata index table
[Strings Data] -> Packed UTF-8 strings
Saving and re takes under 1 millisecond.
Install via pip:
pip install nanovector
python
import nanovector
import numpy as np
index = nanovector.Index(dim=384, metric="cosine")
vec = np.random.randn(384).astype(np.float32)
index.add("doc_1", vec, metadata='{"title": "NanoVector Launch", "author": "eminsk"}')
query = np.random.randn(384).astype(np.float32)
results = index.search(query, top_k=5)
for r in results:
print(f"ID: {r.id} | Score: {r.score:.4f} | Meta: {r.metadata}")
index.save("memory.nvec")
loaded = nanovector.load("memory.nvec")
print(f"Loaded {len(loaded)} vectors in {loaded.dim}D!")
Here is how you give an LLM agent persistent memory without external database infrastructure:
import os
import json
import nanovector
import numpy as np
class AgentMemory:
def __init__(self, filepath="agent_brain.nvec", dim=384):
self.filepath = filepath
self.index = nanovector.load(filepath) if os.path.exists(filepath) else nanovector.Index(dim=dim, metric="cosine")
def remember(self, turn_id: str, embedding: np.ndarray, user_prompt: str, assistant_reply: str):
meta = json.dumps({"prompt": user_prompt, "reply": assistant_reply})
self.index.add(turn_id, embedding, metadata=meta)
self.index.save(self.filepath)
def recall(self, query_embedding: np.ndarray, top_k=3):
return self.index.search(query_embedding, top_k=top_k)
brain = AgentMemory()
memories = brain.recall(current_task_embedding, top_k=3)
If you're tired of 200MB Docker images and 2-second cold imports for simple vector operations, give NanoVector a spin. ⭐ Star the project on GitHub if you believe in lightweight, bare-metal software!