How to Make SQLite Grind Millions of Vectors on a $5 VPS with 2GB RAM (and Not Die from Out-of-Memory)

NGP 4.5 (NetGlyph Knowledge Protocol) developers optimized SQLite to handle millions of vectors on a 2GB RAM VPS without OOM crashes. They refactored database connections to use context managers and native Python time calls, and applied PRAGMA settings like WAL, mmap_size, and cache_size to keep memory flat. The project's vector engine, LossySpinBosonEngine, and document vectorizer now run efficiently on constrained hardware.

Imagine you have a cheap virtual machine with 2 GB of RAM , absolutely no Swap space, and an ambitious goal: to run a distributed AI search engine capable of processing and vectorizing thousands of incoming documents the "Harvest" pipeline . Most developers, upon hearing the words "vector search," immediately rush to deploy heavy enterprise solutions like pgvector, Pinecone, or Milvus. However, on a 2GB RAM machine, these memory-hungry monsters will crash from an Out-of-Memory OOM error before they even finish initializing. For NGP 4.5 NetGlyph Knowledge Protocol , we decided to embrace extreme minimalism and chose the battle-tested, time-proven SQLite . In this article, we'll show you how we tuned our embedded database to handle hundreds of transactions per second, completely eliminated file descriptor leaks, and kept memory consumption flat within a negligible margin. During the development of our vector engine LossySpinBosonEngine and document vectorizer, we encountered a classic architectural friction point. One of our AI agents "Hermes" , responsible for auto-importing data, stored vectors like this: python BAD: A hidden resource leak waiting to happen def save vector to db self, vector id, vector data : cursor = self.conn.cursor Massive descriptor leak sqlite3.connect opens and hangs in memory db time = sqlite3.connect self.db path .execute "SELECT strftime '%Y-%m-%d %H:%M:%S', 'now' " .fetchone 0 cursor.execute "INSERT INTO vectors id, data, created at VALUES ?, ?, ? ", vector id, vector data, db time self.conn.commit sqlite3.connect self.db path directly inside the argument list, ran a query to the SQL function strftime , and... left that connection open. OOM-Killer would ruthlessly terminate our process before we could even process the first hundred documents.The first step in saving the system was a complete refactoring of how we manage database connections. We replaced manual SQL-based time requests with lightweight, native Python system calls and migrated to safe, idiomatic context managers. python import time import sqlite3 import datetime def save vector to db self, vector id, vector data : Method 1: Get Unix Epoch zero overhead, float current timestamp = time.time Method 2: Python-native datetime string no database hits required current timestamp = datetime.datetime.now .strftime "%Y-%m-%d %H:%M:%S" Guaranteed connection closure via context managers with sqlite3.connect self.db path as conn: conn.execute "PRAGMA journal mode=WAL;" cursor = conn.cursor cursor.execute "INSERT OR REPLACE INTO vectors id, data, created at VALUES ?, ?, ? ", vector id, vector data, current timestamp conn.commit What changed: with sqlite3.connect ... as conn: time.time is an incredibly fast, nanosecond-level OS kernel system call. We cut out SQL query parsing and saved precious CPU cycles for actual vectorization.To make SQLite perform as a high-speed, concurrent embedded engine on ultra-constrained hardware, the default out-of-the-box settings simply won't cut it. Here is our optimal "Light-Weight" configuration that squeezed maximum performance on our 2GB RAM server: with sqlite3.connect self.db path as conn: 1. Enable Write-Ahead Logging WAL conn.execute "PRAGMA journal mode=WAL;" 2. Optimize virtual memory mapping mmap Instead of a massive 32GB default, allocate a modest but efficient 256MB conn.execute "PRAGMA mmap size=268435456;" 3. Hard-limit page cache size in RAM to 128MB Negative value in SQLite configures the cache strictly in Kibibytes KiB conn.execute "PRAGMA cache size=-131072;" 4. Prevent Deadlocks under concurrent load conn.execute "PRAGMA busy timeout=5000;" 5. Store temporary tables only in RAM conn.execute "PRAGMA temp store=MEMORY;" 6. Relax disk sync for WAL conn.execute "PRAGMA synchronous=NORMAL;" journal mode=WAL mmap size=256MB cache size=-131072 -131072 KiB = 128 MiB . This is our armor against memory leaks. synchronous=NORMAL NORMAL is fully durable and secure. The database remains consistent in the event of an application crash, but the VPS disk is spared from constant block-level fsync system calls.In a distributed agentic system, multiple workers write to the database concurrently. To avoid the dread sqlite3.OperationalError: database is locked , we implemented a two-level defense: PRAGMA busy timeout=5000 threading.Lock python import threading db write lock = threading.Lock def thread safe vector save self, vector id, vector data : with db write lock: self.save vector to db vector id, vector data Another critical bottleneck we found was checking for table schemas on every single vector insert: python BAD: Slow hot-path with continuous parser locks def save vector to db self, vector id, vector data : Checking schemas on every insert stresses the SQLite parser self.conn.execute "CREATE TABLE IF NOT EXISTS vectors ... " The Fix: Move all schema initializations and migrations CREATE TABLE IF NOT EXISTS strictly into the initialization block init / init db of your database manager class. The hot saving function must perform nothing but the raw, optimized INSERT or REPLACE . To prove the efficiency of this refactoring, we ran a rigorous stress test: 1,000 sequential high-dimensional vector write operations across multiple concurrent threads. 0 all descriptors are automatically closed by Python context managers . gc.collect .Extreme minimalism works. Don't rush to drive nails with a microscope by spinning up heavy, expensive database clusters where a streamlined SQLite setup can get the job done elegantly. Simply tidy up your connection management, apply correct memory PRAGMAs, and isolate your write transactions. Keep your databases monolithic, and your server memory crystal clear 🌲 This article was prepared under the technical sovereignty framework of the NGP 4.5 project. If you'd like to see these optimizations live and test our high-performance production setup yourself, check out our sovereign, lightweight knowledge marketplace at: iskra-ngp.duckdns.org .