How to Make SQLite Grind Millions of Vectors on a $5 VPS with 2GB RAM (and Not Die from Out-of-Memory)

Imagine you have a cheap virtual machine with 2 GB of RAM, absolutely no Swap space, and an ambitious goal: to run a distributed AI search engine capable of processing and vectorizing thousands of incoming documents (the "Harvest" pipeline).

Most developers, upon hearing the words "vector search," immediately rush to deploy heavy enterprise solutions like pgvector, Pinecone, or Milvus. However, on a 2GB RAM machine, these memory-hungry monsters will crash from an Out-of-Memory (OOM) error before they even finish initializing.

For NGP 4.5 (NetGlyph Knowledge Protocol), we decided to embrace extreme minimalism and chose the battle-tested, time-proven SQLite. In this article, we'll show you how we tuned our embedded database to handle hundreds of transactions per second, completely eliminated file descriptor leaks, and kept memory consumption flat within a negligible margin.

During the development of our vector engine (LossySpinBosonEngine

) and document vectorizer, we encountered a classic architectural friction point. One of our AI agents ("Hermes"), responsible for auto-importing data, stored vectors like this:

def save_vector_to_db(self, vector_id, vector_data):
    cursor = self.conn.cursor()
    db_time = sqlite3.connect(self.db_path).execute("SELECT strftime('%Y-%m-%d %H:%M:%S', 'now')").fetchone()[0]
    cursor.execute("INSERT INTO vectors (id, data, created_at) VALUES (?, ?, ?)", 
                   (vector_id, vector_data, db_time))
    self.conn.commit()

sqlite3.connect(self.db_path)

directly inside the argument list, ran a query to the SQL function strftime

, and... left that connection open.OOM-Killer

would ruthlessly terminate our process before we could even process the first hundred documents.The first step in saving the system was a complete refactoring of how we manage database connections. We replaced manual SQL-based time requests with lightweight, native Python system calls and migrated to safe, idiomatic context managers.

import time
import sqlite3
import datetime

def save_vector_to_db(self, vector_id, vector_data):
    current_timestamp = time.time()


    with sqlite3.connect(self.db_path) as conn:
        conn.execute("PRAGMA journal_mode=WAL;")
        cursor = conn.cursor()
        cursor.execute(
            "INSERT OR REPLACE INTO vectors (id, data, created_at) VALUES (?, ?, ?)",
            (vector_id, vector_data, current_timestamp)
        )
        conn.commit()

What changed:

with sqlite3.connect(...) as conn:

time.time()

is an incredibly fast, nanosecond-level OS kernel system call. We cut out SQL query parsing and saved precious CPU cycles for actual vectorization.To make SQLite perform as a high-speed, concurrent embedded engine on ultra-constrained hardware, the default out-of-the-box settings simply won't cut it. Here is our optimal "Light-Weight" configuration that squeezed maximum performance on our 2GB RAM server:

with sqlite3.connect(self.db_path) as conn:
    conn.execute("PRAGMA journal_mode=WAL;")

    conn.execute("PRAGMA mmap_size=268435456;")

    conn.execute("PRAGMA cache_size=-131072;")

    conn.execute("PRAGMA busy_timeout=5000;")

    conn.execute("PRAGMA temp_store=MEMORY;")

    conn.execute("PRAGMA synchronous=NORMAL;")

journal_mode=WAL

mmap_size=256MB

cache_size=-131072

-131072 KiB = 128 MiB

). This is our armor against memory leaks.synchronous=NORMAL

NORMAL

is fully durable and secure. The database remains consistent in the event of an application crash, but the VPS disk is spared from constant block-level fsync()

system calls.In a distributed agentic system, multiple workers write to the database concurrently. To avoid the dread sqlite3.OperationalError: database is locked

, we implemented a two-level defense:

PRAGMA busy_timeout=5000

threading.Lock

import threading

db_write_lock = threading.Lock()

def thread_safe_vector_save(self, vector_id, vector_data):
    with db_write_lock:
        self.save_vector_to_db(vector_id, vector_data)

Another critical bottleneck we found was checking for table schemas on every single vector insert:

def save_vector_to_db(self, vector_id, vector_data):
    self.conn.execute("CREATE TABLE IF NOT EXISTS vectors (...)")

The Fix: Move all schema initializations and migrations (CREATE TABLE IF NOT EXISTS

) strictly into the initialization block __init__

/ _init_db()

of your database manager class. The hot saving function must perform nothing but the raw, optimized INSERT

or REPLACE

.

To prove the efficiency of this refactoring, we ran a rigorous stress test: 1,000 sequential high-dimensional vector write operations across multiple concurrent threads.

0

(all descriptors are automatically closed by Python context managers).gc.collect()

).Extreme minimalism works. Don't rush to drive nails with a microscope by spinning up heavy, expensive database clusters where a streamlined SQLite setup can get the job done elegantly. Simply tidy up your connection management, apply correct memory PRAGMAs, and isolate your write transactions.

Keep your databases monolithic, and your server memory crystal clear! 🌲

This article was prepared under the technical sovereignty framework of the NGP 4.5 project. If you'd like to see these optimizations live and test our high-performance production setup yourself, check out our sovereign, lightweight knowledge marketplace at: ** iskra-ngp.duckdns.org**.