{"slug": "everyone-s-trying-vectors-and-graphs-for-ai-memory-we-went-back-to-sql", "title": "Everyone's Trying Vectors and Graphs for AI Memory. We Went Back to SQL.", "summary": "A developer argues that SQL, not vector databases, is the better default for agentic LLM memory in most workloads, citing deterministic queries, lower ops overhead, and predictable recall. The post demonstrates SQLite-based memory tables with timestamp and tag filters, and shows brute-force embedding similarity in SQL can match vector DB performance for tables under 100k items.", "body_md": "The default in 2024 for agentic LLM memory is a vector DB with a glossy API claiming semantic search at scale. Docs hype embedding-powered lookups and treat SQL as legacy.\n\nFor most agentic workloads, that's backwards. Unless you're at hundred-million-vector scale, vector DBs add sync/async glue, ops overhead, surprising ANN recall quirks, and debugging headaches. You lose the ability to reason predictably about queries by user, timestamp, or tag.\n\nClassic SQL wins when you need precise, scoped recall—\"what facts are in my recent working memory, for topic X, since 10 minutes ago\"—in a single query. Vector search gives you best-effort, top-k returns, often unstable when embeddings drift or you upgrade models. ANN recall is not a drop-in replacement for classic key+filter queries.\n\nVector DBs are for pure semantic smash-and-grab, not layered or episodic memory. SQL handles all the details—structured, temporal, filtered recall—directly.\n\nConcrete example: SQLite “memory” table with text, timestamp, tag, and blob embedding. To fetch agent memories from the past hour tagged ‘alpha’:\n\n``` python\nimport sqlite3\nimport time\n\nconn = sqlite3.connect(\":memory:\")\ncur = conn.cursor()\ncur.execute(\"\"\"\nCREATE TABLE memory (\n    id INTEGER PRIMARY KEY,\n    text TEXT,\n    ts REAL,\n    tag TEXT,\n    embedding BLOB\n)\n\"\"\")\nnow = time.time()\ncur.execute(\"INSERT INTO memory (text, ts, tag) VALUES (?, ?, ?)\", (\"Fix bug in alpha repo\", now - 60, \"alpha\"))\ncur.execute(\"INSERT INTO memory (text, ts, tag) VALUES (?, ?, ?)\", (\"Lunch with team\", now - 120, \"social\"))\ncur.execute(\"INSERT INTO memory (text, ts, tag) VALUES (?, ?, ?)\", (\"Review alpha doc\", now - 180, \"alpha\"))\nconn.commit()\n\nrecent = cur.execute(\n    \"SELECT text FROM memory WHERE tag = ? AND ts > ?\", (\"alpha\", now - 120)\n).fetchall()\nprint([r[0] for r in recent])  # Output: ['Fix bug in alpha repo']\n```\n\nNo ANN approximation, no hybrid postfilter. Deterministic, explainable, instantly extensible.\n\nTry this with Chroma or Pinecone and you end up wrestling with post-hoc filtering, multi-stage APIs, or hand-jamming your own query+filter loop.\n\n\"Fine, but what about semantic similarity?\" You can store embeddings in SQL and run brute-force similarity for agent memory. For tables under 100k items, in-memory SQLite with vector columns remains fast and local.\n\n``` python\nimport numpy as np\nimport sqlite3\n\ndef make_embedding(text):\n    return np.random.rand(384).astype(np.float32)  # Real model would go here\n\nconn = sqlite3.connect(\":memory:\")\ncur = conn.cursor()\ncur.execute(\"\"\"\nCREATE TABLE memory (\n    id INTEGER PRIMARY KEY,\n    text TEXT,\n    embedding BLOB\n)\n\"\"\")\nfor txt in [\"Fix alpha bug\", \"Go to lunch\", \"Review docs\"]:\n    emb = make_embedding(txt).tobytes()\n    cur.execute(\"INSERT INTO memory (text, embedding) VALUES (?, ?)\", (txt, emb))\nconn.commit()\n\ndef cosine_sim(a, b):\n    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))\n\nq_emb = make_embedding(\"alpha bugfix\")\nrows = cur.execute(\"SELECT text, embedding FROM memory\").fetchall()\nsims = [(txt, cosine_sim(np.frombuffer(emb, np.float32), q_emb)) for txt, emb in rows]\ntop = sorted(sims, key=lambda x: -x[1])[0]\nprint(\"Best SQL match:\", top)\npython\n# pip install chromadb sentence-transformers\nimport chromadb\nfrom sentence_transformers import SentenceTransformer\n\nclient = chromadb.Client()\ncollection = client.create_collection(\"memory\")\nmodel = SentenceTransformer('all-MiniLM-L6-v2')\n\ndocs = [\"Fix alpha bug\", \"Go to lunch\", \"Review docs\"]\nids = [str(i) for i in range(len(docs))]\nembeddings = model.encode(docs).tolist()\ncollection.add(documents=docs, ids=ids, embeddings=embeddings)\n\nq_emb = model.encode([\"alpha bugfix\"]).tolist()\nresults = collection.query(query_embeddings=q_emb, n_results=1)\nprint(\"Best vector DB match:\", results['documents'][0][0])\n```\n\nOn tables <100k items, brute-force numpy-in-SQLite hits 5-20ms/query. Networked vector DBs rarely match that for individual queries unless batch-mode or sharded. At 1M+ rows or multi-writer concurrency, SQL falls behind unless you migrate to an ops-heavy faiss or pq-index setup. For agentic memory under 4k tokens or <50 document recall per turn, classic SQL is hard to beat for speed and transparency.\n\nAgent memory isn't a flat event log. There's a recent cache, an episodic/workspace slice, and a long-term archive. All of this models cleanly in SQL.\n\nTypical tables:\n\n`memory_event`\n\n: `id`\n\n, `timestamp`\n\n, `agent_id`\n\n, `level`\n\n(cache/working/long_term), `text`\n\n, `tag`\n\n, `embedding`\n\n`memory_index`\n\n: aggregated contexts, multi-agent links, session chains`(agent_id,level,timestamp)`\n\n, `(embedding)`\n\n**Multi-Tier Recall Diagram:**\n\nPicture three SQL tables:\n\n**Cache**\n\nRows expire via TTL or are purged by time-based jobs. Recent, high-frequency events with tight recall bounds.\n\n**Working Memory**\n\nLast N events per agent/session. Indexed for fast slice by `(agent_id, timestamp)`\n\n.\n\n**Long-Term**\n\nArchive, compressed or vectorized. Used for distant recall, batch jobs, or semantic search.\n\nPromotion and TTL triangle: events move up to long-term; expired/dormant items evicted down. Queries cross levels via SQL unions or joined predicates—no routers, no multi-pass graph walks.\n\nPlain SQL with embeddings breaks for:\n\nThose cases are rare for agentic memory workloads. Unless you need unstructured instant ANN for millions+ items or deep semantic graph reasoning, classic SQL remains the lowest-latency and most understandable answer.\n\n| Use Case | SQL MemTable | Vector DB / Chroma | Knowledge Graph |\n|---|---|---|---|\n| Current working memory (<10k items) | 🟢 Fast, clear | 🟡 Sometimes overkill | 🔴 Overcomplex |\n| Semantic + structural (e.g. tag, time) | 🟢 Single query | 🟡 Needs hybrid query | 🔴 Postfilter or custom walk |\n| Distant context, pure semantic | 🟡 Slow at scale | 🟢 Native support | 🟢 Can map, less explainable |\n| Ultra-large (>1M) unstructured recall | 🔴 Fails | 🟢 Scales up | 🟡 Graph plausible |\n| Multi-hop relationships, concept graphs | 🔴 Not native | 🔴 Not ideal | 🟢 Built for this |\n\nSQL remains the best default for agentic “memory” unless you truly require large-scale semantic or graph search. Bench your workload before assuming vector DBs or knowledge graphs are upgrades. For most agent memory, they’re complexity—SQL is the real fast path.", "url": "https://wpnews.pro/news/everyone-s-trying-vectors-and-graphs-for-ai-memory-we-went-back-to-sql", "canonical_source": "https://dev.to/priyeshdave6/everyones-trying-vectors-and-graphs-for-ai-memory-we-went-back-to-sql-3kmi", "published_at": "2026-08-23 16:42:52+00:00", "updated_at": "2026-08-23 17:14:07.946309+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-infrastructure", "developer-tools"], "entities": ["SQLite", "Chroma", "Pinecone", "SentenceTransformer"], "alternates": {"html": "https://wpnews.pro/news/everyone-s-trying-vectors-and-graphs-for-ai-memory-we-went-back-to-sql", "markdown": "https://wpnews.pro/news/everyone-s-trying-vectors-and-graphs-for-ai-memory-we-went-back-to-sql.md", "text": "https://wpnews.pro/news/everyone-s-trying-vectors-and-graphs-for-ai-memory-we-went-back-to-sql.txt", "jsonld": "https://wpnews.pro/news/everyone-s-trying-vectors-and-graphs-for-ai-memory-we-went-back-to-sql.jsonld"}}