Agentic Memory Patterns in the Wild: Vector, Graph, and SQL Approaches from Recent Open-Source Systems

An analysis of open-source agentic memory systems finds that vector, SQL, and graph backends each fail in predictable ways and cannot be treated as pluggable. Vector stores such as TinyMemory and Butter offer fast fuzzy retrieval (10–40ms per query on a 10k-row local database) but are unauditable and can silently corrupt their index on a mid-write crash, while SQL backends like UltraContext and OpenClaw run 20–110ms with FTS5 (150–650ms with LIKE fallback) and provide full transaction logs and provenance.

Open-source AI agents break without auditable, resilient memory. Most “agent frameworks” claim to address this, but actual open implementations diverge in key ways. Agentic memory is nowhere near generic cache—with embedding, context refetch, audit surface, and latent failure modes baked in. Below is what happens in practice: vector, SQL, and graph each break for specific, predictable reasons. Agent loops aren’t memory-agnostic. Every choice—vector, SQL, or graph—creates tradeoffs that surface in API quirks and catastrophic failures. Control loop code from https://github.com/tinyhumansai/tinymemory https://github.com/tinyhumansai/tinymemory and https://github.com/delicious-roundarch520/openclaw-memory-kit https://github.com/delicious-roundarch520/openclaw-memory-kit shows: memory isn’t pluggable. Backend coupling governs how you propagate errors, batch queries, and chunk user context. Example from TinyMemory https://github.com/tinyhumansai/tinymemory/blob/main/examples/agent.py : python from tinymemory import MemoryStore memory = MemoryStore persist dir="your data" agent = Agent memory=memory OpenClaw’s SQL memory source https://github.com/delicious-roundarch520/openclaw-memory-kit/blob/main/api/memory backend/sql backend.py : python from sql memory import SQLMemory memory = SQLMemory sqlite path="data/memory.sqlite" agent = Agent memory=memory The agent loop must handle batch embedding vector vs. transaction and race issues SQL . Query batching, windowing, and failure reporting—none are abstracted from memory backend. Works until you need to audit the source of an answer. Vector store memory TinyMemory, Butter offers constant-time embedding lookups, fuzzy retrieval, and easy scaling. Integration is trivial: python from tinymemory import MemoryStore memory = MemoryStore persist dir="memdb" observation = { "text": "User asked about Q3 revenue targets.", "tags": "finance", "Q3" , } memory.add observation res = memory.search "What were last quarter's sales targets?", top k=2 for r in res: print r 'text' Retrieval is fuzzy and unauditable. A rephrased query can yield surprising or irrelevant matches. There’s no join, no trace, and no way to map answers back to specific context. Try tracing an LLM mistake to the original input—vector stores are black boxes. Index corruption is silent. Check TinyMemory’s persist https://github.com/tinyhumansai/tinymemory/blob/main/tinymemory/store.py L40 : no transaction log, no locking. A crash mid-write yields a partial, undetectably corrupt index. Butter is similar—fast by default, frail during edge cases. Benchmark: Local 10k-row db, MemoryStore.search : 10–40ms per query, scaling linearly. For small projects: fast and simple. If you need traceability, look elsewhere. SQL backends UltraContext, OpenClaw are slower but make every memory transaction transparent. UltraContext and OpenClaw expose full logs and joins, not just what answer, but why. Example based on OpenClaw-memory-kit https://github.com/delicious-roundarch520/openclaw-memory-kit : python from sql memory import SQLMemory memory = SQLMemory sqlite path="data/memory.sqlite" memory.add { "text": "CEO confirmed Q4 layoffs.", "source": "transcript 2024 03 10.txt", "timestamp": "2024-03-10T12:17:00" } results = memory.search "layoffs", limit=5 for r in results: print r "text" , r "source" Advantages: SQL is natively auditable. Transactions, rollbacks, FTS indices, and permissioning come for free. You can run SELECT FROM observations WHERE ... and reconstruct provenance. Performance: 10k-row SQLite with FTS5: 20–110ms per query. LIKE-based fallback: 150–650ms . Overhead is linear; no built-in replication. Failure: Every fact is traceable, but speed suffers. Graph memory is the “explainability” darling, but shifting from event logs or embeddings to subject-predicate-object isn’t free. It exposes relationship structure—for audit and queries—at the cost of data modeling and operational pain. Sample pattern using networkx inline, not pseudocode : python import networkx as nx class GraphMemory: def init self : self.graph = nx.DiGraph def add observation self, entity, attr, value : self.graph.add edge entity, attr, value=value def query self, entity : return list self.graph entity .items memory = GraphMemory memory.add observation "CEO", "announced", "Q4 layoffs" memory.add observation "Q4 layoffs", "affects", "Engineering" result = memory.query "CEO" print result 'announced', {'value': 'Q4 layoffs'} Problems: Upsides: Prompt → Embedding Query → Top-K Search → Prompt Augmented → LLM | Opaque match, no join trace, silent failures Plug-and-play integration. Failures are silent and fuzzy—no recovery path except trial/error. Prompt → SQL Query → Selected Rows → Prompt Augmented → LLM | Traceable, schema-bound, exceptions available You pay in sync logic and schema migrations, but failures are explicit and auditable. Prompt → Graph Traverse → Nodes/Edges → Prompt Augmented → LLM | Rich context, expensive queries, model complexity Relationship expressivity wins, but debugging cycles or bad traversals is non-trivial. Bottom line: Agentic memory isn’t one hot-swappable line. Everything you care about—latency, audit, recovery, explainability—traces back to this backend choice. Annotated Benchmarks local SSD, Python 3.10, no RAMdisk : | Backend | Query Time 10k rows | Traceability | Failure Reporting | |---|---|---|---| | Vector/TinyMemory | 10–40ms | ❌ | Silent/Fuzzy | | SQL/OpenClaw | 20–110ms FTS5 | ✅ | Exception, rollback | | Graph networkx | 30–500ms | ✅ | Traversal/cycle | Graph: varies by topology. Real agent reliability lives and dies on memory transparency. Choose audit trails SQL , fast hacks vector , or explainability graph , but know what breaks first. The abstraction is a lie; architecture is destiny.