{"slug": "agy-memory-engine-zero-dependency-sqlite-fts5-fact-store-and-mcp-server", "title": "AGY Memory Engine: Zero-dependency SQLite FTS5 fact store and MCP server", "summary": "AGY Memory Engine, a zero-dependency SQLite FTS5 fact store and MCP server, delivers sub-2ms local retrieval with 0 MB extra RAM, serving as a persistent semantic backbone for Google Antigravity (agy) and autonomous agent frameworks. The engine partitions memory into three pillars—episodic, semantic, and procedural—to avoid vector DB bloat and context-stuffing, enabling exact, deterministic recall of core facts like IPs and preferences.", "body_md": "Lightweight, high-performance, standalone dynamic memory layer for Google Antigravity (\n\n`agy`\n\n) and autonomous agent frameworks.\n\nInspired by Hermes Agent's 3-pillar memory architecture, using SQLite FTS5 for ultra-fast local retrieval (<2ms) and autonomous LLM background extraction for continuous long-term learning without prompt bloat.\n\n`agy-memory-engine`\n\nis designed as the persistent semantic backbone of a 24/7 personal autonomous agent stack:\n\n```\n                  ┌────────────────────────────────────────┐\n                  │          User on Telegram UI           │\n                  │   (\"Deploy staging update on prod-server\") │\n                  └──────────────────┬─────────────────────┘\n                                     │\n                                     ▼\n                   ┌───────────────────────────────────┐\n                   │    Telegram Gateway / Sidecar     │\n                   └───────┬───────────────────▲───────┘\n                           │                   │\n               1. Pre-fetch│                   │ 5. Telegram\n                 (< 2ms)   │                   │    Response\n                           ▼                   │\n            ┌─────────────────────────────┐    │\n            │   AGY Memory Engine (FTS5)  │    │\n            │   - Staging IP: 192.168.1.50│    │\n            │   - SSH Port: 2222          │    │\n            │   - User Prefs & Hardware   │    │\n            └──────────────┬──────────────┘    │\n                           │ 2. Injected       │\n                           │    Ephemeral      │\n                           │    Context        │\n                           ▼                   │\n            ┌──────────────────────────────────┴──┐\n            │   Google Antigravity CLI (`agy`)    │\n            │   - Autonomous Execution            │\n            │   - Tool Calls / Skills / MCP       │\n            │   - Multi-Turn Reasoning            │\n            └──────────────┬──────────────────────┘\n                           │\n                           │ 3. Output Stream\n                           ▼\n                    ┌──────────────┐\n                    │ Async Worker │ 4. Background `sync-turn`\n                    │  (no delay)  │───► Extracts new facts & persists\n                    └──────────────┘     into `memory.db` without blocking UI\n```\n\nTo keep the agent razor-sharp across thousands of daily turns without prompt bloat or massive token bills, memory is partitioned into 3 distinct layers:\n\n| Pillar | Type | Scope & Lifecycle | Storage Mechanism |\n|---|---|---|---|\nPillar 1 |\nEpisodic / Working Context |\nTransient day-to-day conversation, ephemeral tasks, session scratchpad. Dies after task completion or referenced via logs. | In-flight context window, Google Tasks, Transcripts |\nPillar 2 |\nSemantic / Long-Term Facts |\nDeterministic facts, server IPs, personal master data, credentials metadata, hardware specs, family profiles. Permanent & instantly searchable. | `~/.gemini/memory.db` (SQLite FTS5 + BM25) |\nPillar 3 |\nProcedural / Skills & Rules |\nHow to execute tasks: API definitions, security rules, playbooks, formatting standards. |\nSystem Rules (`user_global` ), AGY Skills |\n\nMost LLM memory solutions today suffer from two extremes:\n\n**Vector DB / Semantic RAG bloat:** Embedding models, heavy C++/native dependencies (Chroma, FAISS, PyTorch), slow cold-starts, vector drift, and poor keyword/exact match (e.g. failing to cleanly recall exact IP addresses, port numbers, serials, or drug doses).**Context-stuffing everything:** Relying on huge 1M–2M context windows adds massive latency, increases token costs exponentially, and dilutes the agent's attention on long-running multi-turn sessions.\n\n**Exact & Fast > Fuzzy Vectors for Core Facts:** When an agent needs your timezone, server IPs, hardware specs, or personal preferences, SQLite FTS5 with BM25 ranking delivers deterministic, exact results in**< 2ms** with**0 MB extra RAM**.** Telegram UX requires sub-second response starts:**SQLite FTS5 pre-fetches relevant facts locally via standard library Python before the LLM begins streaming.** Zero External Dependencies:**Built entirely on Python’s standard library (`sqlite3`\n\n,`re`\n\n,`difflib`\n\n). Runs anywhere without`pip install`\n\n, wheel compilation issues, or Docker container overhead.**Dynamic Pre-fetching without Prompt Bloat:** Instead of dumping an entire personal wiki into the system prompt,`agy-memory`\n\nextracts key entities, fetches only the 3–5 relevant facts, and injects them as ephemeral context.**Autonomous Background Learning & Compaction:** Ingesting new memories is decoupled from the user interaction (`sync-turn`\n\n). Over time, automated nightly compaction (`compact --apply`\n\n) deduplicates, resolves contradictions, and prunes stale data across all user accounts.\n\n**⚡ Blazing Fast Retrieval (<2ms):** Uses native SQLite FTS5 full-text indexing with BM25 ranking.**🔍 Typo & Fuzzy Fallback:** Automatically handles misspelled terms and queries via`difflib`\n\nvocabulary matching.**🌐 Multilingual & German/English Stopword Filtering:** Filters out noise and matches keywords cross-lingually.**🧠 Zero External Dependencies:** Pure Python 3 standard library (`sqlite3`\n\n,`re`\n\n,`difflib`\n\n,`argparse`\n\n,`json`\n\n,`subprocess`\n\n).**🔌 Dual Interface:****CLI:**`prefetch`\n\n,`sync-turn`\n\n,`add`\n\n,`list`\n\nfor shell scripts, cron jobs, and custom gateway integrations.**MCP Server:** Standard Model Context Protocol (`agy_memory_mcp.py`\n\n) exposing`search_memory`\n\n,`store_memory`\n\n, and`list_memories`\n\n.\n\n**⚙️ Configurable & Portable:** Resolves`agy`\n\nfrom`$PATH`\n\nautomatically; database and cache paths configurable via environment variables (`AGY_MEMORY_DB`\n\n,`AGY_MEMORY_CACHE`\n\n,`AGY_BIN`\n\n).\n\n- Python 3.10+ (standard library only, no\n`pip install`\n\nrequired) - Google Antigravity CLI (\n`agy`\n\n) installed in`$PATH`\n\nor`~/.local/bin/agy`\n\n(optional, needed for automated`sync-turn`\n\n)\n\n```\n# Add a fact\npython3 agy_memory.py add --id \"user.timezone\" --category \"preference\" --fact \"Timezone is UTC (CET/CEST)\" --keywords \"timezone zeit zeitzone time\"\n\n# List stored memories\npython3 agy_memory.py list\n\n# Query / Prefetch context for an upcoming prompt\npython3 agy_memory.py prefetch \"Wann beginnt das nächste Meeting?\"\n```\n\nPass user input and assistant response to extract and persist new facts asynchronously:\n\n```\npython3 agy_memory.py sync-turn --user \"Remember that our staging server IP changed to 192.168.1.150\" --assistant \"Understood, updated the staging IP.\"\n```\n\nOver time, continuous background learning (`sync-turn`\n\n) can accumulate overlapping facts, fragmented notes, or outdated states. The `compact`\n\ncommand serves as an automated knowledge curator.\n\n**Redundancy Elimination:** Merges scattered mentions of the same subject into single, dense canonical entries.**Contradiction & Drift Resolution:** Replaces superseded states (e.g. updated server IPs, new medication doses, changed configurations) while preserving current accuracy.**Zero Data Loss Guarantee:** Retains 100% of concrete details (exact dates, IDs, serial numbers, credentials, URLs).**Keyword Enrichment:** Generates fresh, multi-lingual search keywords (DE/EN synonyms and misspellings) to maximize FTS5 retrieval recall.**Database Optimization:** Rebuilds the FTS5 virtual table index and runs SQLite`VACUUM`\n\nto eliminate fragmentation.\n\n```\n# 1. Dry-run audit: Analyzes memories and displays a detailed diff/preview without writing changes\npython3 agy_memory.py compact\n\n# 2. Apply: Creates a timestamped backup in ~/.gemini/archive/, applies consolidations, and vacuums SQLite\npython3 agy_memory.py compact --apply\n```\n\nOn multi-user servers where multiple local users (e.g. family members or team members) run independent Antigravity instances, memory databases are isolated under each user's home directory (`~/.gemini/memory.db`\n\n).\n\nThe script [ scripts/agy-memory-compact-all.sh](/sbolten/agy-memory-engine/blob/main/scripts/agy-memory-compact-all.sh) automates maintenance across all users:\n\n**Auto-Discovery:** Scans`/home/*`\n\nfor active user accounts with an existing`~/.gemini/memory.db`\n\n.**Permission Isolation:** Executes the compaction strictly within each user's own permission boundary (`su - $username`\n\n), ensuring backups and DB files retain correct ownership (`0600`\n\n/`0700`\n\n).**Plug & Play for New Users:** Any newly created Linux user with an initialized memory database is automatically included without requiring manual configuration.\n\nCreate `/etc/cron.d/agy-memory-compact`\n\n:\n\n```\n# /etc/cron.d/agy-memory-compact\nSHELL=/bin/bash\nPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\nCRON_TZ=UTC\n\n# Run nightly at 04:00 AM before daily system backups\n0 4 * * * root /usr/local/bin/agy-memory-compact-all.sh >/dev/null 2>&1\n```\n\nTo connect the memory engine directly to AGY or any MCP-compatible agent, register the server in your MCP settings:\n\n```\n{\n  \"mcpServers\": {\n    \"memory\": {\n      \"command\": \"python3\",\n      \"args\": [\"/path/to/agy-memory-engine/agy_memory_mcp.py\"],\n      \"env\": {\n        \"AGY_MEMORY_DB\": \"~/.gemini/memory.db\"\n      }\n    }\n  }\n}\n```\n\nMIT License © 2026 Stephan Bolten", "url": "https://wpnews.pro/news/agy-memory-engine-zero-dependency-sqlite-fts5-fact-store-and-mcp-server", "canonical_source": "https://github.com/sbolten/agy-memory-engine", "published_at": "2026-08-21 15:24:06+00:00", "updated_at": "2026-08-21 15:44:23.403003+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "ai-agents", "machine-learning"], "entities": ["AGY Memory Engine", "Google Antigravity", "Hermes Agent", "SQLite FTS5", "BM25"], "alternates": {"html": "https://wpnews.pro/news/agy-memory-engine-zero-dependency-sqlite-fts5-fact-store-and-mcp-server", "markdown": "https://wpnews.pro/news/agy-memory-engine-zero-dependency-sqlite-fts5-fact-store-and-mcp-server.md", "text": "https://wpnews.pro/news/agy-memory-engine-zero-dependency-sqlite-fts5-fact-store-and-mcp-server.txt", "jsonld": "https://wpnews.pro/news/agy-memory-engine-zero-dependency-sqlite-fts5-fact-store-and-mcp-server.jsonld"}}