{"slug": "jordy-zomer-built-a-datalog-engine-so-ai-agents-can-forget-correctly", "title": "Jordy Zomer built a Datalog engine so AI agents can forget correctly", "summary": "Security researcher Jordy Zomer released Lemmalog, a Datalog engine that lets AI agents maintain and retract facts with provenance, on August 28th. The open-source tool, published under the MIT license, separates fuzzy LLM extraction from deterministic reasoning, enabling agents to discard invalid assumptions without re-prompting the model. Early benchmarks show promise but also gaps, according to Zomer's technical account.", "body_md": "# Jordy Zomer built a Datalog engine so AI agents can forget correctly\n\n**Lemmalog separates fuzzy LLM extraction from deterministic facts, provenance and retractions, with early benchmarks showing promise and gaps.**\n\nBy [RuntimeWire Staff](/author/runtimewire-staff)\n· Published\n\nPrimary source: [pwning.systems](https://pwning.systems/posts/llm-memory-program-analysis/)\n\n## Why it matters\n\nLemmalog moves agent memory from transcript retrieval toward auditable state maintenance, giving developers a way to retract bad assumptions without asking an LLM to reconstruct every dependency.\n\nSecurity researcher [Jordy Zomer](https://pwning.systems/about/?ref=runtimewire) published the design and source code for [Lemmalog](https://github.com/JordyZomer/lemmalog?ref=runtimewire) on August 28th, turning a recurring failure in long-running AI-assisted vulnerability research into a Datalog engine that maintains what an agent currently knows.\n\nZomer started with a practical annoyance. After several hours inside a large codebase, an LLM agent would revive discarded exploit paths, forget that an assumption had failed or continue reasoning from an observation invalidated in a debugger. Retrieval could bring the old notes back, but it left the model to reconcile which version remained true.\n\n\"I want it to maintain what we currently know,\" Zomer wrote in [his August 28th technical account](https://pwning.systems/posts/llm-memory-program-analysis/?ref=runtimewire).\n\nThat distinction led Zomer back to the tools he already knew. His security work spans static analysis, fuzzing, reverse engineering, binary exploitation and open-source vulnerability research. Ubuntu has also credited Zomer and Alexandra Sandulescu with discovering two Linux kernel flaws involving speculative-execution barriers, [CVE-2023-0458 and CVE-2023-0459](https://ubuntu.com/security/notices/USN-6096-1?ref=runtimewire).\n\nLemmalog applies the same discipline used in program analysis: record facts, derive conclusions through explicit rules, track their dependencies and update only the affected results when an input changes. The model still interprets source code, debugger output and natural-language notes. Once those observations become structured facts, the deterministic engine takes over.\n\n### Memory becomes maintained state\n\nZomer's example begins with three observations: an attacker controls one object, that object points to a second object, and the second object is a kernel object. A Datalog rule can derive that the attacker controls a kernel object.\n\nIf later debugging disproves the pointer relationship, Lemmalog retracts conclusions that depended on it. A conclusion supported by a second, independent path remains valid. The engine can also produce a provenance tree showing which observations and rules support a result.\n\nThat gives an agent a harder boundary around unsupported claims. The extraction model can still misunderstand evidence or encode a bad fact, a limitation Zomer states directly. Lemmalog makes the resulting state inspectable and gives corrections mechanical consequences once the error is identified.\n\nTemporal facts preserve a history of those changes. An exploit primitive can be recorded as viable during one interval and invalid after a later observation, allowing the agent to answer both what is believed now and why an earlier path was explored.\n\nThe repository, published under the MIT license, includes incremental evaluation, retractions, temporal facts, aggregation, entity resolution, persistence, hybrid retrieval and an MCP server for connecting Lemmalog to agent tools including Claude Code and Kimi CLI.\n\nThe architecture still uses semantic retrieval. Zomer combines structured facts and dependency-aware reasoning with the original source text, embeddings and lexical search. His point is narrower than replacing vector databases: similarity search answers which old information appears relevant, while the deductive state answers which conclusions remain supported.\n\n### The benchmarks support an experiment, not a victory lap\n\nZomer tested Lemmalog on LongMemEval and LoCoMo, two conversational-memory benchmarks rather than vulnerability investigations, as described in [his technical account](https://pwning.systems/posts/llm-memory-program-analysis/?ref=runtimewire). He reports a LongMemEval F1 score of 0.463 +/- 0.010 across three runs, behind the published PropMem result of 0.550 and slightly below SimpleMem at 0.480.\n\nThe strongest LongMemEval category reported in the research material was knowledge updates, where Lemmalog scored 0.579 in a representative run. That category most closely matches Zomer's original problem: deciding what remains true after an earlier belief changes.\n\nMulti-session performance exposed the weak point. Lemmalog scored 0.211, compared with 0.582 for PropMem. Zomer traced many misses to extraction failures. A perfect rule engine cannot reason over an event that the LLM never converted into a fact.\n\nOn LoCoMo, Zomer reports 0.533 +/- 0.001 F1 across three runs. The project also struggled with some inference tasks, illustrating how compressing a conditional preference into a simple tuple can discard the qualification that matters.\n\nAll of those results are project-reported and have not been independently validated. Zomer also cautions that the benchmarks do not reproduce a real vulnerability investigation.\n\nThe efficiency result is still notable. Zomer says the LongMemEval answering model received roughly 2,700 tokens per question through Lemmalog, compared with about 104,000 tokens under full-context prompting. Extraction carries its own cost, paid when new material enters the system. Repeated queries can then operate on maintained state instead of rereading the full transcript each time.\n\n### A different bet in a funded memory market\n\nAgent memory has already become an infrastructure category with venture financing behind it. [Mem0 said it raised $24 million](https://mem0.ai/series-a?ref=runtimewire) across seed and Series A rounds in October 2025. [Cognee announced a $7.5 million seed round](https://www.cognee.ai/cognee-raises-seven-million-five-hundred-thousand-dollars-seed?ref=runtimewire) on February 19th, 2026, for an engine combining graph, vector and relational storage.\n\nOther builders are also moving beyond transcript retrieval. [Letta's Context Repositories](https://www.letta.com/blog/context-repositories/?ref=runtimewire) use Git-backed files to version changes to agent memory, while [Weaviate's Engram](https://weaviate.io/blog/engram-generally-available?ref=runtimewire) is pitched as a managed context service that reconciles evolving facts.\n\nZomer's contribution comes from the security-research end of the problem. Lemmalog treats an agent's beliefs like an analysis artifact whose assumptions, derivations and invalidations should be visible. That approach is particularly suited to work where a stale claim can waste hours or corrupt every conclusion downstream.\n\nThe decisive test remains the one Zomer set for himself: run an agent through a long, complicated vulnerability investigation and measure whether maintained state stops dead hypotheses from returning. The current benchmarks put numbers around where the experiment performs well and where extraction and inference remain weak. They also show how much depends on the probabilistic front end correctly reading the evidence in the first place.", "url": "https://wpnews.pro/news/jordy-zomer-built-a-datalog-engine-so-ai-agents-can-forget-correctly", "canonical_source": "https://runtimewire.com/article/jordy-zomer-lemmalog-datalog-memory-llm-agents", "published_at": "2026-08-29 00:53:20+00:00", "updated_at": "2026-08-29 01:17:55.382251+00:00", "lang": "en", "topics": ["ai-agents", "ai-research", "developer-tools", "machine-learning"], "entities": ["Jordy Zomer", "Lemmalog", "Claude Code", "Kimi CLI", "MIT", "Ubuntu", "Alexandra Sandulescu"], "alternates": {"html": "https://wpnews.pro/news/jordy-zomer-built-a-datalog-engine-so-ai-agents-can-forget-correctly", "markdown": "https://wpnews.pro/news/jordy-zomer-built-a-datalog-engine-so-ai-agents-can-forget-correctly.md", "text": "https://wpnews.pro/news/jordy-zomer-built-a-datalog-engine-so-ai-agents-can-forget-correctly.txt", "jsonld": "https://wpnews.pro/news/jordy-zomer-built-a-datalog-engine-so-ai-agents-can-forget-correctly.jsonld"}}