{"slug": "your-agent-gets-dumber-the-longer-it-runs-heres-how-a-graph-fixes-it", "title": "Your Agent Gets Dumber the Longer It Runs. Here’s How a Graph Fixes It.", "summary": "A graph-based memory layer outperforms flat vector indexes on long-running agent tasks, according to a talk titled \"Memory Engineering for Long-Running AI Agents\" delivered at a16z's Agent Memory Night on October 8, where the speaker cited LongMemEval results showing a graph index answers 89% of questions correctly versus 76% for a flat vector index. The same talk reported that GPT-5.5 accuracy on Graphwalks BFS falls from 74% at 256k tokens to 45% at 1 million tokens, and that FalkorDB held the slowest 1% of reads at 0.5 to 1 ms at 1 million edges with 20 concurrent agents versus 14 to 36 ms for Neo4j. The speaker also cited the MemLeak study (Mudgal et al., 2026), which found 7 in 10 queries returned another user's memory from a shared store, a bug shipped as CVE-2026-41712 in Spring AI.", "body_md": "On October 8 we hosted Agent Memory Night, an a16z #SFTechWeek event at the AWS Builder Loft in San Francisco, with **652 RSVPs**. We built the night for engineers shipping long-running agents, and we asked every speaker to answer one real builder complaint. My talk was **Memory Engineering for Long-Running AI Agents**.\n\nThink about how much you feed an agent today. Slack, Gmail, tickets, your codebase. Each source holds one piece of the picture. The agent has to connect the dots, figure out what is still true, and keep it for next week. That is a memory problem, and it gets harder every hour the agent runs.\n\nHere is the full talk, then the written version below.\n\n## Memory is the SSD\n\nAndrej Karpathy has an analogy I love. The model is the CPU, the context window is the RAM, and memory is the disk. Each part has its own job, and the first two land perfectly.\n\nI'd change the third one. For agents, the disk is too slow. If there is something faster and more efficient for an agent to work with, we should use it. Memory should be the SSD.\n\n## The four kinds of agent memory\n\nWorking memory sits at the top. That's the system prompt, skills, and tool outputs, usually held in the KV cache. Episodic memory comes next, with recent messages and tool logs in an in-memory key-value store.\n\nBelow that is long-term memory, and it comes in two kinds. Semantic memory holds the facts and user preferences an agent should remember across sessions. Procedural memory holds workflows and rules.\n\nA memory layer like [Cognee](https://www.cognee.ai), [Graphiti](https://github.com/getzep/graphiti), or [Mem0](https://mem0.ai) decides what the agent keeps. Underneath it sits the data store, and that's where FalkorDB comes in. One engine gives you graph, vector, and full-text retrieval in a single query. Every user can get their own graph, because FalkorDB is multi-tenant by design. Reads also never wait for writes to finish.\n\n## Where agent memory breaks\n\nAs memory grows, four things go wrong. Here is each one, with the number I showed on stage.\n\n### Accuracy drops\n\nEven the best models struggle once memory gets big. In the Graphwalks BFS results OpenAI published with GPT-5.5, GPT-5.5 gets 74% right at 256k tokens and 45% at 1 million. Claude Opus drops from 77% to 41%. GPT-5.4 falls from 63% to 9%.\n\nA graph helps. On [LongMemEval](https://github.com/xiaowu0162/LongMemEval), a graph index gets 89% of answers right and a flat vector index gets 76% (Hu et al., \"Does Memory Need Graphs?\", ACL 2026). On [GraphRAG-Bench](https://github.com/GraphRAG-Bench), the FalkorDB GraphRAG SDK gets 29% more right answers than vector RAG with the same model and data.\n\n### Latency keeps climbing\n\nLong-running means the history piles up. In my own test, an agent that re-reads its full history every turn starts at about 2 seconds per turn. After 100 messages it takes 10 to 17 seconds. Every new message slows down every turn after it. Nobody waits 17 seconds for an agent.\n\nFalkorDB is built for agent workloads. At 1 million edges with 20 agents writing to the same graph, the slowest 1% of FalkorDB reads stay at 0.5 to 1 ms. Neo4j, on the same data and queries, sits at 14 to 36 ms. Deleting one user's memory takes 0.3 to 0.5 ms on FalkorDB and 150 to 190 ms on Neo4j.\n\n### Users see each other's memory\n\nA lot of teams put every user's memory in one shared store and filter by user ID. The MemLeak study (Mudgal et al., 2026) found that with one shared store and normal similarity search, 7 in 10 queries brought back another user's memory. No attack needed. Spring AI shipped this exact bug as CVE-2026-41712.\n\nFalkorDB gives each user their own graph instead. One customer runs 81,500 graphs in a single production cluster, one per user. Here's what that looks like with the Python client.\n\n``` python\nfrom falkordb import FalkorDB\n\ndb = FalkorDB(host=\"localhost\", port=6379)\nuser_id = \"alice\"\n\n# one graph per user\nmemory = db.select_graph(f\"memory_{user_id}\")\n\n# store a fact, safe to run twice\nmemory.query(\n    \"MERGE (u:User {id: $uid}) \"\n    \"MERGE (p:Preference {name: $name}) \"\n    \"MERGE (u)-[:PREFERS]->(p)\",\n    {\"uid\": user_id, \"name\": \"window seat\"},\n)\n\n# read-only lookup\nresult = memory.ro_query(\n    \"MATCH (u:User {id: $uid})-[:PREFERS]->(p) RETURN p.name\",\n    {\"uid\": user_id},\n)\n\n# user leaves, drop their graph\nmemory.delete()\n```\n\n### Token bills explode\n\nIn my test, one inbox of 200 messages plus 25 questions used 20 million tokens when the agent re-read its full history every turn. That's $200 at list price for one inbox, or $2M for 10,000 users. With graph memory the same run used 4.4 million tokens. That's about 4.5x fewer, with prompt caching on for both.\n\n## Back to the analogy\n\nKarpathy said memory is the disk. Most teams build that disk with flat vector search today, and the numbers above show where it breaks. A graph stores the facts and how they connect, so the agent reads less and gets more right. That's why I call it the SSD.\n\n## How to build agent memory that lasts\n\nHere's the recipe I'd follow. Most of it works on any database, and I'll point out where FalkorDB makes it easier.\n\n### 1. Split memory into two loops\n\nThe slow loop runs in the background. Each new message goes through a small LLM call that pulls out the facts and saves them. In my test that cost $0.31 for 200 messages.\n\nThe fast loop runs on every turn. The agent sends one query and gets back only the facts it needs, and only those go into the context window. For 25 questions that took 0.12 seconds of query time in total. Context is the RAM, memory is the SSD, and memory engineering is deciding what each loop keeps.\n\n### 2. Close old facts instead of appending new text\n\nFacts change. Tom hands the Payments on-call shift to Emma. If you just append, the agent holds both and has to guess which one is true. Instead, end Tom's shift and add Emma's. You keep the history, and \"who's on call now?\" has one answer.\n\n```\n// Tom's shift ends. Close the old fact, keep it for history.\nMATCH (s:Entity {name: 'Payments'})-[r:ON_CALL]->(:Person {name: 'Tom'})\nWHERE r.valid_to IS NULL\nSET r.valid_to = timestamp()\nMERGE (e:Person {name: 'Emma'})\nCREATE (s)-[:ON_CALL {valid_from: timestamp()}]->(e)\n```\n\nOn GraphRAG-Bench, graph scored 71.5 and vector scored 55.4. Closed facts beat similar-looking text.\n\n### 3. Retrieve in one query\n\nA good read does three things. Vector and full-text search find where to start, like the Payments node. Then it follows relationships from there. Then it keeps only the facts that are still true.\n\n``` php\nMATCH (:Entity {name: 'Payments'})-[r:ON_CALL]->(p)\nWHERE r.valid_to IS NULL\nRETURN p.name\n```\n\nThe answer is Emma, plus the fact that closed Tom's shift. That small slice is all that goes into the context window. FalkorDB runs all three steps in one engine, so there's no second system to keep in sync. On one laptop with 10,000 graphs, a 3-hop read took 0.6 to 0.7 ms, and 0.9 to 1.3 ms with 8 agents writing.\n\n### 4. Give every user their own graph\n\nDon't put everyone in one store and filter by user ID. That's how 7 in 10 queries ended up returning someone else's memory. With one graph per user there's no shared index to filter and nothing to leak. Deleting a user means dropping one graph, which takes 0.3 ms in FalkorDB.\n\n### 5. Watch the write side, because it sets the ceiling\n\nThe fast loop can only find what the slow loop saved. In my benchmark, one service got saved under two names, and that broke a dependency chain for graph and SQL alike. Merge duplicate names into one node before you save facts.\n\nAt 200 messages, every setup except vector scored 25 out of 25. At 400, graph, vector and compaction all scored 22, and every setup missed the same chain. A graph doesn't fix bad extraction. It makes good extraction fast, cheap and private.\n\n## Rerun the numbers yourself\n\nThe latency, write and cost numbers come from [agent-memory-bench](https://github.com/zhang-liz/agent-memory-bench), a benchmark I open-sourced. It gives one agent six memory setups (full history, compaction, a memory tool, vector, SQLite and graph). The model, prompts and tools stay the same. Only memory changes.\n\n**Fair by design.** Every engine gets its best setup, and prompt caching is on everywhere. The benchmark also shows where graph memory loses, like questions that need an exact quote from raw text. If you find a setup that's unfair to any engine,\n\n[open an issue](https://github.com/zhang-liz/agent-memory-bench/issues).\n\nYou can start FalkorDB with one Docker command.\n\n```\ndocker run -p 6379:6379 -p 3000:3000 -it --rm falkordb/falkordb:latest\n```\n\nVasilije, Cognee's founder, spoke later that night with a deep dive on how to structure memory, so the two talks pair well. I'd love to hear what you build.\n\n## Try it\n\nFalkorDB is open source and starts with one Docker command. It works with LangChain and LlamaIndex, and with memory frameworks like Cognee, Graphiti, and Mem0. Give your agent one graph per user and watch what happens to accuracy, latency, and cost.\n\n[Get started with FalkorDB](https://docs.falkordb.com/#get-started)\n\n### Thank you\n\nThanks to AWS for hosting us at the Builder Loft, to GMI Cloud and The Multimodal Society for co-hosting, and to everyone who spent a Thursday night talking about agent memory.\n\n*Each number comes from the source named on its slide or from agent-memory-bench runs on one laptop. Your runs will differ.*\n\n### Sources\n\n[agent-memory-bench](https://github.com/zhang-liz/agent-memory-bench) · [FalkorDB on GitHub](https://github.com/FalkorDB/FalkorDB) · [FalkorDB concurrency design](https://docs.falkordb.com/design/concurrency) · [LongMemEval](https://github.com/xiaowu0162/LongMemEval) · [GraphRAG-Bench](https://github.com/GraphRAG-Bench) · [Cognee](https://www.cognee.ai) · [Graphiti](https://github.com/getzep/graphiti) · [Mem0](https://mem0.ai)", "url": "https://wpnews.pro/news/your-agent-gets-dumber-the-longer-it-runs-heres-how-a-graph-fixes-it", "canonical_source": "https://www.falkordb.com/blog/agent-memory-for-long-running-agents/", "published_at": "2026-10-16 10:09:53+00:00", "updated_at": "2026-10-16 10:41:57.824427+00:00", "lang": "en", "topics": ["ai-agents", "artificial-intelligence", "ai-infrastructure", "large-language-models", "ai-safety"], "entities": ["FalkorDB", "Neo4j", "Cognee", "Graphiti", "Mem0", "Andrej Karpathy", "Spring AI", "a16z"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/your-agent-gets-dumber-the-longer-it-runs-heres-how-a-graph-fixes-it", "markdown": "https://wpnews.pro/news/your-agent-gets-dumber-the-longer-it-runs-heres-how-a-graph-fixes-it.md", "text": "https://wpnews.pro/news/your-agent-gets-dumber-the-longer-it-runs-heres-how-a-graph-fixes-it.txt", "jsonld": "https://wpnews.pro/news/your-agent-gets-dumber-the-longer-it-runs-heres-how-a-graph-fixes-it.jsonld"}}