{"slug": "my-custom-memory-graph-actually-beat-memora-on-the-locomo", "title": "My custom memory graph actually beat Memora on the LoCoMo", "summary": "Recordari, a custom memory graph built by developer Corby, scored 0.831 on the LoCoMo benchmark, outperforming Microsoft Research's Memora, which published a score of 0.801. The benchmark, run with the same model and dataset, also showed Recordari achieving 0.869 on multi-session questions versus a 0.760 raw storage baseline. The results are public on GitHub, and the tool works with Claude Code, ChatGPT, and other MCP-compatible agents.", "body_md": "# My custom memory graph actually beat Memora on the LoCoMo\n\n`.md`\n\nfiles or basic [MCP](/en/tags/mcp/)tools to track state, but flat files are too primitive. If you're just using a\n\n`claude.md`\n\nfile and you're happy with it, you don't need a complex system. But once your project scales, you realize you need a knowledge graph that can actually handle contradictions and versioning.I started by building a tool called memoryweb to handle my own workflow. It worked well for a solo loop—recording problems and fixes as \"stories\"—but it fell apart the moment I thought about multi-tenancy. When multiple users are feeding an agent information, you run into a mess: conflicting decisions get written to the same file, stale information lingers, and you lose the \"why\" behind a specific architectural choice.\n\nTo solve this, I built Recordari. Instead of just storing text, it uses typed memories and typed edges. This allows the agent to explicitly label nodes as \"contradicts\" or \"resolved.\" The server can flag semantically similar memories that might conflict, and the agent then has to adjudicate which one is correct. It also tracks ownership, so an agent doesn't just blindly overwrite a human's decision.\n\nI decided to see how this stacked up against the industry standards using the open-source benchmark harness from Microsoft Research's Memora (ICML 2026). I kept everything strictly controlled: same model, same LoCoMo dataset, and the same category-exclusion rules.\n\nThe results were surprising:\n\n**Recordari end-to-end score:** 0.831**Memora published score:** 0.801**Multi-session questions (Recordari):** 0.869**Raw storage baseline:** 0.760\n\nWhat's important here is that these aren't mocked results. The harness runs against the production API—the exact MCP search and recall API a real agent uses—with a\n\n`top_k=30`\n\nconfiguration and a 1-hop edge expansion. This setup works across the board with [Claude Code](/en/tags/claude%20code/), ChatGPT, and any other LLM agent that supports MCP. If you're looking for a real-world AI workflow that doesn't suffer from \"amnesia\" across sessions, moving from flat files to a structured graph is the way to go.\n\nIf you want to check out the benchmark data, the results are public here:\n\n```\nhttps://github.com/corbym/locomo-recordari\n```\n\nAnd for those who want to test an agent against it using the LoCoMo benchmark themselves, you can use the anonymous login here:\n\n```\nhttps://admin.recordar.io/start\n```\n\n[Next PyTorch .backward() is just a graph traversal in disguise →](/en/threads/6087/)", "url": "https://wpnews.pro/news/my-custom-memory-graph-actually-beat-memora-on-the-locomo", "canonical_source": "https://promptcube3.com/en/threads/6191/", "published_at": "2026-08-13 16:13:59+00:00", "updated_at": "2026-08-13 16:25:35.007027+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-research", "ai-tools"], "entities": ["Recordari", "Microsoft Research", "Memora", "LoCoMo", "Claude Code", "ChatGPT", "MCP", "Corby"], "alternates": {"html": "https://wpnews.pro/news/my-custom-memory-graph-actually-beat-memora-on-the-locomo", "markdown": "https://wpnews.pro/news/my-custom-memory-graph-actually-beat-memora-on-the-locomo.md", "text": "https://wpnews.pro/news/my-custom-memory-graph-actually-beat-memora-on-the-locomo.txt", "jsonld": "https://wpnews.pro/news/my-custom-memory-graph-actually-beat-memora-on-the-locomo.jsonld"}}