{"slug": "how-are-you-measuring-your-agent-s-memory-layer", "title": "How are you measuring your agent's memory layer?", "summary": "MemStrata published author-run, development-exposed research reporting 92.6% (463/500) on LongMemEval-S and 78.25% (1205/1540) on LoCoMo for its memory layer, graded by a single automated judge model, GPT 5.5, with sources visible to the judge. The company states the results are not independent replication or a held-out customer evaluation and are not new reader runs or product guarantees; the research reader was Qwen 3.8 27B Q4, a model the product does not include. MemStrata keeps chats, documents and project history in an on-device index, and is offered free for 180 days after activation on Windows, Linux and Apple silicon.", "body_md": "### A changed decision\n\nIllustrative example\nWhen is the project review?\n\nFriday.\n\nThe Tuesday note is still in the record. A later note moves the review to Friday.\n\nMemStrata keeps your chats, documents and project history in an index on your own computer. Ask a question and see the exact passages behind the answer.\n\n[Free for 180 days after activation. Windows, Linux and Apple silicon.](https://www.memstrata.dev/pricing)\n\nMeasured in the open\n\nAuthor-run, development-exposed research using one automated judge model, GPT 5.5—not independent replication or a held-out customer evaluation. The source-aware headlines regrade the same answers with sources visible to the judge. Their reference-aware scores are 92.6% (463/500) on LongMemEval-S and 78.25% (1205/1540) on LoCoMo. These are not new reader runs or product guarantees. [Compare the grading methods](https://www.memstrata.dev/research/methods#cl1-source-aware-regrades).\n\nThe research reader was Qwen 3.8 27B Q4. The product does not include that model; you connect your own AI.\n\nHow it works\n\nPrepare private memory, then add chats, notes or documents.\n\nSetup lists the reader used for Standard and Deep.\n\nAsk in the chat. Standard and Deep use the same index. Open Why this answer and read the passages behind the answer.\n\nIndexes and source processing run on your device. Account, licensing, and optional connectors use MemStrata or destination services. If you choose a cloud answering model, the question and selected passages leave the device for that provider. Credentials stay in the operating-system credential vault. Customer content is not used for training unless the customer separately opts in.\n\nSee the idea\n\nA deadline moves. A count spans many conversations. A detail was never mentioned. Step through three short examples and open the sources behind each answer.\n\nFour products\n\nConversational, Coder, Structured and Governance each do one job well. Start with the one that matches your work.\n\nAsk about imported documents and chats with inspectable evidence\n\nSupply relevant repository context to a supported coding tool\n\nAnswer supported current, history and change questions over records\n\nObserve connected agent actions and use supported controls\n\nRead our published research on arXiv. Every benchmark result on this site keeps its denominator, judge and method label. Research results, measured under the stated conditions.\n\nInstall Conversational, connect a folder of notes or an exported chat, and open the evidence behind your first answer.", "url": "https://wpnews.pro/news/how-are-you-measuring-your-agent-s-memory-layer", "canonical_source": "https://www.memstrata.dev/", "published_at": "2026-10-19 00:32:10+00:00", "updated_at": "2026-10-19 00:48:37.545682+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-products"], "entities": ["MemStrata", "LongMemEval-S", "LoCoMo", "GPT 5.5", "Qwen 3.8 27B Q4", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-are-you-measuring-your-agent-s-memory-layer", "markdown": "https://wpnews.pro/news/how-are-you-measuring-your-agent-s-memory-layer.md", "text": "https://wpnews.pro/news/how-are-you-measuring-your-agent-s-memory-layer.txt", "jsonld": "https://wpnews.pro/news/how-are-you-measuring-your-agent-s-memory-layer.jsonld"}}