cd /news/ai-agents/how-are-you-measuring-your-agent-s-m… · home › topics › ai-agents › article
[ARTICLE · art-152570] src=memstrata.dev ↗ pub= topic=ai-agents verified=true sentiment=· neutral

How are you measuring your agent's memory layer?

MemStrata published author-run, development-exposed research reporting 92.6% (463/500) on LongMemEval-S and 78.25% (1205/1540) on LoCoMo for its memory layer, graded by a single automated judge model, GPT 5.5, with sources visible to the judge. The company states the results are not independent replication or a held-out customer evaluation and are not new reader runs or product guarantees; the research reader was Qwen 3.8 27B Q4, a model the product does not include. MemStrata keeps chats, documents and project history in an on-device index, and is offered free for 180 days after activation on Windows, Linux and Apple silicon.

by read2 min views2 publishedOct 19, 2026
How are you measuring your agent's memory layer?
Image: source

A changed decision

Illustrative example When is the project review?

Friday.

The Tuesday note is still in the record. A later note moves the review to Friday.

MemStrata keeps your chats, documents and project history in an index on your own computer. Ask a question and see the exact passages behind the answer.

Free for 180 days after activation. Windows, Linux and Apple silicon.

Measured in the open

Author-run, development-exposed research using one automated judge model, GPT 5.5—not independent replication or a held-out customer evaluation. The source-aware headlines regrade the same answers with sources visible to the judge. Their reference-aware scores are 92.6% (463/500) on LongMemEval-S and 78.25% (1205/1540) on LoCoMo. These are not new reader runs or product guarantees. Compare the grading methods.

The research reader was Qwen 3.8 27B Q4. The product does not include that model; you connect your own AI.

How it works

Prepare private memory, then add chats, notes or documents.

Setup lists the reader used for Standard and Deep.

Ask in the chat. Standard and Deep use the same index. Open Why this answer and read the passages behind the answer.

Indexes and source processing run on your device. Account, licensing, and optional connectors use MemStrata or destination services. If you choose a cloud answering model, the question and selected passages leave the device for that provider. Credentials stay in the operating-system credential vault. Customer content is not used for training unless the customer separately opts in.

See the idea

A deadline moves. A count spans many conversations. A detail was never mentioned. Step through three short examples and open the sources behind each answer.

Four products

Conversational, Coder, Structured and Governance each do one job well. Start with the one that matches your work.

Ask about imported documents and chats with inspectable evidence

Supply relevant repository context to a supported coding tool

Answer supported current, history and change questions over records

Observe connected agent actions and use supported controls

Read our published research on arXiv. Every benchmark result on this site keeps its denominator, judge and method label. Research results, measured under the stated conditions.

Install Conversational, connect a folder of notes or an exported chat, and open the evidence behind your first answer.

── more in #ai-agents 4 stories · sorted by recency
── more on @memstrata 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/how-are-you-measurin…] indexed:0 read:2min 2026-10-19 · —