{"slug": "memarena-an-ego-centric-benchmark-for-on-device-agentic-personal-memory-at-scale", "title": "MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scale", "summary": "Researchers introduced MemArena, an ego-centric benchmark for on-device agentic personal memory assistants, built with the MASim agent simulator, simulating 50 agents over 15 days (10.3M dialog-text tokens, 24.1K text-only ego-observed tokens/agent/day). Evaluating five open-weight readers with various memory backends, they found that memory-backend choice matters more for content accuracy than reader scaling, with Memobase-to-MemSearch gains of +32.5/+19.2 pp at Qwen3-0.6B, and that permission-aware access fails universally. The benchmark and code will be released upon acceptance.", "body_md": "arXiv:2608.02613v1 Announce Type: new\nAbstract: Edge-deployed personal memory assistants must handle private interpersonal conversations on-device with open-weight models. Yet, existing memory benchmarks often under-test the combination of activity-dense interaction, ego-centric perspective, and coherent multi-session worlds. MemArena fills these gaps with a single-world conversational benchmark built with its MASim agent simulator, for 50 agents over 15 days (10.3M dialog-text tokens, 24.1K text-only ego-observed tokens/agent/day). With the interaction history, it co-generates ground truth over six recall, reasoning, and trustworthiness evaluation dimensions. We evaluate five open-weight readers with Vanilla context, BM25-RAG, Oracle retrieval, Memobase, and MemSearch as memory backends. Three results stand out: (1) Memory-backend choice matters more for content accuracy: At Qwen3-0.6B, Memobase-to-MemSearch gains +32.5/+19.2 pp, exceeding MemSearch reader scaling (+10.6/+6.8 pp). (2) Permission-aware access fails universally, with Oracle leaking heavily and other backends too timid to disclose. (3) Search latency bites only at very small reader: on a Spark GB10 edge node, memory-search adds a moderate and fixed 87/7/48 ms (BM25-RAG/Memobase/MemSearch) that composes a small part of TTFT for most reader-backend combinations. Code, the MASim simulator, and the MemArena-L benchmark will be released upon acceptance.", "url": "https://wpnews.pro/news/memarena-an-ego-centric-benchmark-for-on-device-agentic-personal-memory-at-scale", "canonical_source": "https://arxiv.org/abs/2608.02613", "published_at": "2026-08-05 04:00:00+00:00", "updated_at": "2026-08-05 04:02:57.547381+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-agents", "ai-infrastructure"], "entities": ["MemArena", "MASim", "Qwen3-0.6B", "Memobase", "MemSearch", "BM25-RAG", "Spark GB10", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/memarena-an-ego-centric-benchmark-for-on-device-agentic-personal-memory-at-scale", "markdown": "https://wpnews.pro/news/memarena-an-ego-centric-benchmark-for-on-device-agentic-personal-memory-at-scale.md", "text": "https://wpnews.pro/news/memarena-an-ego-centric-benchmark-for-on-device-agentic-personal-memory-at-scale.txt", "jsonld": "https://wpnews.pro/news/memarena-an-ego-centric-benchmark-for-on-device-agentic-personal-memory-at-scale.jsonld"}}