{"slug": "frontiermemory", "title": "FrontierMemory", "summary": "Letta released FrontierMemory, a benchmark it calls the first to score memory generation by how much it helps an agent solve downstream tasks rather than by recall of past facts. In Letta's leaderboard, Opus 5.5 [max] and GLM-5.3 [max] tied at 73.3% at task costs of $2.80 and $3.58 respectively, ahead of Kimi K3 [max] at 66.7%, GPT-6.1 Sol [max] and Gemini 4 Argon [high] at 60.0%, Muse Wonder [xhigh] at 46.7% and Grok 4.7 [xhigh] at 33.3%. Among harnesses, Letta Dream ranked first at 73.3%, followed by Prime Agent at 60.0%, OpenAI Agents SDK at 40.0%, Claude Managed Agents at 33.3% and Hermes Agent at 26.7%, with memory generation capped at 6 hours and no web browsing allowed.", "body_md": "# FrontierMemory\n\n[by](https://www.letta.com/)\n\nBenchmarking continual learning through memory.\n\n## Models\n\nWhich models are the best at memory generation?\n\n| Rank | Model | Score |  | Task cost | \n|---|---|---|---|---|\n| 1 | Opus 5.5 [max] | 73.3% |  | $2.80 | \n| 2 | GLM-5.3 [max] | 73.3% |  | $3.58 | \n| 3 | Kimi K3 [max] | 66.7% |  | $3.73 | \n| 4 | GPT-6.1 Sol [max] | 60.0% |  | $0.77 | \n| 5 | Gemini 4 Argon [high] | 60.0% |  | $2.60 | \n| 6 | Muse Wonder [xhigh] | 46.7% |  | $4.09 | \n| 7 | Grok 4.7 [xhigh] | 33.3% |  | $3.02 | \n\n- GPT-6.1 Sol\n- Opus 5.5\n- Muse Wonder\n- Gemini 4 Argon\n- GLM-5.3\n- Kimi K3\n- Grok 4.7\n\n- Task effort:\n- low\n- medium\n- high\n- xhigh\n- max\n\n## Harnesses\n\nWhich harnesses are the best at memory generation?\n\n| Rank | Harness | Score |  | Task cost | \n|---|---|---|---|---|\n| 1 | Letta Dream | 73.3% |  | $2.80 | \n| 2 | Prime Agent | 60.0% |  | $3.74 | \n| 3 | OpenAI Agents SDK | 40.0% |  | $0.82 | \n| 4 | Claude Managed Agents | 33.3% |  | $3.21 | \n| 5 | Hermes Agent | 26.7% |  | $2.79 | \n\n- Letta Dream\n- OpenAI Agents SDK\n- Hermes Agent\n- Prime Agent\n- Claude Managed Agents\n\n- Task effort:\n- low\n- medium\n- high\n- xhigh\n- max\n\n## The first memory-native benchmark\n\nExisting memory benchmarks measure recall of past facts. **FrontierMemory** is the first benchmark that measures the quality of memory. Agents are given access to trajectories of past experience and tasked to generate memory, which is then scored by how much it helps an agent solve downstream tasks.\n\n## About FrontierMemory\n\n1. \nAgents are given access to trajectories of agent experience and generate memory.\n2. \nMemory is evaluated based on how much it helps solve downstream tasks.\n\n## Task sources\n\nWe select challenging coding and knowledge-work tasks from public benchmarks, including HarborIndex, DeepSWE, and SkillsBench, both for source transcripts and for downstream tasks. Some tasks are modified so solving them requires knowledge available only in the source transcripts.\n\n## Sample tasks\n\n[See all tasks](https://frontiermemory.com/tasks/)\n\n```\nDeepSWEAwilix Release Tier (Implicit)\nAdd async initialization to Awilix, where the version bump is determined by applying a policy learned from earlier tasks to a release tier found in project metadata.\n\nBigCodeBenchWord2Vec Pipeline\nBuild a Word2Vec component whose interface, preprocessing, and output contract must be inferred from previous runs.\n\nSkillsBenchNetwork Intrusion Analysis\nCompute DAPT2020 network statistics using lessons from failed analyses about counting, entropy, and threat detection.\n```\n\n## Questions and answers\n\n## What is the final metric used for the leaderboard?\n\nEach memory-generation harness and model is ranked by the higher downstream task solve rate of Opus 5.5 and GPT-6 Sol, both at max reasoning. When only one task model is available, its score is used.\n\n## What is the time limit for memory generation?\n\nMemory generation is limited to 6 hours.\n\n## How does FrontierMemory differ from other memory benchmarks?\n\nExisting memory benchmarks mostly measure recall of past facts. FrontierMemory is the first benchmark to measure memory by how much it helps an agent solve real tasks.\n\n## What does the agent see during memory generation?\n\nThe agent sees the source transcripts. These cover the downstream tasks as well as other, unrelated tasks, and each transcript is labeled with whether the attempt succeeded or failed.\n\n## Are agents allowed to use the internet?\n\nNo. Web browsing and online solution lookup are not allowed during memory generation or task execution. Agents work from the provided trajectories and task environment; model API calls remain available.\n\n## What goes in a submission?\n\nFrontierMemory primarily compares models on how well they generate memory. By default, each model generates memory in the Letta Dream harness and is evaluated in Letta Code. Submissions can also bring their own memory generation and task harnesses, to maximize compatibility and allow harness comparisons.\n\n## Submit an agent harness or model\n\nTo submit a harness, it must have a memory system it uses to create memories or skills between sessions.\n\n## Sign up for leaderboard updates\n\nGet notified when new harnesses and models are added to FrontierMemory. To reach our team, email [\\[email protected\\]](https://frontiermemory.com/cdn-cgi/l/email-protection#0e7c6b7d6b6f7c6d664e626b7a7a6f206d6163).", "url": "https://wpnews.pro/news/frontiermemory", "canonical_source": "https://frontiermemory.com/", "published_at": "2026-10-21 17:50:46+00:00", "updated_at": "2026-10-21 18:13:04.924316+00:00", "lang": "en", "topics": ["ai-agents", "ai-research", "large-language-models", "artificial-intelligence"], "entities": ["Letta", "FrontierMemory", "Opus 5.5", "GLM-5.3", "Kimi K3", "GPT-6.1 Sol", "Gemini 4 Argon", "Letta Dream"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/frontiermemory", "markdown": "https://wpnews.pro/news/frontiermemory.md", "text": "https://wpnews.pro/news/frontiermemory.txt", "jsonld": "https://wpnews.pro/news/frontiermemory.jsonld"}}