Reasoning in Agent Memory: Raindrop Hits 67% on Support Tickets — What the Study Says

Raindrop's November 9, 2026 research post found that giving an agent reviewer access to the model's chain of thought raised mean support-ticket resolution on τ²-bench from 59.2% with action-only memory to 66.5% with reasoning memory, versus a 33.5% no-memory baseline. The 66.5% figure, reported across four open agent models with Claude Opus 5.5 acting as the reviewer, was widely misattributed to Anthropic as a 67% Claude support-resolution rate. Raindrop also reported DeepSeek v4 Flash improving from 71% on tickets 2–8 to 93% in the final third of sequences with reasoning memory, while no-memory runs stayed flat near 50%.

A viral headline blamed Anthropic for a 67% support-ticket resolution rate, but the number comes from Raindrop’s November 9, 2026 research post https://www.raindrop.ai/blog/chain-of-thought-memory/?ref=explainx on whether agent memory works better when a reviewer also reads the model’s chain of thought. Raindrop is an observability company; the experiment uses Claude Opus 5.5 as a reviewer , not as proof of a new Claude product feature . This post separates Raindrop’s benchmark story from Anthropic’s agent memory research https://www.anthropic.com/research/components-of-agent-memory?ref=explainx — complementary ideas, different papers, easy to mash together in a social summary. TL;DR | Question | Answer | |---|---| | Who published the 67% figure? | Raindrop Nov 9, 2026 | | Task domain | Telecom customer support on τ²-bench Sierra Research | | Baseline | 33.5% mean resolution with no memory | | Action memory only | 59.2% mean | | + Reasoning in memory | 66.5% mean ~67% in headlines | | Agent models tested | Four open models with exposed reasoning | | Reviewer | Claude Opus 5.5 writes lessons after each ticket | What Raindrop tested Raindrop frames the question as memory-based recursive self-improvement RSI without weight updates: 1. An agent works a ticket using tools account lookup, network resets, MMS settings . 2. A reviewer reads the trace and writes lessons appended to instructions for later tickets. 3. The reviewer either sees actions only or actions plus reasoning . The benchmark is τ²-bench in solo mode no simulated customer , 24-ticket sequences , 12 sequences per model , lessons adapted from Google ReasoningBank , per Raindrop. Ticket themes include mobile data outages and MMS failures — realistic telecom ops, not toy riddles. Results table from Raindrop Raindrop reports mean resolution rates excluding the first ticket in each sequence before lessons exist : | Agent model | No memory | Action memory | Reasoning memory | |---|---|---|---| | gpt-oss-120b | 34.4% | 58.0% | 62.7% | | MiniMax-M2.5 | 35.9% | 64.5% | 64.9% | | DeepSeek v4 Flash | 49.3% | 68.8% | 85.5% | | Qwen3-Next-80B thinking | 14.5% | 45.7% | 52.9% | | Four-model mean | 33.5% | 59.2% | 66.5% | The +7.2 point lift from action memory to reasoning memory on the mean is the “Anthropic 67%” misread — it is Raindrop’s cross-model average , not Claude solving tickets alone. Finding 2: learning over a sequence Raindrop highlights DeepSeek v4 Flash improving from 71% on tickets 2–8 to 93% in the final third of sequences when reasoning memory is enabled, while no-memory runs flat near 50% . Action-only memory rose then fell late in sequences — a hint that shallow lessons can overfit or conflict . How this differs from Anthropic’s memory post explainx.ai already covered Anthropic’s components of agent memory https://explainx.ai/blog/anthropic-components-of-agent-memory-dreaming-cl-bench-pokemon-wiki-stainless-2026 post: - Dreaming is an offline job that reorganizes notes, wikis, and injected context . - Gains appear on CL-Bench , Pokémon , and multi-session corporate wikis . Agent memory hierarchy with dreaming between sessions in Anthropic research diagram Image: Anthropic https://www.anthropic.com/research/components-of-agent-memory?ref=explainx Raindrop stays in customer-support tool loops , uses an automated grader , and asks a narrow question : does exposing chain of thought to the reviewer improve lesson quality? CL-Bench bar chart showing higher scores when Claude models use a memory store Image: Anthropic https://www.anthropic.com/research/components-of-agent-memory?ref=explainx Anthropic’s chart above is not the 67% ticket line — it shows database tasks with persistent memory stores. Keep benchmarks separate when you brief executives. Why action memory already captured most gains Raindrop’s +25.7 point jump from none → action dwarfs the +7.2 from action → reasoning on the mean. For many teams, logging tool traces and outcomes is the first ship , not full chain-of-thought archives . MiniMax-M2.5 basically flatlines on reasoning — plausible if its exposed “thinking” is redacted , unstable , or uncorrelated with tool choices. Implementation notes for builders Memory format - Raindrop stores lessons in instructions , not weight deltas — same pattern as DESIGN.md and agent skills we teach in loop engineering https://explainx.ai/blog/loop-engineering-coding-agents-claude-code-guide-2026 . Reviewer cost - Opus 5.5 reviewing every ticket is expensive. Production systems batch reviews, summarize, or use smaller judges — Raindrop is a research setup , not a cost sheet. Safety - Reasoning logs may contain PII from tickets. Apply retention limits and redaction before lessons re-enter prompts. See semantic caching cost guide https://explainx.ai/blog/semantic-caching-llm-apps-cost-latency-guide-2026 for adjacent infra thinking. Benchmark choice - τ²-bench also appears in IBM ALTK-Evolve https://explainx.ai/blog/ibm-altk-evolve-learned-guidelines-tau2-bench-no-fine-tuning-2026 coverage — useful cross-read when you design guideline learners . When reasoning memory is worth the tokens | Signal | Lean toward reasoning memory | |---|---| | Model exposes stable chain-of-thought aligned with tools | Yes | | Errors are planning mistakes, not tool API misuse | Yes | | Traces are short and low PII | Yes | | Thinking is hidden or post-hoc rationalization | No — stick to action logs | | Reviewer budget tight | No — compress to action summaries | Connection to Anthropic production direction Anthropic’s post argues memory is layered : transcripts, notes, representations, injected context . Dreaming moves information up the stack offline. Raindrop’s RSI is a horizontal slice : lessons → instructions for repeatable tickets . Neither replaces compaction or RAG . They answer “what should persist after success or failure?” Stainless shipped tooling around Anthropic’s memory ideas — see Stainless agent memory SDK notes https://explainx.ai/blog/anthropic-components-of-agent-memory-dreaming-cl-bench-pokemon-wiki-stainless-2026 in our earlier article for API angles. What people are asking Can I buy this from Raindrop? The post is research marketing for observability. Check Raindrop’s product docs for trace capture features; do not assume 67% in your tickets. Does Opus 5.5 as reviewer mean Claude is best at telecom? No. Four open agents did the tickets; Opus only wrote lessons . Will Anthropic ship “reasoning memory” as a toggle? Anthropic documents memory APIs and dreaming jobs separately. Watch release notes , not Raindrop’s blog, for product SLAs . Example lesson schema action-only vs reasoning Raindrop adapts ReasoningBank-style instructions: short, imperative lessons appended to the system prompt. A simplified pattern: Action-only reviewer input - Tool call sequence and grader verdict PASS/FAIL . - Lesson: “When MMS fails, reset APN before escalating to tower ops.” Reasoning-augmented reviewer input - Same tools, plus the agent’s stated plan “user on MVNO, check port status first” . - Lesson: “If reasoning mentions MVNO , query porting table before APN reset to avoid false fixes.” The second lesson is only valuable when reasoning matches tools . If the model narrates steps it never executed, reasoning memory amplifies hallucinated policy . Redact account numbers , IMEIs , and customer names before lessons persist. Raindrop’s telecom setting is synthetic; your production traces are not. Cost model sketch Assume 24 tickets × 12 sequences × 4 models in the published design — thousands of reviewer calls . At Opus -tier pricing, the review bill can exceed the agent bill . Production architectures should: - Review failures only after k successes stabilize. - Distill lessons nightly into a single skills file humans approve weekly — the pattern Janestreet Socratic tutor skills https://explainx.ai/blog/janestreet-agent-skills-socratic-tutor-github-2026 echo for human-gated learning. Metrics beyond pass rate Raindrop focuses on automated resolution . Operations teams also track: - Mean time to resolution and reopens . - Customer sentiment when a simulated customer returns τ²-bench supports richer modes Raindrop disabled . - Lesson collision rate — contradictory bullets after long sequences. Action memory’s late-sequence regression in Raindrop’s charts is a warning: unbounded lesson lists need pruning , similar to Databricks MemCurrent dreaming https://explainx.ai/blog/databricks-memcurrent-agent-memory-3x-pass-rate-dreaming-episodic-derived-genie-2026 themes. Reproduce at small scale 1. Export 20 anonymized support transcripts from your help desk. 2. Write a lesson schema max 5 bullets per case . 3. Run action-only review with your preferred model; measure human resolution on a holdout set. 4. Add reasoning only if step 3 plateaus and chains are clean . Our A2A vs MCP guide https://explainx.ai/blog/what-is-a2a-agent2agent-protocol-mcp-comparison-guide-2026 helps if your tickets span multiple agent handoffs — memory must be per-role , not one global blob. Headline hygiene for comms teams If your marketing drafts say “Anthropic hits 67% on support tickets,” correct it before legal sees it: - Attribute the benchmark to Raindrop and τ²-bench . - Name the four open agents , not Claude Sonnet/Opus as workers. - Note Opus 5.5 as reviewer , which is still Anthropic , but a different role . Misattribution matters because enterprise buyers conflate vendor evals with product warranties . Open-model-specific takeaways | Model | Raindrop pattern | |---|---| | Qwen3-Next-80B thinking | Largest action-memory lift; still low absolute scores | | DeepSeek v4 Flash | Best case for reasoning memory plus sequence learning | | MiniMax-M2.5 | Strong action baseline; reasoning adds ~0 | | gpt-oss-120b | Mid-pack; benefits from both memory types | Teams standardizing on one open agent should read the row , not the mean . Limits Raindrop acknowledges - Solo τ²-bench removes messy user language . - Automated grader may not match CSAT . - 12 sequences per model is small for executive ROI claims. Treat 67% as an experiment mean , not a vendor SLA . For evaluation leads , archive the Raindrop tables alongside your own τ²-bench runs so quarterly business reviews do not confuse lab results with production SLAs . Related reading Sources - Raindrop: The role of reasoning in agent memory https://www.raindrop.ai/blog/chain-of-thought-memory/?ref=explainx , November 9, 2026. - Anthropic: The components of agent memory https://www.anthropic.com/research/components-of-agent-memory?ref=explainx , November 2026. Benchmark configuration and model names follow Raindrop as published.