{"slug": "reasoning-in-agent-memory-raindrop-hits-67-on-support-tickets-what-the-study", "title": "Reasoning in Agent Memory: Raindrop Hits 67% on Support Tickets — What the Study Says", "summary": "Raindrop's November 9, 2026 research post found that giving an agent reviewer access to the model's chain of thought raised mean support-ticket resolution on τ²-bench from 59.2% with action-only memory to 66.5% with reasoning memory, versus a 33.5% no-memory baseline. The 66.5% figure, reported across four open agent models with Claude Opus 5.5 acting as the reviewer, was widely misattributed to Anthropic as a 67% Claude support-resolution rate. Raindrop also reported DeepSeek v4 Flash improving from 71% on tickets 2–8 to 93% in the final third of sequences with reasoning memory, while no-memory runs stayed flat near 50%.", "body_md": "**A viral headline blamed Anthropic for a 67% support-ticket resolution rate, but the number comes from [Raindrop’s November 9, 2026 research post](https://www.raindrop.ai/blog/chain-of-thought-memory/?ref=explainx) on whether agent memory works better when a reviewer also reads the model’s chain of thought.** Raindrop is an observability company; the experiment uses **Claude Opus 5.5** as a **reviewer**, not as proof of a new **Claude product feature**.\n\nThis post separates **Raindrop’s benchmark story** from **[Anthropic’s agent memory research](https://www.anthropic.com/research/components-of-agent-memory?ref=explainx)** — complementary ideas, different papers, easy to mash together in a social summary.\n\n## TL;DR\n\n| Question | Answer | \n|---|---|\n| Who published the 67% figure? | **Raindrop** (Nov 9, 2026) | \n| Task domain | **Telecom customer support** on**τ²-bench** (Sierra Research) | \n| Baseline | **33.5%** mean resolution with**no memory** | \n| Action memory only | **59.2%** mean | \n| + Reasoning in memory | **66.5%** mean (**~67%** in headlines) | \n| Agent models tested | Four **open** models with exposed reasoning | \n| Reviewer | **Claude Opus 5.5** writes lessons after each ticket | \n\n## What Raindrop tested\n\nRaindrop frames the question as **memory-based recursive self-improvement (RSI)** without weight updates:\n\n1. An **agent** works a ticket using tools (account lookup, network resets, MMS settings).\n2. A **reviewer** reads the trace and writes**lessons** appended to instructions for later tickets.\n3. The reviewer either sees **actions only** or**actions plus reasoning** .\n\nThe benchmark is **τ²-bench** in **solo mode** (no simulated customer), **24-ticket sequences**, **12 sequences per model**, lessons adapted from **Google ReasoningBank**, per Raindrop.\n\nTicket themes include **mobile data outages** and **MMS failures** — realistic telecom ops, not toy riddles.\n\n## Results table (from Raindrop)\n\nRaindrop reports mean resolution rates **excluding the first ticket** in each sequence (before lessons exist):\n\n| Agent model | No memory | Action memory | Reasoning memory | \n|---|---|---|---|\n| gpt-oss-120b | 34.4% | 58.0% | 62.7% | \n| MiniMax-M2.5 | 35.9% | 64.5% | 64.9% | \n| DeepSeek v4 Flash | 49.3% | 68.8% | 85.5% | \n| Qwen3-Next-80B thinking | 14.5% | 45.7% | 52.9% | \n| **Four-model mean** | **33.5%** | **59.2%** | **66.5%** | \n\nThe **+7.2 point** lift from action memory to reasoning memory on the mean is the **“Anthropic 67%”** misread — it is **Raindrop’s cross-model average**, not **Claude** solving tickets alone.\n\n### Finding 2: learning over a sequence\n\nRaindrop highlights **DeepSeek v4 Flash** improving from **71%** on tickets 2–8 to **93%** in the final third of sequences when **reasoning memory** is enabled, while **no-memory** runs flat near **50%**. **Action-only** memory **rose then fell** late in sequences — a hint that **shallow lessons** can **overfit** or **conflict**.\n\n## How this differs from Anthropic’s memory post\n\nexplainx.ai already covered Anthropic’s **[components of agent memory](https://explainx.ai/blog/anthropic-components-of-agent-memory-dreaming-cl-bench-pokemon-wiki-stainless-2026)** post:\n\n- **Dreaming** is an**offline job** that reorganizes**notes, wikis, and injected context** .\n- Gains appear on **CL-Bench** ,**Pokémon** , and**multi-session corporate wikis** .\n\nAgent memory hierarchy with dreaming between sessions in Anthropic research diagram\n\n*Image: [Anthropic](https://www.anthropic.com/research/components-of-agent-memory?ref=explainx)*\n\nRaindrop stays in **customer-support tool loops**, uses an **automated grader**, and asks a **narrow question**: **does exposing chain of thought to the reviewer improve lesson quality?**\n\nCL-Bench bar chart showing higher scores when Claude models use a memory store\n\n*Image: [Anthropic](https://www.anthropic.com/research/components-of-agent-memory?ref=explainx)*\n\nAnthropic’s chart above is **not** the 67% ticket line — it shows **database tasks** with persistent memory stores. Keep benchmarks separate when you brief executives.\n\n## Why action memory already captured most gains\n\nRaindrop’s **+25.7 point** jump from **none → action** dwarfs the **+7.2** from **action → reasoning** on the mean. For many teams, **logging tool traces and outcomes** is the **first ship**, not **full chain-of-thought archives**.\n\n**MiniMax-M2.5** basically flatlines on reasoning — plausible if its exposed “thinking” is **redacted**, **unstable**, or **uncorrelated** with tool choices.\n\n## Implementation notes for builders\n\n**Memory format**\n\n- Raindrop stores **lessons in instructions** , not weight deltas — same pattern as**DESIGN.md** and**agent skills** we teach in[loop engineering](https://explainx.ai/blog/loop-engineering-coding-agents-claude-code-guide-2026) .\n\n**Reviewer cost**\n\n- **Opus 5.5** reviewing**every ticket** is expensive. Production systems batch reviews, summarize, or use**smaller judges** — Raindrop is a**research setup** , not a cost sheet.\n\n**Safety**\n\n- Reasoning logs may contain **PII** from tickets. Apply**retention limits** and**redaction** before lessons re-enter prompts. See[semantic caching cost guide](https://explainx.ai/blog/semantic-caching-llm-apps-cost-latency-guide-2026) for adjacent infra thinking.\n\n**Benchmark choice**\n\n- **τ²-bench** also appears in[IBM ALTK-Evolve](https://explainx.ai/blog/ibm-altk-evolve-learned-guidelines-tau2-bench-no-fine-tuning-2026) coverage — useful cross-read when you design**guideline learners** .\n\n## When reasoning memory is worth the tokens\n\n| Signal | Lean toward reasoning memory | \n|---|---|\n| Model exposes **stable** chain-of-thought aligned with tools | Yes | \n| Errors are **planning** mistakes, not tool API misuse | Yes | \n| Traces are **short** and**low PII** | Yes | \n| Thinking is **hidden** or**post-hoc rationalization** | No — stick to action logs | \n| Reviewer budget **tight** | No — compress to action summaries | \n\n## Connection to Anthropic production direction\n\nAnthropic’s post argues **memory is layered**: transcripts, notes, representations, **injected context**. **Dreaming** moves information **up** the stack offline. Raindrop’s RSI is a **horizontal slice**: **lessons → instructions** for **repeatable tickets**.\n\nNeither replaces **compaction** or **RAG**. They answer **“what should persist after success or failure?”**\n\nStainless shipped tooling around Anthropic’s memory ideas — see [Stainless agent memory SDK notes](https://explainx.ai/blog/anthropic-components-of-agent-memory-dreaming-cl-bench-pokemon-wiki-stainless-2026) in our earlier article for API angles.\n\n## What people are asking\n\n**Can I buy this from Raindrop?**\n\nThe post is **research marketing** for observability. Check Raindrop’s product docs for **trace capture** features; do not assume **67%** in **your** tickets.\n\n**Does Opus 5.5 as reviewer mean Claude is best at telecom?**\n\nNo. **Four open agents** did the tickets; **Opus** only **wrote lessons**.\n\n**Will Anthropic ship “reasoning memory” as a toggle?**\n\nAnthropic documents **memory APIs** and **dreaming jobs** separately. Watch **release notes**, not Raindrop’s blog, for **product SLAs**.\n\n## Example lesson schema (action-only vs reasoning)\n\nRaindrop adapts **ReasoningBank-style** instructions: short, imperative **lessons** appended to the system prompt. A simplified pattern:\n\n**Action-only reviewer input**\n\n- Tool call sequence and grader verdict **PASS/FAIL** .\n- Lesson: “When MMS fails, reset APN before escalating to tower ops.”\n\n**Reasoning-augmented reviewer input**\n\n- Same tools, plus the agent’s **stated plan** (“user on MVNO, check port status first”).\n- Lesson: “If reasoning mentions **MVNO** , query**porting table** before**APN reset** to avoid false fixes.”\n\nThe second lesson is only valuable when **reasoning** matches **tools**. If the model **narrates** steps it never executed, reasoning memory **amplifies hallucinated policy**.\n\nRedact **account numbers**, **IMEIs**, and **customer names** before lessons persist. Raindrop’s telecom setting is synthetic; your **production** traces are not.\n\n## Cost model sketch\n\nAssume **24 tickets × 12 sequences × 4 models** in the published design — thousands of **reviewer calls**. At **Opus**-tier pricing, the **review bill** can exceed the **agent bill**. Production architectures should:\n\n- Review **failures only** after**k** successes stabilize.\n- **Distill** lessons nightly into a**single****skills** file humans approve weekly — the pattern[Janestreet Socratic tutor skills](https://explainx.ai/blog/janestreet-agent-skills-socratic-tutor-github-2026) echo for**human-gated** learning.\n\n## Metrics beyond pass rate\n\nRaindrop focuses on **automated resolution**. Operations teams also track:\n\n- **Mean time to resolution** and**reopens** .\n- **Customer sentiment** when a**simulated customer** returns (τ²-bench supports richer modes Raindrop disabled).\n- **Lesson collision rate** — contradictory bullets after long sequences.\n\nAction memory’s **late-sequence regression** in Raindrop’s charts is a warning: **unbounded lesson lists** need **pruning**, similar to [Databricks MemCurrent dreaming](https://explainx.ai/blog/databricks-memcurrent-agent-memory-3x-pass-rate-dreaming-episodic-derived-genie-2026) themes.\n\n## Reproduce at small scale\n\n1. Export **20** anonymized support transcripts from**your** help desk.\n2. Write a **lesson schema** (max 5 bullets per case).\n3. Run **action-only** review with your preferred model; measure**human** resolution on a holdout set.\n4. Add **reasoning** only if step 3 plateaus and chains are**clean** .\n\nOur [A2A vs MCP guide](https://explainx.ai/blog/what-is-a2a-agent2agent-protocol-mcp-comparison-guide-2026) helps if your tickets span **multiple agent handoffs** — memory must be **per-role**, not one global blob.\n\n## Headline hygiene for comms teams\n\nIf your **marketing** drafts say **“Anthropic hits 67% on support tickets,”** correct it before **legal** sees it:\n\n- Attribute the benchmark to **Raindrop** and**τ²-bench** .\n- Name the **four open agents** , not**Claude Sonnet/Opus** as workers.\n- Note **Opus 5.5** as**reviewer** , which is still**Anthropic** , but a**different role** .\n\nMisattribution matters because **enterprise buyers** conflate **vendor evals** with **product warranties**.\n\n## Open-model-specific takeaways\n\n| Model | Raindrop pattern | \n|---|---|\n| **Qwen3-Next-80B thinking** | Largest **action-memory** lift; still low absolute scores | \n| **DeepSeek v4 Flash** | Best case for **reasoning memory** plus**sequence learning** | \n| **MiniMax-M2.5** | Strong **action** baseline; reasoning adds ~0 | \n| **gpt-oss-120b** | Mid-pack; benefits from both memory types | \n\nTeams standardizing on **one** open agent should read the **row**, not the **mean**.\n\n## Limits Raindrop acknowledges\n\n- **Solo τ²-bench** removes**messy user language** .\n- **Automated grader** may not match**CSAT** .\n- **12 sequences** per model is**small** for executive ROI claims.\n\nTreat **67%** as **an experiment mean**, not a **vendor SLA**.\n\nFor **evaluation leads**, archive the **Raindrop** tables alongside your own **τ²-bench** runs so **quarterly business reviews** do not confuse **lab results** with **production SLAs**.\n\n## Related reading\n\n## Sources\n\n- [Raindrop: The role of reasoning in agent memory](https://www.raindrop.ai/blog/chain-of-thought-memory/?ref=explainx) , November 9, 2026.\n- [Anthropic: The components of agent memory](https://www.anthropic.com/research/components-of-agent-memory?ref=explainx) , November 2026.\n\n*Benchmark configuration and model names follow Raindrop as published.*", "url": "https://wpnews.pro/news/reasoning-in-agent-memory-raindrop-hits-67-on-support-tickets-what-the-study", "canonical_source": "https://explainx.ai/blog/raindrop-reasoning-agent-memory-tau2-bench-67-percent-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog", "published_at": "2026-11-10 00:00:00+00:00", "updated_at": "2026-11-10 03:50:28.658160+00:00", "lang": "en", "topics": ["ai-agents", "ai-research", "large-language-models", "artificial-intelligence"], "entities": ["Raindrop", "Anthropic", "Claude Opus 5.5", "DeepSeek v4 Flash", "Qwen3-Next-80B", "gpt-oss-120b", "MiniMax-M2.5", "τ²-bench"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/reasoning-in-agent-memory-raindrop-hits-67-on-support-tickets-what-the-study", "markdown": "https://wpnews.pro/news/reasoning-in-agent-memory-raindrop-hits-67-on-support-tickets-what-the-study.md", "text": "https://wpnews.pro/news/reasoning-in-agent-memory-raindrop-hits-67-on-support-tickets-what-the-study.txt", "jsonld": "https://wpnews.pro/news/reasoning-in-agent-memory-raindrop-hits-67-on-support-tickets-what-the-study.jsonld"}}