A viral headline blamed Anthropic for a 67% support-ticket resolution rate, but the number comes from Raindrop’s November 9, 2026 research post on whether agent memory works better when a reviewer also reads the model’s chain of thought. Raindrop is an observability company; the experiment uses Claude Opus 5.5 as a reviewer, not as proof of a new Claude product feature.
This post separates Raindrop’s benchmark story from Anthropic’s agent memory research — complementary ideas, different papers, easy to mash together in a social summary.
TL;DR #
| Question | Answer |
|---|---|
| Who published the 67% figure? | Raindrop (Nov 9, 2026) |
| Task domain | Telecom customer support onτ²-bench (Sierra Research) |
| Baseline | 33.5% mean resolution withno memory |
| Action memory only | 59.2% mean |
| + Reasoning in memory | 66.5% mean (~67% in headlines) |
| Agent models tested | Four open models with exposed reasoning |
| Reviewer | Claude Opus 5.5 writes lessons after each ticket |
What Raindrop tested #
Raindrop frames the question as memory-based recursive self-improvement (RSI) without weight updates:
- An agent works a ticket using tools (account lookup, network resets, MMS settings).
- A reviewer reads the trace and writeslessons appended to instructions for later tickets.
- The reviewer either sees actions only oractions plus reasoning .
The benchmark is τ²-bench in solo mode (no simulated customer), 24-ticket sequences, 12 sequences per model, lessons adapted from Google ReasoningBank, per Raindrop.
Ticket themes include mobile data outages and MMS failures — realistic telecom ops, not toy riddles.
Results table (from Raindrop) #
Raindrop reports mean resolution rates excluding the first ticket in each sequence (before lessons exist):
| Agent model | No memory | Action memory | Reasoning memory |
|---|---|---|---|
| gpt-oss-120b | 34.4% | 58.0% | 62.7% |
| MiniMax-M2.5 | 35.9% | 64.5% | 64.9% |
| DeepSeek v4 Flash | 49.3% | 68.8% | 85.5% |
| Qwen3-Next-80B thinking | 14.5% | 45.7% | 52.9% |
| Four-model mean | 33.5% | 59.2% | 66.5% |
The +7.2 point lift from action memory to reasoning memory on the mean is the “Anthropic 67%” misread — it is Raindrop’s cross-model average, not Claude solving tickets alone.
Finding 2: learning over a sequence
Raindrop highlights DeepSeek v4 Flash improving from 71% on tickets 2–8 to 93% in the final third of sequences when reasoning memory is enabled, while no-memory runs flat near 50%. Action-only memory rose then fell late in sequences — a hint that shallow lessons can overfit or conflict.
How this differs from Anthropic’s memory post #
explainx.ai already covered Anthropic’s components of agent memory post:
- Dreaming is anoffline job that reorganizesnotes, wikis, and injected context .
- Gains appear on CL-Bench ,Pokémon , andmulti-session corporate wikis .
Agent memory hierarchy with dreaming between sessions in Anthropic research diagram
Image: Anthropic Raindrop stays in customer-support tool loops, uses an automated grader, and asks a narrow question: does exposing chain of thought to the reviewer improve lesson quality?
CL-Bench bar chart showing higher scores when Claude models use a memory store
Image: Anthropic Anthropic’s chart above is not the 67% ticket line — it shows database tasks with persistent memory stores. Keep benchmarks separate when you brief executives.
Why action memory already captured most gains #
Raindrop’s +25.7 point jump from none → action dwarfs the +7.2 from action → reasoning on the mean. For many teams, logging tool traces and outcomes is the first ship, not full chain-of-thought archives.
MiniMax-M2.5 basically flatlines on reasoning — plausible if its exposed “thinking” is redacted, unstable, or uncorrelated with tool choices.
Implementation notes for builders #
Memory format
- Raindrop stores lessons in instructions , not weight deltas — same pattern asDESIGN.md andagent skills we teach inloop engineering .
Reviewer cost
- Opus 5.5 reviewingevery ticket is expensive. Production systems batch reviews, summarize, or usesmaller judges — Raindrop is aresearch setup , not a cost sheet.
Safety
- Reasoning logs may contain PII from tickets. Applyretention limits andredaction before lessons re-enter prompts. Seesemantic caching cost guide for adjacent infra thinking.
Benchmark choice
- τ²-bench also appears inIBM ALTK-Evolve coverage — useful cross-read when you designguideline learners .
When reasoning memory is worth the tokens #
| Signal | Lean toward reasoning memory |
|---|---|
| Model exposes stable chain-of-thought aligned with tools | Yes |
| Errors are planning mistakes, not tool API misuse | Yes |
| Traces are short andlow PII | Yes |
| Thinking is hidden orpost-hoc rationalization | No — stick to action logs |
| Reviewer budget tight | No — compress to action summaries |
Connection to Anthropic production direction #
Anthropic’s post argues memory is layered: transcripts, notes, representations, injected context. Dreaming moves information up the stack offline. Raindrop’s RSI is a horizontal slice: lessons → instructions for repeatable tickets.
Neither replaces compaction or RAG. They answer “what should persist after success or failure?”
Stainless shipped tooling around Anthropic’s memory ideas — see Stainless agent memory SDK notes in our earlier article for API angles.
What people are asking #
Can I buy this from Raindrop?
The post is research marketing for observability. Check Raindrop’s product docs for trace capture features; do not assume 67% in your tickets.
Does Opus 5.5 as reviewer mean Claude is best at telecom?
No. Four open agents did the tickets; Opus only wrote lessons.
Will Anthropic ship “reasoning memory” as a toggle?
Anthropic documents memory APIs and dreaming jobs separately. Watch release notes, not Raindrop’s blog, for product SLAs.
Example lesson schema (action-only vs reasoning) #
Raindrop adapts ReasoningBank-style instructions: short, imperative lessons appended to the system prompt. A simplified pattern:
Action-only reviewer input
- Tool call sequence and grader verdict PASS/FAIL .
- Lesson: “When MMS fails, reset APN before escalating to tower ops.”
Reasoning-augmented reviewer input
- Same tools, plus the agent’s stated plan (“user on MVNO, check port status first”).
- Lesson: “If reasoning mentions MVNO , queryporting table beforeAPN reset to avoid false fixes.”
The second lesson is only valuable when reasoning matches tools. If the model narrates steps it never executed, reasoning memory amplifies hallucinated policy.
Redact account numbers, IMEIs, and customer names before lessons persist. Raindrop’s telecom setting is synthetic; your production traces are not.
Cost model sketch #
Assume 24 tickets × 12 sequences × 4 models in the published design — thousands of reviewer calls. At Opus-tier pricing, the review bill can exceed the agent bill. Production architectures should:
- Review failures only afterk successes stabilize.
- Distill lessons nightly into asingle****skills file humans approve weekly — the patternJanestreet Socratic tutor skills echo forhuman-gated learning.
Metrics beyond pass rate #
Raindrop focuses on automated resolution. Operations teams also track:
- Mean time to resolution andreopens .
- Customer sentiment when asimulated customer returns (τ²-bench supports richer modes Raindrop disabled).
- Lesson collision rate — contradictory bullets after long sequences.
Action memory’s late-sequence regression in Raindrop’s charts is a warning: unbounded lesson lists need pruning, similar to Databricks MemCurrent dreaming themes.
Reproduce at small scale #
- Export 20 anonymized support transcripts fromyour help desk.
- Write a lesson schema (max 5 bullets per case).
- Run action-only review with your preferred model; measurehuman resolution on a holdout set.
- Add reasoning only if step 3 plateaus and chains areclean .
Our A2A vs MCP guide helps if your tickets span multiple agent handoffs — memory must be per-role, not one global blob.
Headline hygiene for comms teams #
If your marketing drafts say “Anthropic hits 67% on support tickets,” correct it before legal sees it:
- Attribute the benchmark to Raindrop andτ²-bench .
- Name the four open agents , notClaude Sonnet/Opus as workers.
- Note Opus 5.5 asreviewer , which is stillAnthropic , but adifferent role .
Misattribution matters because enterprise buyers conflate vendor evals with product warranties.
Open-model-specific takeaways #
| Model | Raindrop pattern |
|---|---|
| Qwen3-Next-80B thinking | Largest action-memory lift; still low absolute scores |
| DeepSeek v4 Flash | Best case for reasoning memory plussequence learning |
| MiniMax-M2.5 | Strong action baseline; reasoning adds ~0 |
| gpt-oss-120b | Mid-pack; benefits from both memory types | Teams standardizing on one open agent should read the row, not the mean.
Limits Raindrop acknowledges #
- Solo τ²-bench removesmessy user language .
- Automated grader may not matchCSAT .
- 12 sequences per model issmall for executive ROI claims.
Treat 67% as an experiment mean, not a vendor SLA.
For evaluation leads, archive the Raindrop tables alongside your own τ²-bench runs so quarterly business reviews do not confuse lab results with production SLAs.
Related reading #
Sources #
- [Raindrop: The role of reasoning in agent memory](https://www.raindrop.ai/blog/chain-of-thought-memory/?ref=explainx) , November 9, 2026.
- [Anthropic: The components of agent memory](https://www.anthropic.com/research/components-of-agent-memory?ref=explainx) , November 2026.
Benchmark configuration and model names follow Raindrop as published.