{"slug": "stop-summarizing-your-agent-s-memory-triage-it", "title": "Stop Summarizing Your Agent's Memory. Triage It", "summary": "A developer built context-triage, an open-source tool on 10xGraph that replaces an LLM agent's single all-or-nothing history summary with a per-turn decision model. A cheap model, Cloudflare Clef, assigns each old turn a keep, compress or forget action with a confidence score, and a small writer model (claude-haiku-5-5) drafts notes only for turns marked compress, while the full conversation stays in persistent history. Decisions are logged with confidence, and suggestions below a 0.7 threshold are ignored so the turn is kept, though the developer cautions the scores are uncalibrated and the gate is a fail-safe rather than an accuracy guarantee.", "body_md": "*A small decision model decides, turn by turn, what an LLM agent keeps, compresses or forgets.*\n\n**Prompt caching is out of scope here.** Editing or removing an earlier message changes the prompt from that point on, so content after the edit cannot reuse the cache. Unchanged prefixes, for example the system prompt and tools, can still be reused. Because triage removes the oldest turns, most of the conversation after the system prompt is affected. The exact impact depends on each provider's caching implementation, and a separate write-up will cover it.\n\nEvery turn and every tool call adds to an agent's context. That context is sent to the model on each call, so it drives the cost and latency of every call that follows. It also fills with material that no longer matters: old logs, abandoned directions, pleasantries. The noise sits next to the facts the agent needs.\n\nThe default answer is one big summary. When the context passes a limit, you ask an LLM to summarize everything so far and replace the history with the result. That is an all-or-nothing rewrite. It is lossy, it happens in a single irreversible step, and you cannot see why a given detail survived or vanished. It also spends your most expensive model on housekeeping.\n\nOne distinction up front. This is context management: what the model carries into its next call. It is not memory management, which is what an agent retains and retrieves across tasks or sessions. It complements memory systems such as Mem0 or Zep rather than replacing them, and could become a foundation for one.\n\n[context-triage](https://github.com/Iamsdt/context-triage), built on [10xGraph](https://10xgraph.com/), replaces the single rewrite with a separate decision for each unit of work. A unit is one turn: a user message and everything that answered it. (The Python package inside is still named `unforget`, its working name.) Each old turn gets one of three actions:\n\n**Illustrative example** (invented for explanation, not output from a run):\n\nAfter triage, the working context holds the Django and PostgreSQL turn verbatim, the one-sentence note, and the newest turns. The full conversation stays in persistent history.\n\nA fourth action, archive, existed in the first version and is covered below as a planned return. The current code does not include it.\n\nThe design rests on a few principles.\n\n**Separate deciding from writing.** A cheap decision model, Cloudflare Clef, decides. It does not write text. For each question it returns a probability for every option plus a confidence. A small writer model, claude-haiku-5-5, then writes notes, and only for the turns marked compress. The main model is never asked to summarize itself.\n\n**Log every decision with its confidence.** Every decision can be recorded with Clef's suggestion and its model-reported confidence score. That gives you an audit trail of what was removed and how sure the model said it was. It also gives you a gate: if the top option falls below a threshold (0.7 here), the suggestion is ignored and the turn is kept. Be careful what the number means. A 0.95 does not mean the decision is right 95% of the time. The scores are not calibrated against a labelled set. The 0.7 gate is a fail-safe, not an accuracy guarantee.\n\n**Keep three views of one conversation.**\n\n`state.context`, the only thing triage edits.\nTake a medical assistant. A user asks for a poem and the bot refuses: \"I only handle medical questions.\" Several turns later the context is over budget, and Clef may decide that exchange is chatter with nothing worth reusing. The model stops carrying it, and the user can still scroll up and read it. Note the difference between data retention and model accessibility. The data is retained, but the model cannot read removed turns back.\n\n**Work in whole turns.** Each turn is an atomic unit. That keeps tool calls with their results and avoids leaving half-finished tool interactions in the context. Provider-specific ordering rules still have to be respected. The newest turns are never candidates, so the work in progress is safe.\n\n**Fail safe.** If the decision model fails, the writer fails, or credentials are missing, the context is left unchanged and the reason is recorded. A failed trim costs you tokens, not correctness.\n\n**Measure, do not estimate.** The trigger is the token usage the provider reported for the last reply, not a character count. The check runs only when that number exceeds the budget.\n\n**Write the policy in plain language.** What counts as keep, compress or drop is not a trained classifier. It is the policy dict handed to the decision model:\n\n```\nACTION_CRITERIA = {\n    \"keep\": \"Needed verbatim: the current work depends on its exact wording, code or numbers.\",\n    \"compress\": (\n        \"Useful but verbose, such as tool output, logs or listings, or it states preferences \"\n        \"or decisions: a 1 to 3 sentence note would keep what matters.\"\n    ),\n    \"drop\": \"Greetings, thanks, small talk or chatter with nothing worth reusing.\",\n}\n```\n\nYour team can read this, argue about it and change it without retraining anything. The policy routes preferences and decisions to compress, not drop. Whether the decision model honours that is untested.\n\nEach turn is clipped to 800 characters (head and tail) before Clef sees it, and this is a named risk. A long requirement whose key constraint sits in the middle can be judged on its edges and dropped. That matters most for code, configs and technical specs. A possible fix is structure-aware extraction before classification, so the decision model sees the parts that carry the constraints.\n\ncontext-triage is a pluggable context manager in 10xGraph. The flow, in order:\n\nThe answer has already streamed by the time this runs, so the user does not wait on it.\n\nThe cost shape is a plain sum:\n\ntotal cost = classification + compression + cache impact\n\nThe net benefit depends on how many future model calls run on the smaller context. We make no savings claims.\n\nThe first version had a fourth action, archive, plus a `recall` tool. We removed both to keep the core loop easy to read. Bringing them back is the next step, because they answer the biggest weakness of the three-action version: once a turn is dropped or compressed, the model cannot get the detail back.\n\nIn that version an archived turn left the working set, the full turn was stored verbatim in thread state with a short label, and the context kept an index of labels. When the agent called recall with an item's number, the turn came back as a tool result and the archive itself was never modified.\n\nWhy this matters to a leader:\n\nA recall costs a tool call, with its latency and tokens, and the text re-enters the context when used. There is also a trust boundary. Recalled content is historical data, not fresh instructions. It can contain user instructions, tool output, external documents or prompt-injection attempts, so it must be returned as data, with that boundary made clear to the model.\n\nWe have one live run so far, a smoke test on 2026-11-01. It is the only evidence we have.\n\n| Item | Value | \n|---|---|\n| Token budget | 500 | \n| Messages sent | \"Hi\", \"what is 1 + 1\", \"what is mars\" | \n| Reported context size | 1183 tokens | \n| Decision: \"Hi\" | drop at 0.73 (an earlier check at 0.546 was below the gate, so it was kept) | \n| Decision: \"what is 1 + 1\" | drop at 0.955 | \n| Model context afterwards | 2 messages | \n| Compressed | none | \n\nWhat it shows:\n\nWhat it does not show:\n\nThe compress path has not run live yet. We report no benchmarks or accuracy numbers.\n\nWhen we first removed turns from a conversation with extended thinking, the next request failed with a live 400: \"Invalid signature in thinking block.\" Claude signs each thinking block against the whole prefix before it. Remove earlier turns and the blocks you kept no longer match the prefix they were signed against.\n\nThe lesson is general. Provider-signed state is bound to its history, so any system that edits context, whether by trimming, summarizing or triage, has to plan for it.\n\nOur fix is blunt: after any change, strip every signed or redacted thinking block from the remaining messages, not just the first invalid one. The cost is that the model loses its earlier reasoning blocks. This strip route is what context-triage does through 10xGraph.\n\nThere is another route, which we verified in a separate small script on the direct Claude API with claude-sonnet-5-5 (not part of this project). In 10xGraph you pass these as extra Agent kwargs:\n\n```\nthinking={\"type\": \"adaptive\", \"block_binding\": {\"prefix_mismatch_behavior\": \"drop_block\"}},\nextra_headers={\"anthropic-beta\": \"thinking-binding-controls-2026-08-01\"},\n```\n\nWith `drop_block`, the request proceeds instead of erroring. It does not preserve the invalidated reasoning: the API discards the thinking blocks whose prefix no longer matches. Because triage removes the oldest turns, in practice most of the earlier reasoning is discarded. It is a beta, and we only verified it on the direct Claude API. The binding check is enforced by default for accounts created on or after 2026-08-31, and setting the field opts a request in.\n\ncontext-triage is a context-management layer that can sit next to a memory store. It makes a separate decision per turn with a cheap model that is not your main model, needs no training, and logs its confidence scores. We do not claim to be first, and we have not measured it against any of these.\n\nEach of these is a decision point, not a footnote.\n\nThe architecture separates three responsibilities that a single summary call mixes together:\n\nThis separation is the part worth taking away even if you never use Clef. You can swap the decision model, change the writer, or add an archive without touching the other two.\n\nWhat we will measure next:\n\nThe code and setup instructions are on GitHub: [github.com/Iamsdt/context-triage](https://github.com/Iamsdt/context-triage).\n\ncontext-triage is built on [10xGraph](https://10xgraph.com/), a framework for building production AI agents.", "url": "https://wpnews.pro/news/stop-summarizing-your-agent-s-memory-triage-it", "canonical_source": "https://dev.to/shudiptotrafder/stop-summarizing-your-agents-memory-triage-it-3771", "published_at": "2026-11-01 17:37:40+00:00", "updated_at": "2026-11-01 17:52:24.156878+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-tools", "developer-tools"], "entities": ["context-triage", "10xGraph", "Cloudflare Clef", "claude-haiku-5-5", "Mem0", "Zep"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/stop-summarizing-your-agent-s-memory-triage-it", "markdown": "https://wpnews.pro/news/stop-summarizing-your-agent-s-memory-triage-it.md", "text": "https://wpnews.pro/news/stop-summarizing-your-agent-s-memory-triage-it.txt", "jsonld": "https://wpnews.pro/news/stop-summarizing-your-agent-s-memory-triage-it.jsonld"}}