Cross-posted from the ByteChef blog. Part 7 of the AI Agentic Patterns in ByteChef series.
Every chat agent has the same slow leak. Each message grows the history, and the whole history goes into the next prompt. Long chats get slower, cost more and eventually hit the model's context limit.
The usual fix is "keep the last N messages". It works fine until the agent uses tools.
A tool call and its result are two separate messages. Trim the history between them and the model gets one without the other: a result with no call, or a call with no result.
user: What's the status of order 4512?
assistant: [tool call] getOrder(4512) <- window cuts here
tool: { "status": "shipped" } <- this survives
assistant: It shipped yesterday.
Nothing throws. The agent just starts answering strangely, and you spend an afternoon figuring out why.
Chat Memory v2 in ByteChef fixes this by compacting the conversation in whole turns instead of loose messages.
Chat Memory v2 is built on the Spring AI community Session library. It stores a conversation as an ordered log of events. Each user message, assistant reply, tool call and tool result is one event.
A turn is one user message plus everything that follows it (replies, tool calls, tool results) up to the next user message. Compaction only ever removes whole turns. A tool call and its result stay together or leave together.
With the Turn window strategy:
Max turns.
Compacted events leave the prompt, but they stay in storage.
Turn on Enable conversation search tool and the agent gets a conversation_search tool. It passes a keyword, and the tool searches the whole conversation, including archived events that are still kept. Results come back in pages (Search page size, 10 by default).
So the agent's view is short, but its reach is long.
Two limits to know:
bytechef.ai.memory.session-max-archived-events.
The fastest way to see this is the Chat that compacts its history template.
Three steps:
Chat Memory is configured like this:
| Field | Value |
|---|---|
| Conversation ID | ${chat_1.conversationId} |
| Compaction Strategy | Turn window (by turns) |
| Max turns | 3 |
| Enable conversation search tool | on |
The system prompt tells the agent what to expect:
Your visible history is short. If the user asks about something said earlier that you no longer see, use the conversation search tool to find it, and say that you searched.
Three turns is far too small for real use. It's set that low so compaction shows up after a few messages.
To run it:
By then the first turn is archived. The agent can't see it, so it calls conversation_search and tells you it searched.
That's a real test run. The code name was in the first message, which had already left the three-turn window, so the agent searched the archive to answer.
To watch it happen, open the Agent Playbook in the editor. Its tool-call cards show conversation_search running, with the keyword the agent picked and the messages it found.
Past the demo, raise Max turns to 10-20, or switch to Token count or Recursive summarization. The search tool works the same with any strategy.
Look at the template again. The Chat Memory element has its own model, Claude Haiku, separate from the agent's Claude Sonnet.
Only Recursive summarization uses it. With Turn window, old turns are archived and no model is called. Haiku is there so you can flip the dropdown to Recursive summarization without adding anything, and summaries come from a smaller, cheaper model than the one talking to your users.
Pick Recursive summarization with no model on Chat Memory and the run fails with a clear error.
Compaction Strategy is a dropdown. Each option shows only the fields it needs.
| Strategy | Fields | What it does |
|---|---|---|
| None | - | Never compacts. The default. |
| Sliding window (by events) | Max events | Keeps the most recent Max events once the history exceeds it. |
| Turn window (by turns) | Max turns | Keeps the most recent Max turns once the history exceeds it. |
| Token count | Max tokens | Keeps the history within an estimated token budget. |
| Recursive summarization (LLM) | Max events, Max events to keep, Overlap size | Over Max events , a model summarizes older events and the most recentMax events to keep stay in full. |
For Recursive summarization, ByteChef checks two rules when the run starts:
Max events must be greater than Max events to keep. Overlap size must be less than Overlap size is how many of the kept events the summarizer also sees, so the summary connects to what comes after it.
Chat Memory v2 ships as several components with the same fields. Only the storage differs:
bytechef.ai.memory.provider: JDBC, Redis, AWS S3 or in-memory.
The template uses the built-in one, so it needs no extra connection.
The S3 backend is ByteChef's own implementation of the Session library's storage interface. Each session is one JSON object, and writes use S3 conditional writes (If-Match on the ETag), so two writers can't silently overwrite each other's events.
The compaction strategies, triggers and conversation_search come from the Spring AI community Session library. In plain Java you build the memory advisor, pick a trigger and a strategy, and register the search tool yourself. ByteChef runs the library directly and adds:
conversation_search also asks the model for a private "inner monologue", and Claude refuses the whole turn rather than fill it in. ByteChef registers the tool with only a keyword and a page number.
Chat Memory keeps one conversation coherent. It doesn't carry facts into the next one. For that, ByteChef has the Auto Memory Tool: the agent writes down what's worth keeping and reads it back in later chats. It works in deployed runs, not editor test runs.
Both can sit on the same agent: Chat Memory in the Memory slot, the Auto Memory Tool in the Tools slot.
Same rule as Part 1 of the series: load each kind of knowledge only when it's needed. Skills load procedures on demand. Compaction keeps old turns out of the prompt until the agent searches for them.
Dropping old messages one at a time can cut a tool call in half. Chat Memory v2 compacts by whole turns, archives what it removes and lets the agent search the archive. You pick the strategy and the storage on the canvas.
Try it: import Chat that compacts its history, connect Anthropic, send five messages and ask what you said first.
ByteChef is open source (Apache 2.0) and built on Java and Spring. If this was useful, a star on GitHub helps a lot.