Automatic Compaction: How Shelley Keeps Going and Going Without Losing the Plot

Shelley, the coding agent from the blog.exe.dev team, has shipped its third compaction implementation and enabled "auto compaction" by default, inserting a nudge message at about 200k tokens and giving the model a tool to truncate specific tool outputs and squish history sections into shorter rewritten segments with pointers to the originals. A linter verifies each compaction is neither too big nor too little, typically requiring a second model attempt, and compacted data remains queryable in SQLite. A recent long Opus 5.5 conversation cost about $25 across 4 compactions with context capping out near 200k tokens, with $1 cache misses marked where price spiked.

When it comes to LLMs, context rules everything around us. An agent loop is inherently quadratic: messages accumulate and every iteration you send the full stack of messages to the model. That stack of messages is the context , and it starts with a system prompt, descriptions of the tools, and your instructions. Messages upon messages get added. Tool responses in other words, the agent reading your source code usually end up being the majority of the context. This impacts cost see Expensively Quadratic: the LLM Agent Cost Curve https://blog.exe.dev/expensively-quadratic and quality. You want to give the model just the "right amount" of context, but how? Compaction is the process of taking your current long conversation and rewriting or removing stuff to make it a shorter conversation. Shelley's approach to compaction is to insert a "nudge" message from the harness at about 200k tokens and to give the model a compaction tool that lets it a truncate specific tool outputs and b "squish" together sections of history into smaller, rewritten segments which will then have pointers to the originals . 200k seems like the sweet spot for current frontier models, but we've left the exact number configurable. The compaction tool requires a few rounds. First, the model gets an "index" so that it can identify the messages by a sequential id. Then, the model suggests messages to collapse or trim. Shelley has a "linter" that verifies that the compaction isn't too big or too little. The model tries again, and usually the second time is just right. After compaction, the conversation continues as it always did, except the squished chunks are shorter, the verbose tool outputs are trimmed, and evidence of the compaction is no longer visible to the LLM. From a data model perspective, a Shelley conversation has some mutable state slug name, model choice, current draft and an append-only log of messages. Compaction is materialized as special "compaction" messages, and the harness, when concatenating all the messages together to send to the LLM, replaces the relevant chunks with their rewrites and trimmed results. If the model ever wants to read something that got compacted away, the data is in SQLite, and Shelley has a skill to hint at the right query. One of Shelley's principles is the ability to be introspective. I tuned the linter heuristics by having Shelley introspect and backtest its own past conversations. This is Shelley's third implementation of compaction. The previous two implementations started a new "generation" of a conversation inside the same conversation, with a brand-new context and a summary first implementation , and preserving the most recent messages second implementation . This third implementation keeps it going as one conversation, but the history is somewhat rewritten. I'd love to believe this is our last implementation, but model capabilities have been changing quickly, and I expect there will be more iteration. From the cost perspective, this is the cost graph for a recent long Opus 5.5 conversation. It adds up to about $25 and has gone through 4 compactions, with the context length capping out at about 200k. The little arrows where the price spiked are $1 cache misses, probably when I stepped away from the computer. You can get this graph in your Shelley session by clicking on the context length counter. We've been rocking this implementation for a few weeks, and we think it works quite well. We've enabled "auto compaction" by default. I used to click on "Compact and Send" every once in a while and look at the context length. Now, I just talk, and talk, and talk, and it's all good. I've also started letting my agentic conversations meander a bit more, do follow-ups and so on. Tune in next week to hear about switching to asynchronous tool and subagent calls. Or maybe we'll talk more about multiple input modalities If you're interested in more of our writing about coding agents: - The Unreasonable Effectiveness of an LLM Agent Loop with Tool Use https://sketch.dev/blog/agent-loop - Expensively Quadratic: the LLM Agent Cost Curve https://blog.exe.dev/expensively-quadratic - The Topology of an LLM Agent Loop With Tool Use https://blog.exe.dev/topology P.S.: Here's a behind-the-scenes look at yak-shaving involved in building the animation above. I built it using Shelley, after having earlier built a bespoke program using Excalidraw https://github.com/excalidraw , and after adding animations to said bespoke drawing program.