After a coding agent's first compaction, about one-third of the anti-cheating rules it was given are gone — 0.92 of three rule categories, on average. And in 269 constraints pulled from real user sessions, instructions about how to work and what not to do are dropped roughly six times more often than specifications of what to build.
Both numbers come from "Benchmarks Are Surprisingly Compaction-Friendly," a preprint by Jeanne Salle, Hardik Bhatnagar, Matthias Bethge, Sahar Abdelnabi, and Maksym Andriushchenko (arXiv:2610.31778, submitted 22 October 2026). It is a preprint, not peer reviewed. The finding is narrower than "agents forget things": it locates which kind of instruction disappears first, and why the benchmarks most teams rely on will not tell them.
The authors fix the scaffold to OpenCode with five compaction strategies: default self-summarization, tool-output pruning, filesystem off, reasoning-block discarding, and summarization by a smaller model. They run four frontier models — Claude Opus 4.6, GPT-5.4, GPT-5.3-Codex with a 400K context, and Gemini 3.1 Pro Preview — on three long-horizon benchmarks: PaperBench, ProgramBench, and PostTrainBench. Each task and configuration runs twice.
Constraint erosion is measured on ProgramBench. Its first user message contains three rules that appear exactly once: do not copy or vendor the reference implementation, do not use the network, and write an original implementation. After a compaction, those rules remain in context only if the summary restates them. Claude Sonnet 5 acts as judge, reading the summary and the trajectory and reporting which of the three categories is absent.
The design matters: it asks not whether the agent remembered the task, but whether it still had the rule constraining how the task is done.
About 0.92 of the three rule categories are missing after the first compaction — roughly one-third. The number rises monotonically: 1.93 by the tenth compaction, 2.41 by the fifteenth, 2.65 by the thirtieth.
GPT-5.3-Codex drops 2.73 categories per compaction and loses all three in 80% of compactions. GPT-5.4 drops 1.76. Claude Opus 4.6 drops 0.51 and never loses all three.
The loss compounds because each summary is written from the previous one: a rule removed at compaction three has no reason to reappear at four, and nothing restates it. The summarization prompt explicitly asks for "key user requests, constraints, or preferences that should persist." The rules still fall out.
The controlled benchmark has only three rules. For a broader view, the authors sample SWE-chat, real human-agent coding sessions, and take 44 sessions containing at least one compaction. A model extracts the constraints the user gave before that compaction, 269 in total; a judge labels each as preserved or dropped in the next summary.
Overall, 81% of constraints survive and 19% are dropped. The aggregate hides the asymmetry. Feature requirements, the largest category at 35.7% of constraints, survive far more often than process instructions ("how to work") and scope exclusions ("don't touch X"). The paper puts the gap at about six times.
A summarizer preserves what the agent needs to continue. A goal can be reread from a file, a failing test, or the next tool call; progress lives in the repository. A rule like "do not modify the public API" produces no next step and cannot be checked from the transcript, so under compression it looks like the cheapest thing to cut.
That is why the loss is silent: the agent does not announce that it has stopped following a rule; it simply continues without it. In these runs, reward hacking begins before the first compaction, so rule loss cannot explain when cheating starts; why a clean run drifts afterward remains an open question.
The paper's title is not irony. Task scores are surprisingly robust to compaction. For the two 1M-context models, moving the compaction threshold between 160K and 380K tokens barely changes performance. GPT-5.3-Codex, with the smaller window, does worse at lower thresholds on PaperBench. No compaction strategy beats the default self-summarization, and strategies that discard more — dropping reasoning blocks, summarizing with a smaller model — cost some models up to 20 points while leaving others unchanged.
The benchmarks are compaction-friendly by construction: the objective is fixed, the run receives no user input, and progress stays on disk. Most of what compaction discards can be recovered by rereading. Constraints cannot, because they exist only in the conversation. A benchmark that scores the final artifact will not see a rule that quietly stopped being followed.
What the study establishes is narrower and more useful: compaction preserves what the next step needs, and a rule is usually not what the next step needs. If your agent's rules live only in the conversation, every compaction is a chance to lose one.
I write about agent reliability and small-team engineering at hhfa008.github.io/personal/cases.html.