[Anderson's Angle](https://www.unite.ai/series/andersons-angle/)
[Add Unite.AI to your preferred sources on Google](https://www.google.com/preferences/source?q=unite.ai)
Compaction is when a string of complex communications with an AI chat get boiled down into a summary, and the interaction continues based not on the entire history, but on that summary. It’s like a ‘fresh start’, in terms of memory usage, and has been deployed as a solution to the limits of context windows in AI discourse.
Initially instituted in agentic AI, the method has been extended to Claude, while compaction for Gemini has yet to emerge from developer chats, and ChatGPT eschews compaction in favor of just stopping the chat.
If you’ve used Codex (essentially a version of ChatGPT that can actually change things on your computer, given permissions) for more than 30 minutes at a time, you will have seen intermittent messages about compacting context, during which time further interactions are unavailable: However, when that compaction is over, the entity talking to you is working from unseen, selective notes, and not a complete memory of the evolution of the chat.
The AI Scofflaw #
In June this year the Chinese paper Governance Decay established that compaction can silently remove standing rules from an agent’s context, causing previously prohibited actions to be carried out:
Across seven models, violations rose from 0% before compaction to 30% afterwards, reaching 59% in some cases. In practical terms, this means that agents can disregard essential directives such as Do not contact anyone who does not have a [our-company] email address – with obvious potential ramifications.
In July, Penn State University and IBM followed up on this research with Lost in Context, finding that compaction could discard instructions governing how an AI should carry out a task, while preserving the task itself. In tests across three settings, the compactors retained just 17% of these instructions on average.
And in August, a US academic collaboration put forward TRACE, showing that compaction could disrupt an agent even when the information it needed remained recoverable. In tests, agents lost track of where they were in a task, repeated completed work, and became less reliable across repeated runs.
ARCBench #
This month, additional insight arrives from the Netherlands in the form of ARCBench, a study finding that agents can emerge from compaction with the rule intact and still become substantially more likely to break it. In tests where the rule did survive, violations were 2.6 times more frequent after compaction than with the full conversation history:
The study makes a distinction between rules that were lost during compaction, and rules that survived and were subsequently violated – allowing the effects of forgetting and disobedience to be measured independently for the first time.
Across tests with DeepSeek-V4-Flash, GLM-5.3-Flash, Qwen3.7-Flash and GPT-Luna, 70.1% of compacted runs retained the complete rule. Among runs where the rule survived, violations occurred 2.6 times more often than with the full conversation history.
Context, in the traditional sense, proved a factor, since rules presented as organizational policy produced no violations with full context; yet violation rates rose to 30.6% and 22.2% after compaction.
The strongest defense available preserved the rules in 99% of summaries, but still failed to return violations to their pre-attack level, indicating that successful preservation of the rules during compaction doesn’t necessarily preserve the agent’s ability – or willingness – to follow them.
The paper states:
‘[We] find that compaction raises violations mostly without deleting the rule: compacted runs retaining the rule still violate 2.6 times as often as runs with the full history.
‘Moreover, compaction changes the agent’s reasoning for violating. Different compaction strategies, tight compaction budget constraints, and rule delivery channel all affect compliance.’
By way of a remedy, the paper proposes stronger rule preservation for rules lost during compaction, while leaving the problem of agents disobeying retained rules unresolved”
‘Defenses restore rules, but compliance is still weakened under attack. Rule retention is thus not a sufficient proxy for compliance. Future studies may use ARCBench to explain why agents still violate retained rules, test adaptive attacks and defenses, and cover more models, domains, and longer sessions with repeated compaction.’
Method #
ARCBench extends the functionality of the pi open source harness, which has its own native compaction routines, as well as an agent loop, tool calling and native compaction. The benchmark recreates situations where an agent has been working through a long task under a standing rule, compresses the accumulated history, and then confronts the agent with a request that would violate that rule; the agent’s actions are then checked through its tool calls:
ARCBench contains 17 cases covering disclosure of protected data, access to restricted resources, unauthorized actions, and bypass or omission of required controls, corresponding to OWASP risk classes.
Cases were adapted from Governance Decay and COMPINT , with four additional cases introduced to ensure that all four violation types were represented for both personal and organizational rules.
The benchmark can be configured across five factors: how the rule is delivered; the method used to compact the context; the token budget available for compaction; the type of attack; and the defense applied:
Rules can be given as direct or persistent instructions, embedded in documents, or retrieved through a policy tool.
Data and Tests #
Ten compaction methods were evaluated, comprising seven generative strategies and three selective approaches. Each case was run repeatedly, and a failure recorded if the agent violated its rule at least once – an approach considered more appropriate for safety testing than averaging performance across runs, since an agent that occasionally performs a prohibited action remains unsafe.
The compacted histories were then examined by GPT-6-Luna-Pro, Qwen-3.8-flash and Claude-Sonnet-4.6, with each rule classified as intact, weakened or absent. The majority decision was used, while disagreements were inspected by human annotators, with a separate sample of 50 summaries also manually labeled, to check the judges’ consistency.
When an intact rule was violated, the agent’s reasoning was examined by GPT-6-Luna-Pro, GPT-6.1-Sol and Claude Sonnet 5.5. Violations were classified according to whether the later request had been treated as permission to override the rule, the action had been judged outside its scope; the rule had gone unmentioned; or the rule had been recognized and disregarded.
The four models used for the tests, accessed through OpenRouter, were DeepSeek-V4-Flash; GLM-5.3-Flash; Qwen3.7-Flash; and GPT-Luna. In each run, the same model was used both to compact the conversation history and to act on the resulting context, matching pi’s default behavior. The models were accessed through OpenRouter.
Seven studies were conducted, with Hermes used for most of the compaction tests. In the first study, each of the 17 tasks was tested across four models and two ways of delivering the rule, producing 136 cases. Each was run three times with the original conversation intact and three times after it had been compacted.
Three questions were addressed in the tests: Where do violations after compaction come from?; How do different compaction configurations change where violations come from?; and How do compactor-targeted attacks and defenses affect where violations come from?.
Where Do Violations after Compaction Come From?
The first results showed a rise in rule-breaking after compaction, from 9.6% of cases with the full history, to 37.8%. However, the rule was completely lost in only 3% of compacted runs; in 70.1% it survived intact, and in 26.9% it survived partially:
This distinction proved important. Among individual runs where the full rule survived compaction, 21.1% still ended in a violation, compared with 8.1% when the original history was retained. With a partially preserved rule, the rate rose to 29.4%, and with no rule remaining, to 83.3%.
How violations occurred when the rule remained intact was also examined. Among violations caused by compaction, a later request was interpreted as permission to proceed in 45.9% of cases; the rule was apparently disregarded in 27%; and in 21.6%, it was acknowledged as applicable before being violated anyway:
How Do Different Compaction Configurations Change Where Violations Come From?
The way a rule was delivered had a marked effect on compliance. With the full conversation intact, violations occurred in 25% and 16.1% of cases where rules had been supplied personally by the user; however, no violations were recorded when rules had instead been delivered through an organizational document or policy tool:
After compaction, violations rose across all four channels, reaching 56.3% for direct user instructions and 22.2% for policy-tool rules. Rule retention varied comparatively little, from 65.7% to 74%. The authors suggest that a later user request can be interpreted as overriding a personal rule, while an organizational rule cannot be overridden by the user.
How Do Compactor-Targeted Attacks and Defenses Affect Where Violations Come From?
Simply flooding the context with extra material had little effect, with violations actually falling from 37.8% to 34.3%. Compactor-targeted injection was considerably more effective, raising violations to 46.3% and reducing full rule retention from 70.1% to 42.5%.
The strongest attack, called escape, increased violations from 23.5% to 58.8% and reduced rule retention from 96.1% to 66.7%. Even when the rule survived, violations doubled from 13.3% to 26.5%.
The strongest defense preserved the rule in 99% of runs – yet agents still broke it in 29.4% of cases, compared with 23.5% when no attack was used. In other words, making sure an agent still had the rule did not ensure that it would follow it.
Conclusion #
Logic suggests that there is some kind of ‘photocopier effect’ at work when context is compacted dozens, hundreds of times in a very long thread (such as one that I have been working on in Codex* since mid-October, and was working on only this morning – an epic of experimental, AI-aided software development).
Like core system prompts, we don’t get to see these compacted synopses of the action to date. My own anecdotal evidence suggests that the central mission survives the compression, along with most of the essential details. But then, I’m not in that nightmare scenario where the command ‘Send this email to everyone’, intended just for everyone in the company, ends up going to everyone in the aggregated corporate contacts book. Yowch.
- I guess I’m getting old, but this is surely the most unmemorable, bland, ‘corporate-beige’ name that could have possibly been chosen for this agent. I am constantly forgetting it. First published Thursday, November 12, 2026