{"slug": "my-ai-agent-kept-compressing-the-same-conversation-here-s-how-i-fixed-the-anti", "title": "My AI Agent Kept Compressing the Same Conversation. Here's How I Fixed the Anti-Thrashing Bug.", "summary": "A developer fixed an anti-thrashing bug in their AI agent's context compression logic. The in-memory counter that blocked repeated useless compressions reset on process restart, causing the agent to re-compress already compacted conversations. The fix persisted the counter to a SQLite-backed session state, mirroring the pattern used for other durable counters in the codebase.", "body_md": "I noticed something weird. Every time my AI agent's process restarted, it would compress the conversation again — even though the last five compressions had been useless.\n\nThe prompt was already clean. The system prompt and tool schemas added up to 30K tokens of incompressible floor. Shrinking the message history wasn't going to help. But the agent didn't know that. It ran the compaction loop every single time, burning tokens and time for zero benefit.\n\nThe root cause? A single in-memory counter that disappeared on restart.\n\nContext compression is how long-running AI agent conversations stay under the model's context window. You have a threshold — say 50K tokens. When the provider reports the prompt approaching that limit, you compact: summarize old messages into a single system-level entry, drop the full history, and keep rolling.\n\nSimple, right?\n\nBut there's a corner case: when the system prompt and tool schemas alone are already close to the threshold, compaction can't help. The message history shrinks, sure, but the total prompt stays over the line. So the next turn triggers compaction again. And again. And again.\n\nTo stop this, I added a thrashing guard — a counter that ticks up each time compaction runs but doesn't clear the threshold. After two ineffective rounds, it blocks further compression. Clean.\n\n```\n# The anti-thrashing guard (before the fix)\nclass CompressionState:\n    def __init__(self):\n        # In-memory only — vanishes on restart\n        self._ineffective_compression_count = 0\n\n    def update_from_response(self, usage):\n        \"\"\"Called after each API response with real token counts.\"\"\"\n        if self._verify_compaction_cleared_threshold:\n            if self.last_prompt_tokens >= self.threshold_tokens:\n                self._ineffective_compression_count += 1\n            else:\n                self._ineffective_compression_count = 0\n\n    def should_compress(self):\n        \"\"\"Gate check — blocks compression after 2 strikes.\"\"\"\n        return self._ineffective_compression_count < 2\n```\n\nThe guard works perfectly in a single session. But it lives in memory. When the process restarts — deployment, crash recovery, even a graceful restart — the counter resets to zero. The guard disarms. And the agent compresses the already-compacted conversation one more time.\n\nThe pattern was already in the codebase for two other counters: the compression failure cooldown (which prevents retrying a failing provider) and the fallback streak (which tracks how many deterministic fallback summaries were inserted in a row). Both persisted through a durable session-state channel backed by SQLite.\n\nI applied the same pattern to the anti-thrashing counter. Three pieces:\n\n**1. A database column.** I added `compression_ineffective_count`\n\nto the sessions table, with accessor methods that return the value or write it back.\n\n```\n# hermes_state.py — persistent counter accessors\ndef get_compression_ineffective_count(self, session_id: str) -> int:\n    row = self._conn.execute(\n        \"SELECT compression_ineffective_count FROM sessions WHERE session_id = ?\",\n        (session_id,)\n    ).fetchone()\n    return row[0] if row else 0\n\ndef set_compression_ineffective_count(self, session_id: str, count: int) -> None:\n    self._conn.execute(\n        \"UPDATE sessions SET compression_ineffective_count = ? WHERE session_id = ?\",\n        (count, session_id)\n    )\n```\n\n**2. A centralized verdict recorder.** Every time the `update_from_response()`\n\nmethod decides whether the last compaction was effective or not, it routes through `_record_ineffective_compression_verdict()`\n\n. This method updates both the in-memory counter and the database row — atomically, in the same code path.\n\n``` python\ndef _record_ineffective_compression_verdict(self, was_ineffective: bool):\n    if was_ineffective:\n        self._ineffective_compression_count += 1\n    else:\n        self._ineffective_compression_count = 0\n\n    # Persist through the same channel used by every other durable guard\n    setter = getattr(self._session_db, \"set_compression_ineffective_count\", None)\n    if callable(setter):\n        try:\n            setter(self._session_id, self._ineffective_compression_count)\n        except Exception as exc:\n            logger.debug(\"persist ineffective count failed: %s\", exc)\n```\n\n**3. Load on bind.** When the compressor binds to a resumed session, it reads the persisted counter back into memory.\n\n``` python\ndef bind_session_state(self, session_db, session_id):\n    self._session_db = session_db\n    self._session_id = session_id\n    self._ineffective_compression_count = 0  # fallback\n\n    # Load the durable count\n    getter = getattr(session_db, \"get_compression_ineffective_count\", None)\n    if callable(getter):\n        try:\n            stored = getter(session_id)\n            self._ineffective_compression_count = max(0, int(stored))\n        except Exception:\n            pass\n```\n\nThat's it. The bug only existed because the codebase had two different patterns — one durable (cooldown, fallback streak) and one in-memory (anti-thrashing counter). The fix wasn't new architecture. It was making the third counter use the same durable channel as the first two.\n\nThe key insight is that **a session is a durable entity, but its compression state was ephemeral**. Every other part of the session — messages, config, metadata — lives in SQLite. The anti-thrashing counter was the odd one out.\n\nBy moving it into the same persistent channel, three things happen automatically:\n\nThe reset semantics didn't change, either. Any real provider response that reads below the threshold still clears the counter — and now it clears durably.\n\n**Don't persist no-change verdicts.** If the counter didn't change — say, a response came in but `_verify_compaction_cleared_threshold`\n\nwas False (meaning no compaction happened this cycle) — skip the DB write. Two reasons: it's wasted I/O, and it creates a false positive \"someone touched this\" signal in the session row's modification time.\n\n**Mind the null-coalescing.** A session row created before the column existed will return `None`\n\n, not `0`\n\n. Always `max(0, int(stored))`\n\nto avoid `TypeError`\n\non a fresh `None`\n\n.\n\n**The guard must check the in-memory value first, then refresh.** `should_compress()`\n\nruns on the hot path — every turn start. The in-memory copy covers the fast path. If it says \"blocked\", use the durable fallback and skip the DB read. Only refresh from the database when the in-memory copy is zero (disarmed) and you need to prove another agent didn't arm it since `bind_session_state()`\n\n.\n\n**Don't remove the in-memory copy just because the DB is durable.** Writing to SQLite on every response is fine (it's a single-row UPDATE on an indexed primary key), but reading from it on `should_compress()`\n\nwhen you already have the value in RAM is wasteful. Keep both, write through, read from RAM.\n\nHave you ever found a bug that was just \"this value is in memory but everything else is in the database\"? I'd love to hear what patterns you use to prevent state fragmentation in agent systems — especially if you're dealing with context windows and compression. Drop your story in the comments.", "url": "https://wpnews.pro/news/my-ai-agent-kept-compressing-the-same-conversation-here-s-how-i-fixed-the-anti", "canonical_source": "https://dev.to/chenyuan20509/my-ai-agent-kept-compressing-the-same-conversation-heres-how-i-fixed-the-anti-thrashing-bug-3ok2", "published_at": "2026-07-25 11:41:37+00:00", "updated_at": "2026-07-25 12:01:22.905500+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "developer-tools"], "entities": ["SQLite"], "alternates": {"html": "https://wpnews.pro/news/my-ai-agent-kept-compressing-the-same-conversation-here-s-how-i-fixed-the-anti", "markdown": "https://wpnews.pro/news/my-ai-agent-kept-compressing-the-same-conversation-here-s-how-i-fixed-the-anti.md", "text": "https://wpnews.pro/news/my-ai-agent-kept-compressing-the-same-conversation-here-s-how-i-fixed-the-anti.txt", "jsonld": "https://wpnews.pro/news/my-ai-agent-kept-compressing-the-same-conversation-here-s-how-i-fixed-the-anti.jsonld"}}