AutoGen's hidden token tax: why a 3-agent chat costs 15 what you expect

Microsoft's AutoGen multi-agent framework has a hidden token tax that causes costs to scale quadratically with the number of turns, not linearly. An audit reveals that each agent's default unbounded memory accumulates every prior message, so a 3-agent chat with 10 turns costs roughly 15× what a naive estimate would suggest. 'The N cancels. Total cost scales as T² regardless of how many agents you add,' the developer notes.

Cost-audit series, episode 2. This series began with an AI agent that burned 136M tokens overnight →. AutoGen is Microsoft's multi-agent framework. It's genuinely good at orchestrating agents that hand off work to each other. But its default memory model has a cost shape that surprises almost every team that hits it in production. This audit shows you exactly where the tokens go, with line numbers. The canonical AutoGen pattern is a RoundRobinGroupChat with N agents taking turns on a task. Here's the minimal version from the docs: python from autogen agentchat.agents import AssistantAgent from autogen agentchat.teams import RoundRobinGroupChat from autogen agentchat.conditions import MaxMessageTermination planner = AssistantAgent "planner", model client=client, system message="You plan." coder = AssistantAgent "coder", model client=client, system message="You code." reviewer = AssistantAgent "reviewer", model client=client, system message="You review." team = RoundRobinGroupChat planner, coder, reviewer , termination condition=MaxMessageTermination max messages=10 , await team.run task="Build a web scraper for Hacker News." Three agents, 10 turns total ~3–4 turns each . Seems cheap. It isn't. Every AssistantAgent gets its own UnboundedChatCompletionContext by default: autogen-agentchat/src/autogen agentchat/agents/ assistant agent.py, init L708 if model context is not None: self. model context = model context else: self. model context = UnboundedChatCompletionContext UnboundedChatCompletionContext.get messages returns self. messages — the full list, no cap, no truncation: autogen-core/.../model context/ unbounded chat completion context.py a ~20-line file async def get messages self - List LLMMessage : """Get at most buffer size recent messages.""" return self. messages The docstring says "at most buffer size " — that's a copy-paste artifact from BufferedChatCompletionContext . There is no buffer. It returns everything. When an agent's turn arrives, on messages stream adds all incoming messages to its own context before calling the LLM: assistant agent.py, in on messages stream STEP 1: "Add new user/handoff messages to the model context" await self. add messages to context model context=model context, messages=messages, ← the full message thread from the group manager ... And add messages to context appends each one: assistant agent.py, static method add messages to context await model context.add message llm msg ... await model context.add message msg.to model message The group manager BaseGroupChatManager maintains a single message thread and appends every response to it: base group chat manager.py self. message thread: List BaseAgentEvent | BaseChatMessage = ... await self.update message thread delta called after every agent response So at turn T, the agent receiving the baton gets T-1 messages added to its already-growing context. Its context now contains everything it has ever seen. Let's be precise. Define: Each agent speaks every N turns. When agent i speaks on turn t , its context contains all t-1 prior messages because it has been accumulating them since turn 1 . Tokens consumed by agent i on turn t: context tokens t = t - 1 × m Total tokens for agent i across all its turns it speaks at turns N, 2N, 3N, … up to T : Σ kN - 1 × m for k = 1 to T/N ≈ m × N × T/N ² / 2 = m × T² / 2N Total tokens across all N agents: N × m × T² / 2N = m × T² / 2 The N cancels. Total cost scales as T² regardless of how many agents you add. | Turn | Agent | Context size messages | Tokens in this call | |---|---|---|---| | 1 | planner | 0 prior + system | ~300 | | 2 | coder | 1 prior + system | ~600 | | 3 | reviewer | 2 prior + system | ~900 | | 4 | planner | 3 prior + system | ~1,200 | | 5 | coder | 4 prior + system | ~1,500 | | 6 | reviewer | 5 prior + system | ~1,800 | | 7 | planner | 6 prior + system | ~2,100 | | 8 | coder | 7 prior + system | ~2,400 | | 9 | reviewer | 8 prior + system | ~2,700 | | 10 | planner | 9 prior + system | ~3,000 | Total | ~16,500 tokens | Naïve expectation 10 calls × 300 tokens each : 3,000 tokens Actual : ~16,500 tokens — 5.5× more . At 20 turns it's ~63,000 tokens vs 6,000 expected — 10.5× more . At 30 turns: ~139,500 tokens vs 9,000 — 15.5× more . The multiplier grows linearly with T. This is the same O T² shape as ConversationBufferMemory in LangChain — but AutoGen's version is per-agent , so it's easy to miss in per-call logs. If you're watching your LLM provider's per-call token counts, you see something like: call 1: 300 tokens ✓ cheap call 2: 600 tokens ✓ fine call 3: 900 tokens ✓ ok ... call 10: 3,000 tokens ← this one looks expensive Each call looks like a modest increase. The cumulative total — 16,500 — only shows up when you sum across the run. Most observability dashboards show per-call costs, not per-run totals. The runaway is invisible until the bill arrives. AutoGen ships two bounded alternatives, named in the AssistantAgent. init docstring around L1034 of assistant agent.py : BufferedChatCompletionContext limits message count and TokenLimitedChatCompletionContext limits tokens : BufferedChatCompletionContext sliding window python from autogen core.model context import BufferedChatCompletionContext coder = AssistantAgent "coder", model client=client, model context=BufferedChatCompletionContext buffer size=5 , last 5 messages Cost shape becomes O T × buffer size — linear. For buffer size=5 and 30 turns: ~42,000 tokens vs 139,500 unbounded. 3.3× cheaper. TokenLimitedChatCompletionContext token budget python from autogen core.model context import TokenLimitedChatCompletionContext coder = AssistantAgent "coder", model client=client, model context=TokenLimitedChatCompletionContext token limit=2000 , Caps the context at a fixed token budget. More predictable than a message count because message sizes vary. | Scenario | Recommendation | |---|---| | Short tasks ≤10 turns | Default is fine; monitor cumulative cost | | Long tasks 10 turns | BufferedChatCompletionContext buffer size=8–12 | | Strict cost budget | TokenLimitedChatCompletionContext token limit=N | | Need full history | Default + add per-run cost alerting see below | The pattern is detectable statically: any file that instantiates AssistantAgent without a model context= argument is using the unbounded default. Flag unbounded AssistantAgent instantiations grep -rn "AssistantAgent " src/ | grep -v "model context=" For dynamic detection — measuring actual token growth across a run — this is exactly what tokenscope https://www.npmjs.com/package/@wartzar-bee/tokenscope does: it instruments LLM calls, tracks per-run cumulative cost, and can block a CI build when a PR's token delta exceeds a threshold. The wartzar-bee/ci-guardrail https://github.com/wartzar-bee/ci-guardrail GitHub Action wraps tokenscope into a one-line workflow addition: - uses: wartzar-bee/ci-guardrail@v1 with: token threshold: 50000 block if PR adds 50k tokens/run github token: ${{ secrets.GITHUB TOKEN }} | Naïve expectation | Actual unbounded | With BufferedContext 5 | | |---|---|---|---| | 10 turns, 3 agents | 3,000 tokens | ~16,500 tokens | ~12,000 tokens | | 20 turns, 3 agents | 6,000 tokens | ~63,000 tokens | ~27,000 tokens | | 30 turns, 3 agents | 9,000 tokens | ~139,500 tokens | ~42,000 tokens | The default UnboundedChatCompletionContext is correct for short tasks and full-history use cases. It becomes a cost trap in long multi-agent conversations. The fix is one constructor argument — but you have to know to add it. The broader pattern : every major agent framework defaults to unbounded context because it's the safest correctness choice. Cost is a second-class citizen in the default config. That's the gap this series documents. Next in the series: CrewAI — the delegation overhead. How hierarchical agent trees multiply your token bill. tokenscope on npm https://www.npmjs.com/package/@wartzar-bee/tokenscope · wartzar-bee/ci-guardrail https://github.com/wartzar-bee/ci-guardrail · @wartzarbee on dev.to https://dev.to/wartzarbee