{"slug": "autogen-s-hidden-token-tax-why-a-3-agent-chat-costs-15-what-you-expect", "title": "AutoGen's hidden token tax: why a 3-agent chat costs 15 what you expect", "summary": "Microsoft's AutoGen multi-agent framework has a hidden token tax that causes costs to scale quadratically with the number of turns, not linearly. An audit reveals that each agent's default unbounded memory accumulates every prior message, so a 3-agent chat with 10 turns costs roughly 15× what a naive estimate would suggest. 'The N cancels. Total cost scales as T² regardless of how many agents you add,' the developer notes.", "body_md": "*Cost-audit series, episode 2. This series began with an AI agent that burned 136M tokens overnight →.*\n\nAutoGen is Microsoft's multi-agent framework. It's genuinely good at orchestrating agents that hand off work to each other. But its default memory model has a cost shape that surprises almost every team that hits it in production.\n\nThis audit shows you exactly where the tokens go, with line numbers.\n\nThe canonical AutoGen pattern is a `RoundRobinGroupChat`\n\nwith N agents taking turns on a task. Here's the minimal version from the docs:\n\n``` python\nfrom autogen_agentchat.agents import AssistantAgent\nfrom autogen_agentchat.teams import RoundRobinGroupChat\nfrom autogen_agentchat.conditions import MaxMessageTermination\n\nplanner  = AssistantAgent(\"planner\",  model_client=client, system_message=\"You plan.\")\ncoder    = AssistantAgent(\"coder\",    model_client=client, system_message=\"You code.\")\nreviewer = AssistantAgent(\"reviewer\", model_client=client, system_message=\"You review.\")\n\nteam = RoundRobinGroupChat(\n    [planner, coder, reviewer],\n    termination_condition=MaxMessageTermination(max_messages=10),\n)\nawait team.run(task=\"Build a web scraper for Hacker News.\")\n```\n\nThree agents, 10 turns total (~3–4 turns each). Seems cheap. It isn't.\n\nEvery `AssistantAgent`\n\ngets its own `UnboundedChatCompletionContext`\n\nby default:\n\n```\n# autogen-agentchat/src/autogen_agentchat/agents/_assistant_agent.py, __init__ (L708)\nif model_context is not None:\n    self._model_context = model_context\nelse:\n    self._model_context = UnboundedChatCompletionContext()\n```\n\n`UnboundedChatCompletionContext.get_messages()`\n\nreturns `self._messages`\n\n— the full list, no cap, no truncation:\n\n```\n# autogen-core/.../model_context/_unbounded_chat_completion_context.py (a ~20-line file)\nasync def get_messages(self) -> List[LLMMessage]:\n    \"\"\"Get at most `buffer_size` recent messages.\"\"\"\n    return self._messages\n```\n\n(The docstring says \"at most `buffer_size`\n\n\" — that's a copy-paste artifact from `BufferedChatCompletionContext`\n\n. There is no buffer. It returns everything.)\n\nWhen an agent's turn arrives, `on_messages_stream`\n\nadds all incoming messages to its own context before calling the LLM:\n\n```\n# _assistant_agent.py, in on_messages_stream (STEP 1: \"Add new user/handoff messages\n# to the model context\")\nawait self._add_messages_to_context(\n    model_context=model_context,\n    messages=messages,          # ← the full message_thread from the group manager\n    ...\n)\n```\n\nAnd `_add_messages_to_context`\n\nappends each one:\n\n```\n# _assistant_agent.py, static method _add_messages_to_context\nawait model_context.add_message(llm_msg)\n...\nawait model_context.add_message(msg.to_model_message())\n```\n\nThe group manager (`BaseGroupChatManager`\n\n) maintains a single `_message_thread`\n\nand appends every response to it:\n\n```\n# _base_group_chat_manager.py\nself._message_thread: List[BaseAgentEvent | BaseChatMessage] = []\n...\nawait self.update_message_thread(delta)   # called after every agent response\n```\n\nSo at turn T, the agent receiving the baton gets `T-1`\n\nmessages added to its already-growing context. Its context now contains everything it has ever seen.\n\nLet's be precise. Define:\n\nEach agent speaks every N turns. When agent *i* speaks on turn *t*, its context contains all *t-1* prior messages (because it has been accumulating them since turn 1).\n\n**Tokens consumed by agent i on turn t:**\n\n```\ncontext_tokens(t) = (t - 1) × m\n```\n\n**Total tokens for agent i across all its turns** (it speaks at turns N, 2N, 3N, … up to T):\n\n```\nΣ (kN - 1) × m  for k = 1 to T/N\n≈ m × N × (T/N)² / 2\n= m × T² / (2N)\n```\n\n**Total tokens across all N agents:**\n\n```\nN × m × T² / (2N) = m × T² / 2\n```\n\nThe N cancels. Total cost scales as **T²** regardless of how many agents you add.\n\n| Turn | Agent | Context size (messages) | Tokens in this call |\n|---|---|---|---|\n| 1 | planner | 0 prior + system | ~300 |\n| 2 | coder | 1 prior + system | ~600 |\n| 3 | reviewer | 2 prior + system | ~900 |\n| 4 | planner | 3 prior + system | ~1,200 |\n| 5 | coder | 4 prior + system | ~1,500 |\n| 6 | reviewer | 5 prior + system | ~1,800 |\n| 7 | planner | 6 prior + system | ~2,100 |\n| 8 | coder | 7 prior + system | ~2,400 |\n| 9 | reviewer | 8 prior + system | ~2,700 |\n| 10 | planner | 9 prior + system | ~3,000 |\nTotal |\n~16,500 tokens |\n\n**Naïve expectation** (10 calls × 300 tokens each): **3,000 tokens**\n\n**Actual**: **~16,500 tokens** — **5.5× more**.\n\nAt 20 turns it's **~63,000 tokens** vs 6,000 expected — **10.5× more**.\n\nAt 30 turns: **~139,500 tokens** vs 9,000 — **15.5× more**.\n\nThe multiplier grows linearly with T. This is the same O(T²) shape as `ConversationBufferMemory`\n\nin LangChain — but AutoGen's version is *per-agent*, so it's easy to miss in per-call logs.\n\nIf you're watching your LLM provider's per-call token counts, you see something like:\n\n```\ncall 1:  300 tokens  ✓ cheap\ncall 2:  600 tokens  ✓ fine\ncall 3:  900 tokens  ✓ ok\n...\ncall 10: 3,000 tokens  ← this one looks expensive\n```\n\nEach call looks like a modest increase. The *cumulative* total — 16,500 — only shows up when you sum across the run. Most observability dashboards show per-call costs, not per-run totals. The runaway is invisible until the bill arrives.\n\nAutoGen ships two bounded alternatives, named in the `AssistantAgent.__init__`\n\ndocstring (around L1034 of `_assistant_agent.py`\n\n): `BufferedChatCompletionContext`\n\n(limits message count) and `TokenLimitedChatCompletionContext`\n\n(limits tokens):\n\n`BufferedChatCompletionContext`\n\n(sliding window)\n\n``` python\nfrom autogen_core.model_context import BufferedChatCompletionContext\n\ncoder = AssistantAgent(\n    \"coder\",\n    model_client=client,\n    model_context=BufferedChatCompletionContext(buffer_size=5),  # last 5 messages\n)\n```\n\nCost shape becomes **O(T × buffer_size)** — linear. For buffer_size=5 and 30 turns: ~42,000 tokens vs 139,500 unbounded. **3.3× cheaper.**\n\n`TokenLimitedChatCompletionContext`\n\n(token budget)\n\n``` python\nfrom autogen_core.model_context import TokenLimitedChatCompletionContext\n\ncoder = AssistantAgent(\n    \"coder\",\n    model_client=client,\n    model_context=TokenLimitedChatCompletionContext(token_limit=2000),\n)\n```\n\nCaps the context at a fixed token budget. More predictable than a message count because message sizes vary.\n\n| Scenario | Recommendation |\n|---|---|\n| Short tasks (≤10 turns) | Default is fine; monitor cumulative cost |\n| Long tasks (>10 turns) | `BufferedChatCompletionContext(buffer_size=8–12)` |\n| Strict cost budget | `TokenLimitedChatCompletionContext(token_limit=N)` |\n| Need full history | Default + add per-run cost alerting (see below) |\n\nThe pattern is detectable statically: any file that instantiates `AssistantAgent`\n\nwithout a `model_context=`\n\nargument is using the unbounded default.\n\n```\n# Flag unbounded AssistantAgent instantiations\ngrep -rn \"AssistantAgent(\" src/ | grep -v \"model_context=\"\n```\n\nFor dynamic detection — measuring actual token growth across a run — this is exactly what [tokenscope](https://www.npmjs.com/package/@wartzar-bee/tokenscope) does: it instruments LLM calls, tracks per-run cumulative cost, and can block a CI build when a PR's token delta exceeds a threshold.\n\nThe [wartzar-bee/ci-guardrail](https://github.com/wartzar-bee/ci-guardrail) GitHub Action wraps tokenscope into a one-line workflow addition:\n\n```\n- uses: wartzar-bee/ci-guardrail@v1\n  with:\n    token_threshold: 50000   # block if PR adds >50k tokens/run\n    github_token: ${{ secrets.GITHUB_TOKEN }}\n```\n\n| Naïve expectation | Actual (unbounded) | With BufferedContext(5) | |\n|---|---|---|---|\n| 10 turns, 3 agents | 3,000 tokens | ~16,500 tokens | ~12,000 tokens |\n| 20 turns, 3 agents | 6,000 tokens | ~63,000 tokens | ~27,000 tokens |\n| 30 turns, 3 agents | 9,000 tokens | ~139,500 tokens | ~42,000 tokens |\n\nThe default `UnboundedChatCompletionContext`\n\nis correct for short tasks and full-history use cases. It becomes a cost trap in long multi-agent conversations. The fix is one constructor argument — but you have to know to add it.\n\n**The broader pattern**: every major agent framework defaults to unbounded context because it's the safest correctness choice. Cost is a second-class citizen in the default config. That's the gap this series documents.\n\n*Next in the series: CrewAI — the delegation overhead. How hierarchical agent trees multiply your token bill.*\n\n[tokenscope on npm](https://www.npmjs.com/package/@wartzar-bee/tokenscope) · [wartzar-bee/ci-guardrail](https://github.com/wartzar-bee/ci-guardrail) · [@wartzarbee on dev.to](https://dev.to/wartzarbee)", "url": "https://wpnews.pro/news/autogen-s-hidden-token-tax-why-a-3-agent-chat-costs-15-what-you-expect", "canonical_source": "https://dev.to/wartzarbee/autogens-hidden-token-tax-why-a-3-agent-chat-costs-15x-what-you-expect-21o5", "published_at": "2026-07-23 00:03:55+00:00", "updated_at": "2026-07-23 00:59:40.714553+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-infrastructure"], "entities": ["Microsoft", "AutoGen"], "alternates": {"html": "https://wpnews.pro/news/autogen-s-hidden-token-tax-why-a-3-agent-chat-costs-15-what-you-expect", "markdown": "https://wpnews.pro/news/autogen-s-hidden-token-tax-why-a-3-agent-chat-costs-15-what-you-expect.md", "text": "https://wpnews.pro/news/autogen-s-hidden-token-tax-why-a-3-agent-chat-costs-15-what-you-expect.txt", "jsonld": "https://wpnews.pro/news/autogen-s-hidden-token-tax-why-a-3-agent-chat-costs-15-what-you-expect.jsonld"}}