{"slug": "pydantic-ai-keeps-one-growing-message-list-per-run-and-re-sends-the-whole-thing", "title": "Pydantic AI keeps one growing message list per run — and re-sends the whole thing every step", "summary": "A developer's analysis of Pydantic AI's run loop reveals that the framework maintains a single mutable conversation list per run and re-sends the entire accumulated history—including tool outputs—on every model step, leading to O(n²) cumulative token usage. The developer recommends measuring and pricing real runs, and introduces open-source tools tokenscope and ci-guardrail to quantify and gate token costs in CI.", "body_md": "Pydantic AI gives you a clean, typed agent: define an `Agent`\n\n, hand it tools, call `agent.run(...)`\n\n, and it loops — model call, tool call, model call — until it produces a validated result. The typed ergonomics are great. What the quickstart doesn't spell out is *what the model receives on each pass of that loop.*\n\nI read the run graph (`pydantic_ai_slim/pydantic_ai/_agent_graph.py`\n\non `main`\n\n) to find out. The mechanism is structural, and it's the same shape I found in the OpenAI Agents SDK and smolagents.\n\nEach run holds a single mutable conversation list on its state:\n\n```\nmessage_history: list[_messages.ModelMessage] = dataclasses.field(default_factory=list[_messages.ModelMessage])\n```\n\nOn every model step the graph appends to it — first the outgoing request, then the model's response:\n\n```\nctx.state.message_history.append(self.request)\n...\nctx.state.message_history.append(response)\n```\n\nNothing is removed. The list only grows: request, response, request, response — with tool calls and, crucially, **tool outputs** riding inside those messages.\n\nWhen the graph builds the input for the next model call, it takes the entire accumulated history — a full copy:\n\n```\nmessages = ctx.state.message_history[:]\n...\nmessages[:] = _clean_message_history(ctx.state.message_history)\n```\n\nThat `[:]`\n\nis the whole conversation to date. So on step 1 the model sees your prompt; on step 2 it sees your prompt + step 1's request + step 1's response (including the tool output); on step 5 it sees all of that plus steps 2–4. The payload you pay for grows every single step, and the heaviest passengers are usually the tool outputs — the search results, file contents, and API responses you least want re-uploaded five times.\n\nA run of *n* steps sends roughly `1 + 2 + 3 + … + n`\n\ncopies of history — **O(n²) cumulative tokens** in the step count. A 3-step agent is fine. A 12-step agent that reads a couple of files is not: each file's contents rides along on every later step. The run still *succeeds*, your tests still pass — the only artifact is a bigger number on the usage line, and you don't see it until the invoice.\n\nPydantic AI hands you the full transcript back (`result.all_messages()`\n\n) and every run method takes a `message_history`\n\nparameter:\n\n```\nmessage_history: Sequence[_messages.ModelMessage] | None = None\n```\n\n— *\"History of the conversation so far.\"* That's the lever: **across** a multi-turn conversation you decide what prior history to replay, so you can pass a trimmed or summarized history into the next run instead of the raw accumulation. Inside a single deep tool-loop the re-send is inherent to how tool-calling works (it's true of every framework) — which is exactly why the move is to *measure* it, not assume it's free.\n\nBefore refactoring anything, put a number on it — the token cost of your run, priced in dollars, not tokens. That's what [ @wartzar-bee/tokenscope](https://www.npmjs.com/package/@wartzar-bee/tokenscope) does (\n\n`npm i @wartzar-bee/tokenscope`\n\n): it takes real usage and prices each bucket — input, output, cache-write (~1.25×), cache-read (~0.1×) — into an actual per-run dollar figure, so \"the 12-step version costs 4× the 4-step version\" stops being a hunch.And if this runs in CI, gate it: [ wartzar-bee/ci-guardrail](https://github.com/wartzar-bee/ci-guardrail) is an Apache-2.0 GitHub Action (built on tokenscope) that fails the check when a run crosses an absolute\n\n`max-usd`\n\nceiling — so the quadratic step doesn't reach production as a silent 4× before anyone notices.\n\n```\n- uses: wartzar-bee/ci-guardrail@v1\n  with:\n    max-usd: \"0.50\"\n```\n\nIf you run Pydantic AI: how many steps does your deepest agent take, and how big are the tool outputs it re-sends on every one? Worth pricing one real run before the next invoice does it for you.", "url": "https://wpnews.pro/news/pydantic-ai-keeps-one-growing-message-list-per-run-and-re-sends-the-whole-thing", "canonical_source": "https://dev.to/wartzarbee/pydantic-ai-keeps-one-growing-message-list-per-run-and-re-sends-the-whole-thing-every-step-4o7b", "published_at": "2026-08-22 12:37:48+00:00", "updated_at": "2026-08-22 13:14:19.502889+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure", "ai-agents", "mlops"], "entities": ["Pydantic AI", "OpenAI Agents SDK", "smolagents", "tokenscope", "ci-guardrail", "GitHub Actions"], "alternates": {"html": "https://wpnews.pro/news/pydantic-ai-keeps-one-growing-message-list-per-run-and-re-sends-the-whole-thing", "markdown": "https://wpnews.pro/news/pydantic-ai-keeps-one-growing-message-list-per-run-and-re-sends-the-whole-thing.md", "text": "https://wpnews.pro/news/pydantic-ai-keeps-one-growing-message-list-per-run-and-re-sends-the-whole-thing.txt", "jsonld": "https://wpnews.pro/news/pydantic-ai-keeps-one-growing-message-list-per-run-and-re-sends-the-whole-thing.jsonld"}}