{"slug": "your-mcp-agent-re-sends-7000-tokens-of-tool-schemas-on-every-turn", "title": "Your MCP agent re-sends 7,000 tokens of tool schemas on every turn", "summary": "A developer wired three MCP servers into an agent and found that tool schemas consume about 7,000 tokens per turn, and because LLM API calls are stateless, those tokens are re-sent on every turn, leading to a 40-turn session paying for schemas forty times. The developer proposes a graph-based approach to compress tool definitions by tracking tool execution sequences, arguing that embeddings are the wrong tool for this problem.", "body_md": "I wired three MCP servers into my agent, then did something I should have done first: I counted what they cost.\n\nOne filesystem server, 28 tools: **about 7,000 tokens**. Just the tool definitions. Before a system prompt, before any conversation, before the user has typed anything.\n\nThen I noticed the part that actually matters.\n\nAn LLM API call is stateless. There is no server-side memory of a conversation — you re-send everything, every time. That includes the tool definitions.\n\nSo the 7,000 tokens are not paid once when the agent connects. They are paid on turn one, and turn two, and turn twenty. A 40-turn session with three MCP servers connected pays for tool schemas *forty times*.\n\nMeanwhile, on any given turn, the agent calls one tool. Maybe two.\n\nYou are paying full price for 26 tools that were never in play.\n\nDo not take my number. Capture your own `tools/list`\n\nresponse and measure it:\n\n``` python\nimport json, tiktoken\n\nenc = tiktoken.get_encoding(\"cl100k_base\")\ntools = json.load(open(\"tools.json\"))[\"result\"][\"tools\"]\n\npayload = json.dumps(tools, separators=(\",\", \":\"))\nprint(f\"{len(tools)} tools\")\nprint(f\"{len(payload):,} bytes\")\nprint(f\"{len(enc.encode(payload)):,} tokens per turn\")\n```\n\nMost people I have shown this to guess low by a factor of three. The schemas are bigger than they feel, because nobody reads them — the client fetches them and hands them straight to the model.\n\nHere is one tool from my fixture, verbatim, as a real MCP server would send it:\n\n```\n{\n  \"name\": \"fs_write_file\",\n  \"description\": \"Create a new file or completely overwrite an existing file with new content. Use with caution as it will overwrite existing files without warning. Handles text content with proper encoding.\",\n  \"inputSchema\": {\n    \"type\": \"object\",\n    \"properties\": {\n      \"path\":    { \"type\": \"string\",  \"description\": \"Destination path of the file to write.\" },\n      \"content\": { \"type\": \"string\",  \"description\": \"Full textual content to write into the file.\" },\n      \"encoding\":{ \"type\": \"string\",  \"enum\": [\"utf-8\",\"utf-16\",\"latin-1\"], \"default\": \"utf-8\" },\n      \"createDirectories\": { \"type\": \"boolean\", \"default\": false },\n      \"mode\":    { \"type\": \"string\",  \"description\": \"POSIX file mode in octal.\" }\n    },\n    \"required\": [\"path\",\"content\"],\n    \"additionalProperties\": false\n  }\n}\n```\n\n605 bytes. And that is a *small* one — the GitHub and Slack tools in my set run past 900.\n\nNow split that into two piles.\n\nThe model needs the **name** and a rough sense of what the tool does in order to *decide* to call it. It needs the **full parameter schema** only once it has already decided — to actually format the call.\n\nAlmost every tool in your list is in the first situation on any given turn. You are shipping pile two for all of them anyway.\n\nThat is the whole thing. Keep the tools the agent is plausibly about to call at full fidelity. Compress the rest down to what is needed to *name* them.\n\nThe hard part is the word \"plausibly\".\n\nThe obvious reach is embeddings: index the tool descriptions, embed the conversation, retrieve top-K. I think that is the wrong instrument here, for two reasons.\n\nFirst, tool names are not prose. They are `fs_read_file`\n\n, `git_commit`\n\n, `github_create_pull_request`\n\n— dense identifiers with maybe a sentence of docs. Embeddings smooth exactly the signal you want sharp.\n\nSecond, you are on the hot path of `tools/list`\n\n. Every millisecond you add is a millisecond in front of the user, on every turn.\n\nThere is a much cheaper signal sitting in plain sight: **agent workflows are extremely repetitive**.\n\n`git_status`\n\n→ `git_diff`\n\n→ `git_commit`\n\n. `fs_read_file`\n\n→ `fs_edit_file`\n\n→ `fs_read_file`\n\n. Over and over.\n\nSo build a graph. Every time a tool executes, draw an edge from the previously executed tool to this one and increment its weight. After a handful of turns you can read off a probability directly:\n\n```\nP(next = B | last = A) = weight(A → B) / total_out_weight(A)\n```\n\nThat is a first-order Markov chain over tool calls, and it is embarrassingly cheap — a map lookup and a division. In my implementation it accounts for the largest single weight in the ranking.\n\nNotice the graph must **not** be acyclic. Real workflows loop; `read → edit → read`\n\nis signal, not noise. Every \"DAG\" framing of this problem I have seen throws away information.\n\nFour more signals fill the gaps, each normalised to `[0,1]`\n\nand summed with a weight:\n\n`A → B → A`\n\nalternation.`2^(-steps_since_last_use / halflife)`\n\n.That last one needs one detail to work at all. Tool names are `snake_case`\n\n, so you have to split on `_`\n\n**and** on camelCase boundaries before matching. `fs_read_file`\n\nbecomes `read`\n\n+ `file`\n\n, which then matches an agent that has been talking about reading a file. Skip that split and the lexical signal is dead on arrival — I know because mine was, for an afternoon.\n\nTotal cost of ranking 28 tools: **76 microseconds**. For 500 tools, 1.3 ms.\n\nFor the tools that lose, the compressed form is:\n\n```\n{\n  \"description\": \"Create a new file or completely overwrite an existing file with new content. Use with…\",\n  \"inputSchema\": { \"type\": \"object\" },\n  \"name\": \"fs_write_file\"\n}\n```\n\n605 bytes → 152.\n\nThe one decision worth explaining is `inputSchema`\n\n. The tempting move is to delete it. Don't — an absent `inputSchema`\n\nis invalid per the MCP schema, and strict clients will reject the tool outright. `{\"type\":\"object\"}`\n\nis a legal, maximally permissive object schema that satisfies every validator for 19 bytes.\n\nThis is the part that decides whether the whole idea is usable or a footgun.\n\nA compressed tool **keeps its name**. The model can still see that it exists and can still decide to call it — it just does not have the parameter schema in front of it. So when the prediction misses, the model reaches for the tool anyway.\n\nAt that moment the proxy:\n\n`notifications/tools/list_changed`\n\nto the client;`tools/list`\n\n, and now that tool comes back at full fidelity.Worst case is one extra round-trip. Nothing is ever unrecoverable.\n\nOne protocol detail: that notification is only sent if the upstream server declared `capabilities.tools.listChanged`\n\nduring `initialize`\n\n. Inventing a capability the server never advertised is how you get a client that ignores you, or worse, errors. If the capability is missing, compression still applies and the reveal lands on the client's next natural `tools/list`\n\n.\n\nActually removing tools from the list is possible in my implementation, but it is **off by default**. A compressed tool is recoverable; a deleted one is not. That asymmetry should decide the default, and in most tools of this kind it doesn't.\n\nHere is the part I did not anticipate, and the reason I think most naive implementations of this idea *lose* money.\n\nBoth Anthropic and OpenAI cache prompt prefixes. The cache key is an **exact byte prefix**. Tool definitions sit near the front of the prompt, which makes them prime cached content.\n\nNow imagine a pruner that re-ranks and re-orders the tool list on every turn, putting the most relevant tool first. It feels right. It is a disaster: every turn produces a different prefix, so every turn is a cache miss. You save 50% of the tool tokens and lose the discount on *everything* that follows them.\n\nSo the output has to be deterministic, and it has to be positionally stable:\n\nI have a test that replays the same event sequence 50 times from a cold state and asserts byte equality of the output. It is the single most valuable test in the repo.\n\nWhile verifying that pass-through was truly byte-identical, a test failed on a diff I could not explain at first:\n\n```\n got <email>.\"}},\"required\":[\"repo\",\"message\"]\nwant <email>.\"}},\"required\":[\"repo\",\"message\"]\n```\n\nGo's `encoding/json`\n\n**HTML-escapes by default**. `<`\n\n, `>`\n\nand `&`\n\nbecome six-byte `\\uXXXX`\n\nsequences. Every schema containing something like `Name <email>`\n\nwas being silently inflated — by the tool whose entire job is to shrink schemas.\n\nThe fix is one line:\n\n```\nenc := json.NewEncoder(&buf)\nenc.SetEscapeHTML(false)\n```\n\nWorth checking in any Go proxy that re-encodes JSON it did not author. You are not just wasting bytes, you are modifying a payload you promised to forward transparently.\n\nApple M5, Go 1.26, 28-tool fixture of realistic MCP schemas:\n\n| Metric | Value |\n|---|---|\nTokens per `tools/list`\n|\n7,079 → 3,074 (-56.6%) |\nBytes per `tools/list`\n|\n26,283 → 10,943 (-58.4%) |\n| 500-tool list | -78% |\n| Ranking latency, 28 tools | 76 µs mean, 0.50 ms worst |\n| Full frame rewrite (decode + prune + re-encode) | 0.44 ms |\n| Retained state after 20k tool calls | 2.1 MiB |\n\nTwo honesty notes, because both invite a fair challenge:\n\n**Token counts are estimates.** The token figures come from a built-in heuristic — word runs at roughly 4 chars per token, punctuation runs at roughly 2 — not from a real BPE tokenizer, because I did not want to ship a vocabulary file. It is checked against `cl100k_base`\n\nreference counts and stays within a 0.6×–1.8× band. Byte counts are exact. Both sides of the before/after ratio carry the same bias, so the *reduction* holds even where the absolute number drifts.\n\n**Cold start is weaker.** The first `tools/list`\n\nof a session has no execution history, so it ranks on lexical signal alone and falls back to upstream order. It still roughly halves the payload — the compression does that on its own — but the *selection* gets meaningfully better after a few tool calls.\n\nI packaged this as **mcp-diet** — a transparent stdio proxy, Go, no dependencies outside the standard library, MIT.\n\nYou can measure your own setup without installing anything into your agent:\n\n```\ngo install github.com/albererinofigo-droid/mcp-diet/cmd/mcp-diet@latest\nmcp-diet analyze your-tools-list.json\nphp\ntools       28 (full 8, compressed 20, dropped 0)\nbytes       26283 -> 10943  (-58.4%)\nest tokens  7079 -> 3074  (-56.6%)\nprune time  0.455 ms\n```\n\nTo actually use it, wrap the server command you already run:\n\n```\n{\n  \"mcpServers\": {\n    \"filesystem\": {\n      \"command\": \"mcp-diet\",\n      \"args\": [\"--server\", \"npx -y @modelcontextprotocol/server-filesystem /srv\"]\n    }\n  }\n}\n```\n\nNothing else changes. Same protocol on both sides, and everything that is not a `tools/list`\n\nresponse is forwarded byte-for-byte.\n\nCurrent limitation worth stating plainly: **stdio only**. SSE and streamable-HTTP transports are not implemented yet, though the pruning core is transport-agnostic and usable as a library.\n\nRepo: [https://github.com/albererinofigo-droid/mcp-diet](https://github.com/albererinofigo-droid/mcp-diet)\n\nIf you run the `analyze`\n\ncommand on your own setup, I would genuinely like to see the number. My guess is that most people are paying more than they think, on every single turn.", "url": "https://wpnews.pro/news/your-mcp-agent-re-sends-7000-tokens-of-tool-schemas-on-every-turn", "canonical_source": "https://dev.to/szabo_75/your-mcp-agent-re-sends-7000-tokens-of-tool-schemas-on-every-turn-2ep2", "published_at": "2026-08-16 21:58:01+00:00", "updated_at": "2026-08-16 22:42:17.556029+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "developer-tools"], "entities": ["MCP", "GitHub", "Slack"], "alternates": {"html": "https://wpnews.pro/news/your-mcp-agent-re-sends-7000-tokens-of-tool-schemas-on-every-turn", "markdown": "https://wpnews.pro/news/your-mcp-agent-re-sends-7000-tokens-of-tool-schemas-on-every-turn.md", "text": "https://wpnews.pro/news/your-mcp-agent-re-sends-7000-tokens-of-tool-schemas-on-every-turn.txt", "jsonld": "https://wpnews.pro/news/your-mcp-agent-re-sends-7000-tokens-of-tool-schemas-on-every-turn.jsonld"}}