Your MCP agent re-sends 7,000 tokens of tool schemas on every turn A developer wired three MCP servers into an agent and found that tool schemas consume about 7,000 tokens per turn, and because LLM API calls are stateless, those tokens are re-sent on every turn, leading to a 40-turn session paying for schemas forty times. The developer proposes a graph-based approach to compress tool definitions by tracking tool execution sequences, arguing that embeddings are the wrong tool for this problem. I wired three MCP servers into my agent, then did something I should have done first: I counted what they cost. One filesystem server, 28 tools: about 7,000 tokens . Just the tool definitions. Before a system prompt, before any conversation, before the user has typed anything. Then I noticed the part that actually matters. An LLM API call is stateless. There is no server-side memory of a conversation — you re-send everything, every time. That includes the tool definitions. So the 7,000 tokens are not paid once when the agent connects. They are paid on turn one, and turn two, and turn twenty. A 40-turn session with three MCP servers connected pays for tool schemas forty times . Meanwhile, on any given turn, the agent calls one tool. Maybe two. You are paying full price for 26 tools that were never in play. Do not take my number. Capture your own tools/list response and measure it: python import json, tiktoken enc = tiktoken.get encoding "cl100k base" tools = json.load open "tools.json" "result" "tools" payload = json.dumps tools, separators= ",", ":" print f"{len tools } tools" print f"{len payload :,} bytes" print f"{len enc.encode payload :,} tokens per turn" Most people I have shown this to guess low by a factor of three. The schemas are bigger than they feel, because nobody reads them — the client fetches them and hands them straight to the model. Here is one tool from my fixture, verbatim, as a real MCP server would send it: { "name": "fs write file", "description": "Create a new file or completely overwrite an existing file with new content. Use with caution as it will overwrite existing files without warning. Handles text content with proper encoding.", "inputSchema": { "type": "object", "properties": { "path": { "type": "string", "description": "Destination path of the file to write." }, "content": { "type": "string", "description": "Full textual content to write into the file." }, "encoding":{ "type": "string", "enum": "utf-8","utf-16","latin-1" , "default": "utf-8" }, "createDirectories": { "type": "boolean", "default": false }, "mode": { "type": "string", "description": "POSIX file mode in octal." } }, "required": "path","content" , "additionalProperties": false } } 605 bytes. And that is a small one — the GitHub and Slack tools in my set run past 900. Now split that into two piles. The model needs the name and a rough sense of what the tool does in order to decide to call it. It needs the full parameter schema only once it has already decided — to actually format the call. Almost every tool in your list is in the first situation on any given turn. You are shipping pile two for all of them anyway. That is the whole thing. Keep the tools the agent is plausibly about to call at full fidelity. Compress the rest down to what is needed to name them. The hard part is the word "plausibly". The obvious reach is embeddings: index the tool descriptions, embed the conversation, retrieve top-K. I think that is the wrong instrument here, for two reasons. First, tool names are not prose. They are fs read file , git commit , github create pull request — dense identifiers with maybe a sentence of docs. Embeddings smooth exactly the signal you want sharp. Second, you are on the hot path of tools/list . Every millisecond you add is a millisecond in front of the user, on every turn. There is a much cheaper signal sitting in plain sight: agent workflows are extremely repetitive . git status → git diff → git commit . fs read file → fs edit file → fs read file . Over and over. So build a graph. Every time a tool executes, draw an edge from the previously executed tool to this one and increment its weight. After a handful of turns you can read off a probability directly: P next = B | last = A = weight A → B / total out weight A That is a first-order Markov chain over tool calls, and it is embarrassingly cheap — a map lookup and a division. In my implementation it accounts for the largest single weight in the ranking. Notice the graph must not be acyclic. Real workflows loop; read → edit → read is signal, not noise. Every "DAG" framing of this problem I have seen throws away information. Four more signals fill the gaps, each normalised to 0,1 and summed with a weight: A → B → A alternation. 2^ -steps since last use / halflife .That last one needs one detail to work at all. Tool names are snake case , so you have to split on and on camelCase boundaries before matching. fs read file becomes read + file , which then matches an agent that has been talking about reading a file. Skip that split and the lexical signal is dead on arrival — I know because mine was, for an afternoon. Total cost of ranking 28 tools: 76 microseconds . For 500 tools, 1.3 ms. For the tools that lose, the compressed form is: { "description": "Create a new file or completely overwrite an existing file with new content. Use with…", "inputSchema": { "type": "object" }, "name": "fs write file" } 605 bytes → 152. The one decision worth explaining is inputSchema . The tempting move is to delete it. Don't — an absent inputSchema is invalid per the MCP schema, and strict clients will reject the tool outright. {"type":"object"} is a legal, maximally permissive object schema that satisfies every validator for 19 bytes. This is the part that decides whether the whole idea is usable or a footgun. A compressed tool keeps its name . The model can still see that it exists and can still decide to call it — it just does not have the parameter schema in front of it. So when the prediction misses, the model reaches for the tool anyway. At that moment the proxy: notifications/tools/list changed to the client; tools/list , and now that tool comes back at full fidelity.Worst case is one extra round-trip. Nothing is ever unrecoverable. One protocol detail: that notification is only sent if the upstream server declared capabilities.tools.listChanged during initialize . Inventing a capability the server never advertised is how you get a client that ignores you, or worse, errors. If the capability is missing, compression still applies and the reveal lands on the client's next natural tools/list . Actually removing tools from the list is possible in my implementation, but it is off by default . A compressed tool is recoverable; a deleted one is not. That asymmetry should decide the default, and in most tools of this kind it doesn't. Here is the part I did not anticipate, and the reason I think most naive implementations of this idea lose money. Both Anthropic and OpenAI cache prompt prefixes. The cache key is an exact byte prefix . Tool definitions sit near the front of the prompt, which makes them prime cached content. Now imagine a pruner that re-ranks and re-orders the tool list on every turn, putting the most relevant tool first. It feels right. It is a disaster: every turn produces a different prefix, so every turn is a cache miss. You save 50% of the tool tokens and lose the discount on everything that follows them. So the output has to be deterministic, and it has to be positionally stable: I have a test that replays the same event sequence 50 times from a cold state and asserts byte equality of the output. It is the single most valuable test in the repo. While verifying that pass-through was truly byte-identical, a test failed on a diff I could not explain at first: got