I wired three MCP servers into my agent, then did something I should have done first: I counted what they cost.
One filesystem server, 28 tools: about 7,000 tokens. Just the tool definitions. Before a system prompt, before any conversation, before the user has typed anything.
Then I noticed the part that actually matters.
An LLM API call is stateless. There is no server-side memory of a conversation — you re-send everything, every time. That includes the tool definitions.
So the 7,000 tokens are not paid once when the agent connects. They are paid on turn one, and turn two, and turn twenty. A 40-turn session with three MCP servers connected pays for tool schemas forty times.
Meanwhile, on any given turn, the agent calls one tool. Maybe two.
You are paying full price for 26 tools that were never in play.
Do not take my number. Capture your own tools/list
response and measure it:
import json, tiktoken
enc = tiktoken.get_encoding("cl100k_base")
tools = json.load(open("tools.json"))["result"]["tools"]
payload = json.dumps(tools, separators=(",", ":"))
print(f"{len(tools)} tools")
print(f"{len(payload):,} bytes")
print(f"{len(enc.encode(payload)):,} tokens per turn")
Most people I have shown this to guess low by a factor of three. The schemas are bigger than they feel, because nobody reads them — the client fetches them and hands them straight to the model.
Here is one tool from my fixture, verbatim, as a real MCP server would send it:
{
"name": "fs_write_file",
"description": "Create a new file or completely overwrite an existing file with new content. Use with caution as it will overwrite existing files without warning. Handles text content with proper encoding.",
"inputSchema": {
"type": "object",
"properties": {
"path": { "type": "string", "description": "Destination path of the file to write." },
"content": { "type": "string", "description": "Full textual content to write into the file." },
"encoding":{ "type": "string", "enum": ["utf-8","utf-16","latin-1"], "default": "utf-8" },
"createDirectories": { "type": "boolean", "default": false },
"mode": { "type": "string", "description": "POSIX file mode in octal." }
},
"required": ["path","content"],
"additionalProperties": false
}
}
605 bytes. And that is a small one — the GitHub and Slack tools in my set run past 900.
Now split that into two piles.
The model needs the name and a rough sense of what the tool does in order to decide to call it. It needs the full parameter schema only once it has already decided — to actually format the call.
Almost every tool in your list is in the first situation on any given turn. You are shipping pile two for all of them anyway.
That is the whole thing. Keep the tools the agent is plausibly about to call at full fidelity. Compress the rest down to what is needed to name them.
The hard part is the word "plausibly".
The obvious reach is embeddings: index the tool descriptions, embed the conversation, retrieve top-K. I think that is the wrong instrument here, for two reasons.
First, tool names are not prose. They are fs_read_file
, git_commit
, github_create_pull_request
— dense identifiers with maybe a sentence of docs. Embeddings smooth exactly the signal you want sharp.
Second, you are on the hot path of tools/list
. Every millisecond you add is a millisecond in front of the user, on every turn.
There is a much cheaper signal sitting in plain sight: agent workflows are extremely repetitive.
git_status
→ git_diff
→ git_commit
. fs_read_file
→ fs_edit_file
→ fs_read_file
. Over and over.
So build a graph. Every time a tool executes, draw an edge from the previously executed tool to this one and increment its weight. After a handful of turns you can read off a probability directly:
P(next = B | last = A) = weight(A → B) / total_out_weight(A)
That is a first-order Markov chain over tool calls, and it is embarrassingly cheap — a map lookup and a division. In my implementation it accounts for the largest single weight in the ranking.
Notice the graph must not be acyclic. Real workflows loop; read → edit → read
is signal, not noise. Every "DAG" framing of this problem I have seen throws away information.
Four more signals fill the gaps, each normalised to [0,1]
and summed with a weight:
A → B → A
alternation.2^(-steps_since_last_use / halflife)
.That last one needs one detail to work at all. Tool names are snake_case
, so you have to split on _
and on camelCase boundaries before matching. fs_read_file
becomes read
file
, which then matches an agent that has been talking about reading a file. Skip that split and the lexical signal is dead on arrival — I know because mine was, for an afternoon.
Total cost of ranking 28 tools: 76 microseconds. For 500 tools, 1.3 ms.
For the tools that lose, the compressed form is:
{
"description": "Create a new file or completely overwrite an existing file with new content. Use with…",
"inputSchema": { "type": "object" },
"name": "fs_write_file"
}
605 bytes → 152.
The one decision worth explaining is inputSchema
. The tempting move is to delete it. Don't — an absent inputSchema
is invalid per the MCP schema, and strict clients will reject the tool outright. {"type":"object"}
is a legal, maximally permissive object schema that satisfies every validator for 19 bytes.
This is the part that decides whether the whole idea is usable or a footgun.
A compressed tool keeps its name. The model can still see that it exists and can still decide to call it — it just does not have the parameter schema in front of it. So when the prediction misses, the model reaches for the tool anyway.
At that moment the proxy:
notifications/tools/list_changed
to the client;tools/list
, and now that tool comes back at full fidelity.Worst case is one extra round-trip. Nothing is ever unrecoverable.
One protocol detail: that notification is only sent if the upstream server declared capabilities.tools.listChanged
during initialize
. Inventing a capability the server never advertised is how you get a client that ignores you, or worse, errors. If the capability is missing, compression still applies and the reveal lands on the client's next natural tools/list
.
Actually removing tools from the list is possible in my implementation, but it is off by default. A compressed tool is recoverable; a deleted one is not. That asymmetry should decide the default, and in most tools of this kind it doesn't.
Here is the part I did not anticipate, and the reason I think most naive implementations of this idea lose money.
Both Anthropic and OpenAI cache prompt prefixes. The cache key is an exact byte prefix. Tool definitions sit near the front of the prompt, which makes them prime cached content.
Now imagine a pruner that re-ranks and re-orders the tool list on every turn, putting the most relevant tool first. It feels right. It is a disaster: every turn produces a different prefix, so every turn is a cache miss. You save 50% of the tool tokens and lose the discount on everything that follows them.
So the output has to be deterministic, and it has to be positionally stable:
I have a test that replays the same event sequence 50 times from a cold state and asserts byte equality of the output. It is the single most valuable test in the repo.
While verifying that pass-through was truly byte-identical, a test failed on a diff I could not explain at first:
got <email>."}},"required":["repo","message"]
want <email>."}},"required":["repo","message"]
Go's encoding/json
HTML-escapes by default. <
, >
and &
become six-byte \uXXXX
sequences. Every schema containing something like Name <email>
was being silently inflated — by the tool whose entire job is to shrink schemas.
The fix is one line:
enc := json.NewEncoder(&buf)
enc.SetEscapeHTML(false)
Worth checking in any Go proxy that re-encodes JSON it did not author. You are not just wasting bytes, you are modifying a payload you promised to forward transparently.
Apple M5, Go 1.26, 28-tool fixture of realistic MCP schemas:
| Metric | Value |
|---|---|
Tokens per tools/list |
|
| 7,079 → 3,074 (-56.6%) | |
Bytes per tools/list |
|
| 26,283 → 10,943 (-58.4%) | |
| 500-tool list | -78% |
| Ranking latency, 28 tools | 76 µs mean, 0.50 ms worst |
| Full frame rewrite (decode + prune + re-encode) | 0.44 ms |
| Retained state after 20k tool calls | 2.1 MiB |
Two honesty notes, because both invite a fair challenge:
Token counts are estimates. The token figures come from a built-in heuristic — word runs at roughly 4 chars per token, punctuation runs at roughly 2 — not from a real BPE tokenizer, because I did not want to ship a vocabulary file. It is checked against cl100k_base
reference counts and stays within a 0.6×–1.8× band. Byte counts are exact. Both sides of the before/after ratio carry the same bias, so the reduction holds even where the absolute number drifts.
Cold start is weaker. The first tools/list
of a session has no execution history, so it ranks on lexical signal alone and falls back to upstream order. It still roughly halves the payload — the compression does that on its own — but the selection gets meaningfully better after a few tool calls.
I packaged this as mcp-diet — a transparent stdio proxy, Go, no dependencies outside the standard library, MIT.
You can measure your own setup without installing anything into your agent:
go install github.com/albererinofigo-droid/mcp-diet/cmd/mcp-diet@latest
mcp-diet analyze your-tools-list.json
php
tools 28 (full 8, compressed 20, dropped 0)
bytes 26283 -> 10943 (-58.4%)
est tokens 7079 -> 3074 (-56.6%)
prune time 0.455 ms
To actually use it, wrap the server command you already run:
{
"mcpServers": {
"filesystem": {
"command": "mcp-diet",
"args": ["--server", "npx -y @modelcontextprotocol/server-filesystem /srv"]
}
}
}
Nothing else changes. Same protocol on both sides, and everything that is not a tools/list
response is forwarded byte-for-byte.
Current limitation worth stating plainly: stdio only. SSE and streamable-HTTP transports are not implemented yet, though the pruning core is transport-agnostic and usable as a library.
Repo: https://github.com/albererinofigo-droid/mcp-diet
If you run the analyze
command on your own setup, I would genuinely like to see the number. My guess is that most people are paying more than they think, on every single turn.