{"slug": "i-put-a-proxy-on-the-mcp-pipe-for-90-trials-most-of-one-client-s-calls-never-the", "title": "I put a proxy on the MCP pipe for 90 trials. Most of one client's calls never reached the server", "summary": "A developer's proxy-based study of MCP server calls across 90 trials found that one client's calls failed internally before reaching the server, mimicking a low-effort model. The run, covering three servers and two clients, logged 87 successes and revealed per-call token costs vary by client, not just server.", "body_md": "In [the last post](https://dev.to/lopster568/i-measured-what-14-mcp-servers-cost-a-context-window-claude-counts-them-64-higher-than-tiktoken-10pj), on 2026-08-18, I published what 14 MCP servers cost a context window before an agent does any work, and said the next thing was Tier 2: real clients, real tasks, every frame logged. That has now run. Ninety trials, three servers, two clients, fifteen scripted tasks, three trials each, through a proxy on the MCP stdio pipe.\n\nThe finding worth the post is not in the token table. It came out of the shakedown hours earlier that same day, on the client version the run then replaced: one of the clients was failing calls inside itself, before a byte reached the server, and on the wire that looked exactly like a model that tried very little and answered wrong.\n\nThe matrix is `filesystem`\n\n, `playwright`\n\nand `github`\n\n, five scripted tasks each, three trials per task per client, suite version 1.0.1. Ninety trials, 87 successes. Servers pinned at `@modelcontextprotocol/server-filesystem@2026.7.10`\n\n, `@playwright/mcp@0.0.79`\n\n, and the `ghcr.io/github/github-mcp-server`\n\ncontainer, launched untagged, which reported itself as v1.9.0. Clients: Claude Code 2.1.235 on `claude-sonnet-5`\n\n, Gemini CLI 0.55.1 on `gemini-2.5-flash`\n\n, both model ids read back out of each trial's own client JSON rather than assumed from the flag. Claude Code also invokes `claude-haiku-4-5`\n\non every trial for its own bookkeeping, under a thousand input tokens a time. That never touches the MCP pipe and is in none of the figures below, but it is in the manifest, so it is worth knowing it is there.\n\nThe three Gemini failures are one per server and they are three different things: an off-by-one line count on FS-04, a response its own tool layer rejected on GH-05 after six calls reached the server, and a PW-01 final message that declared the task done without restating the price the check looks for.\n\nThree trials per cell buys per-cell counts and nothing statistical, so these stay counts and never rates, per section 3.3 of the spec: Claude Code finished 45 of 45 trials, Gemini CLI 42 of 45. 87 of 90 state checks passed, and 87 of 90 trials also classify as `tool_use_success`\n\n; the two are different fields that happen to agree this run. Three failures do not tell you one client is more reliable than the other.\n\nMedian call tokens per trial, meaning the `tools/call`\n\narguments plus the results the server sent back, counted with `o200k_base`\n\nso the figure shares Tier 1's token basis. Each column is a median over 15 trials, five scripted tasks by three trials, so a different task mix moves these figures. They are not a per-server price:\n\n| server | Claude Code | Gemini CLI |\n|---|---|---|\n| filesystem | 525 | 170 |\n| playwright | 629 | 522 |\n| github | 1,698 | 223 |\n\nBoth clients median one tool call on github. On GH-01 both called `get_file_contents`\n\nwith identical arguments, and the answer measured 1,561 result tokens for Claude Code against 161 for Gemini CLI.\n\nEvery response in the Claude Code session carried a `_meta`\n\nblock, `io.modelcontextprotocol/serverInfo`\n\n, holding the server's name, version and two PNG icons inlined as base64: 2,215 characters wrapped around a 472-character answer. Across the 15 github trials on Claude Code, all 81 responses carried it. Across the 15 on Gemini CLI, none of the 93 did. The two sessions negotiated different protocol revisions, `2026-07-28`\n\nagainst `2025-06-18`\n\n. One server version against two clients does not prove the server keys on the revision. It does establish that a per-call cost figure is not a property of the server alone, which is the reason no row here publishes one number.\n\nNone of this is in the 90. The shakedown that preceded the run, one trial per task, ran on Gemini CLI 0.18.4, and its rows survive as superseded records in the same day's manifest. All five playwright tasks failed there, each trial with exactly one call on the wire. Read from the frames alone, the classifier bucketed it as a capability failure, once as a decline because the model apologised.\n\nThat reading is false. 0.18.4 validated each call's arguments against the schema the server advertised, and its bundled validator had no JSON Schema draft 2020-12 meta-schema registered. `@playwright/mcp@0.0.79`\n\ndeclares 2020-12 on all 24 of its tools, so most of its calls died inside the client with `no schema with key or ref \"https://json-schema.org/draft/2020-12/schema\"`\n\nand never reached the pipe: each trial got exactly one onto the wire and lost three or four more inside. The same client ran filesystem and github clean the same day, because `@modelcontextprotocol/server-filesystem`\n\ndeclares draft-07 and `github-mcp-server`\n\ndeclares no `$schema`\n\nat all. The dialect decided it, not the server.\n\nWhat caught it was the tool-call gap field, which exists for a different purpose (spec 4.2). The tool-call gap takes every tool name the server advertised, subtracts the wire frames the proxy logged from the calls the client's own usage output attributes to that tool, and sums the positive differences. Zero is the normal state. Those five trials read 3, 4, 4, 4 and 3, meaning the client formed calls to tools the server offered and those calls never left it. No classification could have said that, because a classification only sees what reached the wire.\n\nUpstream had already fixed the validator in 0.28.0, before this shakedown ran (gemini-cli issue #14970, PR #15060): it dispatches a dedicated 2020-12 instance on the schema's own `$schema`\n\nand falls back to skip-and-warn for unknown dialects. The stale client was mine. Upgrading to 0.55.1 and re-running the five tasks gave five passes, a zero gap on every trial, and no schema error in stderr.\n\nThe upgrade nearly broke the detector: the log keys the server's bare tool name, but the client's usage output moved to `mcp_<server>_<tool>`\n\nbetween versions. Matched on the bare name alone, every difference came out negative, and the gap would have read a clean zero, a zero from a working detector and a zero from a broken one are the same character in the output. The runner now sums both spellings.\n\nFS-04 asks the client to find the longest file in a fixture tree and write its path and line count to a file. The answer is `logs/access.log`\n\nand 137. Gemini CLI passed two of three FS-04 trials. The third wrote `logs/access.log`\n\nand 138, and reported the task complete.\n\nThe client made three calls, `directory_tree`\n\n, `read_multiple_files`\n\n, `write_file`\n\n, all on the wire with a gap of zero, and the `read_multiple_files`\n\nresponse carried the complete log: 137 lines, `item-0001`\n\nthrough `item-0137`\n\n, no truncation and no elision marker. That server concatenates files into one text block separated by a blank line and a marker line, so the last log line is followed by an empty line before the separator. The log cannot show which of those the model counted. It does show that nothing upstream lost or added a line.\n\nEach of these produced a number that would have read as a capability result.\n\nGH-03 came back from Claude Code with zero tool calls and this: *\"I have a standing instruction (from your global CLAUDE.md) that I never send outbound communications, including creating issues, myself.\"* The trial measured my own memory file, and the classifier called it a hallucination. The runner now passes `--setting-sources \"\"`\n\nand stamps the value in the run header.\n\nFS-01 came back correct with zero frames on the wire, because Claude Code answered it with its own built-in `Read`\n\ntool and the server never saw the task. Right answer, no measurement. That has a bucket of its own, `answered_without_tools`\n\n, counted as a failure for every tool-use metric and never folded into a decline, and the runner now denies the built-in surface.\n\nGemini CLI answered a GitHub read task by posting the answer as a comment on the fixture issue, then saying only that it had completed the request. The check failed it on its merits, but the answer stayed in the repo and the fixture verifier still reported baseline, because nothing compared comment counts. The next trial would have found the previous one's answer waiting in the thread. The reset script now deletes comments and reports them as drift.\n\nEleven faults are enumerated in the spec: seven fixed on 2026-08-18, four more forced by the client upgrade the next day, two of which killed a run outright.\n\nThe proxy measures wire traffic on the MCP stdio pipe: the arguments of every `tools/call`\n\nrequest plus the results the server sent back. That is all it can see.\n\nIt does not measure the tool-definition schema footprint any client loaded, because that happens inside the client and never crosses the pipe, and neither client exposes a session-start figure that isolates it: every token field they report contains the system prompt and the conversation alongside the tool definitions. The spec used to claim that metric and now records it as `NOT CAPTURED`\n\nrather than dropping the row, so the hole stays visible.\n\nSo nothing here converts a MODELED label from Tier 1. Tool search stays modeled, because its total needs that per-session figure and a `k`\n\nno client reports. Code mode stays modeled for a blunter reason: neither client has a code mode at all, so no trial was ever in one. The label rule used to make a Tier 2 run for a server the condition for relabelling it, which would have licensed exactly that mistake; it was corrected in methodology 0.3.1 before a run could trigger it.\n\nTwo more limits, both on the published file's face. Each figure is over five scripted tasks per server, so a different task mix moves the same server's call traffic. And the Tier 2 github pin is not the Tier 1 one: Tier 1 measured the remote endpoint, Tier 2 ran the `ghcr.io`\n\ncontainer untagged, so whatever `latest`\n\npointed at on 2026-08-18, with no image digest recorded; the server reported itself as `v1.9.0`\n\n, and that row is reproducible to a self-reported version and not to an image, and the two github numbers are not the same artifact.\n\nThe run recorded one Gemini GH-03 trial as a client error, because Gemini reports an API-side failure by populating an `error`\n\nobject on an otherwise well-formed document. That trial's state check had already passed: exactly one open issue with the expected title, one `tools/call`\n\nframe on the wire. The runner checked the error first and bucketed a trial that plainly succeeded as an infrastructure fault. The fix is a precedence rule: a passed success check outranks a client-reported error, and the error is kept as its own flag so neither hides the other. Every trial was then reclassified from its own stored fields rather than re-run, and exactly one moved, taking Gemini's `tool_use_success`\n\ncount from 41 of 45 to 42 of 45.\n\nPost #6 promised one thing on the back of the `fetch`\n\ncorrection: that the harness would record which dependency versions a resolve actually produced, so an `unreachable`\n\nrow could explain itself. That shipped in methodology 0.3.0. Every acquisition now records the resolved dependency set, the command that read it, and the environment it read from, plus the SDK version lifted out as its own field because that is what a reader of a broken row looks for first. The 2026-08-18 Tier 1 rows predate it; the next monthly run is the first to carry it.\n\nNothing here was metered: Claude Code rode plan quota, Gemini CLI an OAuth session on the free tier, and the servers ran local or in a container. Claude Code still reports a `cost_usd`\n\nper trial, $4.85 across the 90, and that number is sitting in the manifest in the repo. It is what this work would have cost at API rates, not what it cost here. Both runners strip the provider API key variables out of the subprocess environment, so a stray key in a parent directory cannot quietly move a trial onto metered billing.\n\nRepo: [github.com/lopster568/loadline](https://github.com/lopster568/loadline). Calculator: [loadline-dev.netlify.app](https://loadline-dev.netlify.app). The Tier 1 run is [post #6](https://dev.to/lopster568/i-measured-what-14-mcp-servers-cost-a-context-window-claude-counts-them-64-higher-than-tiktoken-10pj).\n\n`slack`\n\nis still out; its operator credential has not landed. Three questions stay open in the spec:\n\nEvery trial in this run resolved to its pin. The container-digest gap in those github rows is closed from the next run on: since 2026-08-20 the runner resolves the image digest before the first trial and re-resolves it after the last, and a digest that moved mid-run stamps the run as drifted, the same rule already applied to client versions. The 2026-08-18 rows keep the tag they were recorded with. Nothing is republished.\n\nIf a number here is wrong, the frame logs, the per-trial client JSON, the manifest and the summary are in the repo, so disagreement can point at a line. Corrections go in the log with your name on them.\n\nI do this work for hire: auditing what your agent's tool surface costs before it does any work, and cutting it down without gutting what the search can still find. Scope and pricing at [roshansingh.systems/#hire](https://roshansingh.systems/#hire), or write to [inbox@roshansingh.systems](mailto:inbox@roshansingh.systems) and tell me what your agents are loading.", "url": "https://wpnews.pro/news/i-put-a-proxy-on-the-mcp-pipe-for-90-trials-most-of-one-client-s-calls-never-the", "canonical_source": "https://dev.to/lopster568/i-put-a-proxy-on-the-mcp-pipe-for-90-trials-most-of-one-clients-calls-never-reached-the-server-20p", "published_at": "2026-08-21 07:12:53+00:00", "updated_at": "2026-08-21 07:44:09.313695+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "large-language-models"], "entities": ["Claude Code", "Gemini CLI", "filesystem", "playwright", "github", "MCP", "Claude Sonnet 5", "Gemini 2.5 Flash"], "alternates": {"html": "https://wpnews.pro/news/i-put-a-proxy-on-the-mcp-pipe-for-90-trials-most-of-one-client-s-calls-never-the", "markdown": "https://wpnews.pro/news/i-put-a-proxy-on-the-mcp-pipe-for-90-trials-most-of-one-client-s-calls-never-the.md", "text": "https://wpnews.pro/news/i-put-a-proxy-on-the-mcp-pipe-for-90-trials-most-of-one-client-s-calls-never-the.txt", "jsonld": "https://wpnews.pro/news/i-put-a-proxy-on-the-mcp-pipe-for-90-trials-most-of-one-client-s-calls-never-the.jsonld"}}