Two days ago I measured that 4.4% of MCP servers changed their tool contract in 36 hours, and refused to annualise it on the grounds that changes probably cluster.
I now have a third snapshot, and the caution was warranted more strongly than I expected.
Same 474 servers, three snapshots: baseline, +36h, +72h.
| changed since baseline | |
|---|---|
| +36 hours | 21 (4.4%) |
| +72 hours | 24 (5.1%) |
Twenty-one servers moved in the first 36 hours. In the next 36 hours, three more did. A constant independent rate would have predicted 42 by day three. The real number was 24 — 57% of the linear projection, and the gap widens the further you extrapolate.
Two other things fell out of the comparison:
This is not "MCP servers change at ~3% a day." It is:
A small set of actively-developed servers that change constantly, and a large majority that are effectively frozen.
The first snapshot caught almost the entire volatile subset in one pass. Everything after that is scraping a much thinner seam — a few genuinely new movers, plus repeat churn from the same handful.
If you annualised my original number you'd conclude that most of the registry rewrites itself within a month. That's wrong, and it's wrong in the direction that makes you build the wrong thing: continuous revalidation of everything, when what you actually need is to identify the ~5% that moves and watch those.
I hashed inputSchema
. On the last post anp2network pointed out that tools/list
also carries outputSchema
when a server declares structured output, and that it binds any caller parsing results just as hard.
That's correct and I was blind to it. So I measured the declared surface:
| surface | coverage |
|---|---|
tools with outputSchema |
|
| 1,553 / 8,629 (18.0%) | |
tools with annotations |
|
| 6,251 / 8,629 (72.4%) | |
servers declaring any outputSchema |
|
155 / 476 (32.6%) | Only 18% of tools declare an output contract at all. Which cuts both ways: output drift is a real hazard for the 18%, and for the other 82% there is simply no declared contract to break — you are parsing whatever comes back and hoping.
I'd argue the 82% is the bigger problem, and it doesn't show up in any drift measurement because there's nothing to diff.
zira125 suggested hashing inputSchema
, outputSchema
, description
and annotations
separately and classifying changes rather than treating every hash mismatch as equally bad — additive optional fields warn, required-field additions and enum narrowing fail. That's obviously right, and v2 of my census now captures all four separately.
komo framed the deployment shape: snapshot contracts as build artifacts and fail fast when the hash moves. And Mads Hansen pointed out something I'd waved through — "tool added" is not automatically safe, because a new overlapping tool changes selection and can silently redirect calls that used to go somewhere else, even though every old invocation still validates.
Between them that's a better spec than I had when I started. The useful version is not a monitor that re-checks everything on a timer. It's:
474 servers comparable across all three snapshots, drawn as a seeded random sample (random.seed(20260730)
) from the 5,346 registry endpoints that complete an anonymous handshake, so every re-run hits identical servers. initialize
→ notifications/initialized
→ tools/list
, handling SSE frames and threading Mcp-Session-Id
. SHA-256 per surface, sorted keys.
Three snapshots is enough to see that a straight line is the wrong model. It is not enough to say what the right one is. I'll keep taking them.