The MCP Server Pre-Publish Checklist

Incultnitollc released mcp-probe, an open-source CLI tool that scores MCP servers on a 0-100 publishability scale across five axes. The tool enforces a 10-point checklist addressing common failures such as thin tool descriptions, naming collisions, and missing install metadata. Official Anthropic reference servers score only 60/100 on description quality, while typical first-draft community servers land in the 40s.

Before you publish an MCP server, run 10 checks. Most servers fail at least three — and the failures are invisible until an agent picks the wrong tool, hallucinates an argument, or silently drops your server on connect. This is the checklist we built mcp-probe to enforce, distilled to what actually breaks in the wild. TL;DR — A publishable MCP server connects cleanly, names tools unambiguously, describes every argument, validates inputs, and ships install metadata. The single most common failure is thin tool descriptions: even the five official Anthropic reference servers cap at 60/100on description quality. MCP Inspector answers "does my server connect and list tools?" That's necessary, not sufficient. The agent doesn't experience your server the way you do in a UI — it experiences your tool descriptions and schemas as text in a context window , and it picks tools by reading them. A server can pass Inspector and still be functionally unpublishable because the model can't tell your tools apart. So the pre-publish question isn't "does it run?" It's "is it publishable?" — will a real agent, with no docs and no human in the loop, use it correctly? initialize . initialize fast. create issue exists in a dozen servers. If yours collides, the model guesses. Namespace or specify. server.json so the official MCP Registry can discover you. Devs find tools at install-time, not search-time.You can walk this list by hand, or run it: npx @incultnitollc/mcp-probe score "node ./your-server.js" mcp-probe connects to your server, runs all ten checks, and returns a 0–100 publishability score across five axes — description quality, enum/shape correctness, mutation legibility, anti-"restate the name" clauses, and distribution metadata. A passing server clears ~80. The official reference servers sit at 60 the description cap fires on every one . A typical first-draft community server lands in the 40s. Wire it into CI so it runs on every release: .github/workflows/publishability.yml - run: npx @incultnitollc/mcp-probe score "node ./dist/server.js" --fail-under 80 The exit code gates the publish. Your server can't regress below the bar you set. If you do nothing else: rewrite your tool descriptions so a model with no context could choose correctly between yours and a similarly named tool. That single fix moves more servers across the publishable line than any other on this list — and it's the one almost everyone skips. mcp-probe is an open-source CLI for testing and scoring MCP servers before you publish. npx @incultnitollc/mcp-probe · github.com/incultnitollc/mcp-probe https://github.com/incultnitollc/mcp-probe