mcp-probe v1.0.0: A CI readiness gate for MCP servers

mcp-probe started as a small CLI for checking whether an MCP server starts and exposes tools.

That was useful, but after feedback from developers running real MCP servers in agent workflows, the gap became obvious:

A server can start, pass

tools/list

, and still fail every real tool call because OAuth, browser auth, or downstream permissions are broken.

So I shipped mcp-probe v1.0.0 as a CI-ready readiness gate for MCP servers.

Install #

npx @k08200/mcp-probe@latest <server>

Example:

npx @k08200/mcp-probe@latest @modelcontextprotocol/server-memory

What it checks #

• MCP protocol handshake tools/list

• optional resources and prompts discovery

• tool schema shape

• actual tool-call dry-runs

• stderr classification

• latency

• batch/fleet CI status

Tool-call dry-runs #

npx @k08200/mcp-probe@latest <server> --probe-tools

This closes the gap between “the server registered tools” and “those tools actually work in an agent loop.”

Sidecar inputs #

Auto-generated inputs are fallback only. For real CI, v1 supports sidecar files:

{
  "tools": {
    "logs_query": {
      "input": {
        "query": "service:web status:error",
        "timeframe": "1h"
      },
      "expect": {
        "not_error_code": [401, 403]
      }
    }
  }
}

Run:

npx @k08200/mcp-probe@latest datadog-mcp --probe-tools --tools-file .mcp-probe.json

This lets CI validate meaningful tool calls instead of just schema-minimum empty strings.

Batch checks #

npx @k08200/mcp-probe@latest --config mcp-probe.config.json

Useful when a team runs multiple MCP servers and wants one readiness gate.

GitHub Actions output #

npx @k08200/mcp-probe@latest --config mcp-probe.config.json --github-summary

v1 writes GitHub step summaries, emits annotations, and can generate a shields-compatible badge JSON file.

HTTP and SSE #

mcp-probe now supports stdio, Streamable HTTP, and legacy SSE:

npx @k08200/mcp-probe@latest https://example.com/mcp --header "Authorization: Bearer TOKEN"

Stderr classification #

Some servers print harmless startup warnings; others print fatal init errors. v1 adds explicit rules:

npx @k08200/mcp-probe@latest <server> \
  --stderr-allow "deprecated" \
  --stderr-fatal "missing required api key"

Recipes #

The repo includes starter recipes for Datadog, Supabase, Gmail, single-server GitHub Actions checks, fleet checks, and remote HTTP checks.

GitHub: https://github.com/k08200/mcp-probe

Release: https://github.com/k08200/mcp-probe/releases/tag/v1.0.0

npm:

npm install -g @k08200/mcp-probe