{"slug": "mcp-ci-gates-need-receipts-tools-list-is-not-enough", "title": "MCP CI gates need receipts: tools/list is not enough", "summary": "A developer released mcp-probe@1.8.0, a CI readiness gate for MCP servers that goes beyond basic startup checks. The tool enforces that servers not only advertise tools but also pass real calls with meaningful sample inputs, catching failures from auth, permissions, and environment issues that standard `tools/list` validation misses. With `--fail-on-warn`, the tool can block CI workflows on degraded states like 401 errors rather than just hard crashes.", "body_md": "MCP servers are starting to look like normal infrastructure.\n\nThat means they need boring infrastructure checks.\n\nThe mistake I kept seeing is this:\n\n\"The server starts, and\n\n`tools/list`\n\nreturns a clean schema. Therefore it works.\"\n\nThat is not enough.\n\nAn MCP server can pass `initialize`\n\n, advertise every expected tool, and still fail every real call because auth, scopes, tenant boundaries, environment variables, downstream permissions, or read-only roles are broken.\n\nSo I pushed `mcp-probe@1.8.0`\n\nfurther toward being a real CI readiness gate for MCP servers.\n\n```\nnpx @k08200/mcp-probe@latest --config mcp-probe.config.json --github-summary --fail-on-warn\n```\n\nBy default, warnings still exit `0`\n\n. That keeps existing users from getting surprise CI failures.\n\nBut production gates often need stricter behavior:\n\n```\nmcp-probe --config mcp-probe.config.json --fail-on-warn\n```\n\nWith `--fail-on-warn`\n\n, auth handoff issues, permission warnings, or incomplete readiness receipts can block the workflow.\n\nThat matters because many MCP failures are not hard crashes. They are degraded states:\n\n`401`\n\n`mcp-probe doctor`\n\nalready checked whether a GitHub Actions workflow existed.\n\nBut that is not enough either.\n\nThe new behavior is stricter: the required flags must appear on the same actual `mcp-probe`\n\nrun step.\n\nThis should pass:\n\n```\n- run: npx @k08200/mcp-probe@latest --config mcp-probe.config.json --github-summary --fail-on-warn\n```\n\nThis should not count as a complete gate:\n\n```\n- run: npx @k08200/mcp-probe --config mcp-probe.config.json\n- run: npx @k08200/mcp-probe ./server.js --github-summary --fail-on-warn\n```\n\nThe flags are present somewhere in the workflow, but no single run step proves the intended config is actually being checked with CI summaries and strict warning handling.\n\nThat is the difference between \"we have a gate\" and \"the gate is enforcing the thing we trust.\"\n\nFor config-based checks, you can declare the expected tool catalog:\n\n```\n{\n  \"servers\": [\n    {\n      \"name\": \"datadog\",\n      \"target\": \"https://mcp.example.com/mcp\",\n      \"transport\": \"http\",\n      \"headers\": {\n        \"Authorization\": \"Bearer ${DATADOG_MCP_TOKEN}\"\n      },\n      \"expectedTools\": [\"logs_query\"],\n      \"forbiddenTools\": [\"delete_dashboard\", \"rotate_api_key\"],\n      \"toolsFile\": \"./datadog.tools.json\"\n    }\n  ]\n}\n```\n\nIf `expectedTools`\n\nand `toolsFile`\n\nare both set, every expected tool needs a sidecar sample input.\n\nThat means CI checks not just \"is the tool advertised?\" but \"did we actually provide a meaningful dry-run sample for the tool an agent depends on?\"\n\nAuto-generated inputs are useful for smoke tests, but they mostly hit schema validation.\n\nReal readiness checks need meaningful inputs:\n\n```\n{\n  \"tools\": {\n    \"logs_query\": {\n      \"input\": {\n        \"query\": \"service:web status:error\",\n        \"timeframe\": \"1h\"\n      },\n      \"expect\": {\n        \"status\": \"pass\",\n        \"not_error_code\": [401, 403],\n        \"requiredFields\": [\"source\", \"freshness\"],\n        \"maxRows\": 100\n      }\n    }\n  }\n}\n```\n\nFor database-backed MCP servers, these assertions are the interesting part:\n\n```\nnpm install -D @k08200/mcp-probe\n```\n\nOr run directly:\n\n```\nnpx @k08200/mcp-probe@latest doctor\nnpx @k08200/mcp-probe@latest --config mcp-probe.config.json --github-summary --fail-on-warn\n```\n\nGitHub: [https://github.com/k08200/mcp-probe](https://github.com/k08200/mcp-probe)\n\nnpm: [https://www.npmjs.com/package/@k08200/mcp-probe](https://www.npmjs.com/package/@k08200/mcp-probe)\n\nThe goal is simple: CI for MCP should test the contract an agent will actually depend on, not just whether the process starts.", "url": "https://wpnews.pro/news/mcp-ci-gates-need-receipts-tools-list-is-not-enough", "canonical_source": "https://dev.to/k08200/mcp-ci-gates-need-receipts-toolslist-is-not-enough-29o4", "published_at": "2026-05-28 11:44:32+00:00", "updated_at": "2026-05-28 11:52:51.416421+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "ai-agents", "mlops"], "entities": ["MCP", "mcp-probe", "GitHub Actions"], "alternates": {"html": "https://wpnews.pro/news/mcp-ci-gates-need-receipts-tools-list-is-not-enough", "markdown": "https://wpnews.pro/news/mcp-ci-gates-need-receipts-tools-list-is-not-enough.md", "text": "https://wpnews.pro/news/mcp-ci-gates-need-receipts-tools-list-is-not-enough.txt", "jsonld": "https://wpnews.pro/news/mcp-ci-gates-need-receipts-tools-list-is-not-enough.jsonld"}}