{"slug": "are-your-mcp-server-s-safety-labels-honest-a-one-command-check-and-what-it-found", "title": "Are your MCP server's safety labels honest? A one-command check (and what it found on 31 popular servers)", "summary": "A developer created actlint, a tool that checks whether MCP servers' safety labels match their actual capabilities. Scanning 541 tools across 31 popular servers—including official ones from Stripe, GitHub, Microsoft, Google, MongoDB, and PayPal—found that 11 servers were completely clean, while four carried labels that deny something the tool's own schema says it can do. The tool runs as a one-command check and can be integrated into CI pipelines.", "body_md": "You wrote this once, on the day the tool was born:\n\n```\n{\n  \"name\": \"search_docs\",\n  \"annotations\": {\n    \"readOnlyHint\": true,\n    \"destructiveHint\": false,\n    \"openWorldHint\": false\n  }\n}\n```\n\nThen the tool kept changing. A parameter got added. A backend call got wired in. The schema grew.\n\nThe annotations block stayed exactly as you left it.\n\nThe MCP spec calls these annotations hints and tells clients not to trust them blindly. Plenty of clients use them anyway, because they arrive at tool-list time and they are cheap to act on. A common pattern: `readOnlyHint: true`\n\nmeans skip the confirmation prompt.\n\nSo a stale label costs more than a confused reviewer. It can quietly remove a safety step that would otherwise have fired.\n\nThe protocol has no step that re-checks the label against the schema as the tool evolves. That is the gap [actlint](https://github.com/formael/actlint) fills. It reads what a server advertises over `tools/list`\n\n, works out what each tool most likely does from the name, description, and input schema, and reports where that disagrees with what the tool declares. It never calls a tool.\n\nWe pointed it at 541 tools across 31 widely used servers, including official ones from Stripe, GitHub, Microsoft, Google, MongoDB, and PayPal. Eleven came back completely clean. Four carry a label that denies something the tool's own schema says it can do. Full scorecard, per-server grades, methodology, and the two findings that did not survive our own hand review: ** MCP Safety Labels Run on the Honor System**.\n\nThis post is about running it on yours.\n\nLocal stdio server, launched by actlint:\n\n```\nnpx actlint npx -y @modelcontextprotocol/server-filesystem .\n```\n\nHosted server over HTTP, with a credential if it needs one to list its tools:\n\n```\nnpx actlint --http https://mcp.example.com/mcp --header \"Authorization: Bearer ${MCP_TOKEN}\"\n```\n\nServer that needs an environment variable to start.\n\n```\nnpx actlint --env STRIPE_SECRET_KEY npx -y @stripe/mcp --tools=all\n```\n\nTalking to the server is the only step that touches the network. Everything after it is pure, so you can save the manifest and replay it as often as you like with identical results:\n\n```\n# Once, where the credential lives\nnpx actlint --capture tools.json --http https://mcp.example.com/mcp --header \"Authorization: Bearer ${MCP_TOKEN}\"\n\n# Every run after this, offline, no credential, no socket\nnpx actlint --manifest tools.json --fail-on high\nnpx actlint --manifest tools.json --json  -o report.json\nnpx actlint --manifest tools.json --sarif -o results.sarif\n```\n\nThat split is also the right shape for CI: capture in a job with network access, gate in one without.\n\n```\n- uses: formael/actlint/packages/github-action@df516d81ba3719687d1d72227737887c3534e1db # v0.1.0\n  with:\n    args: --manifest manifest.json --fail-on high\n```\n\nTurning it on for a server that already has findings, without a red build on day one:\n\n```\nnpx actlint --manifest tools.json --write-baseline baseline.json\nnpx actlint --manifest tools.json --baseline baseline.json --fail-on low\n```\n\nBaseline entries are keyed by fingerprint, so they survive re-ordering and unrelated edits.\n\n```\n  actlint  ▸  stdio server                                      honesty grade: A\n  ──────────────────────────────────────────────────────────────────────────────\n  14 tools scanned · 0 under-declared · 0 undeclared · 0 over-declared\n\n  advisories — capability hygiene, not honesty verdicts\n  ◐ ADV   move_file  no-scope-constraint\n      This is a sensitive action whose input schema carries no narrowing constraint …\n      ↳ OWASP ASI02:2026 · OWASP MCP02:2025 · CoSAI MCP-T3 · EU AI Act Art.15 · NIST …\n```\n\nThree verdicts, in order of how much they should worry you:\n\n| Verdict | Meaning |\n|---|---|\nunder-declared |\nThe tool claims less risk than it has. This is the one that matters. |\nundeclared |\nSays nothing either way. The spec default already protects the client, so this is a nudge. |\nover-declared |\nHonest excess caution, paid for in alarm fatigue. |\n\nAdvisories are capability notes rather than honesty verdicts, so they never move the grade and never fail a build.\n\nThe grade is an honesty grade, not a safety score. A server full of honestly declared destructive tools earns an A. Any under-declared finding caps it at B.\n\nEvery finding carries a plain-English rationale and a mapping to OWASP ASI, the OWASP MCP Top 10, CoSAI, the EU AI Act, and NIST. Any rule id explains itself offline:\n\n```\nnpx actlint explain write-as-readonly\n```\n\nTwo design choices are worth knowing before you trust the output.\n\n**It doesn't look at your annotations while deriving.** The derivation step reads only the name, description, and input schema. Your declared hints are the thing being audited, not an input to the reasoning, so you can't nudge the linter toward a friendlier answer by editing the label it is grading.\n\n**The schema is read structurally, not as text.** A `format: \"uri\"`\n\nfield nested a few objects deep gets found. The substring \"url\" sitting inside a description does not create a signal. Schema shape is weighted more heavily than the name and description signals, which are natural language and therefore ambiguous.\n\nWhere signals conflict or are missing, the result resolves toward concern with lowered confidence. `uncertain`\n\nis a real, non-failing outcome. Under-claiming a risk beats crying wolf, because a linter people stop believing is not worth running.\n\nIt is a linter, in the lineage of `eslint`\n\n, `tflint`\n\n, and `hadolint`\n\n. It never runs, routes, or blocks anything. It reads labels and tells you whether they hold up.\n\nTool poisoning, runtime over-privilege, and auth are real problems, and they belong to other tools.\n\nA clean grade is not a safety certificate. It means the server told the truth about what its tools can do. Those tools can still be dangerous, and truthful labels are what let a client handle that properly.\n\n```\nnpx actlint <command-to-launch-your-server>\nnpx actlint --http https://your-mcp-host/mcp\n```\n\nIf it gets your server wrong, that report is the most useful thing you can send us. A false positive caught by the maintainer who knows the tool is how the rules improve.\n\nRepo: [github.com/formael/actlint](https://github.com/formael/actlint)\n\nWhat does your server grade?", "url": "https://wpnews.pro/news/are-your-mcp-server-s-safety-labels-honest-a-one-command-check-and-what-it-found", "canonical_source": "https://dev.to/formael/are-your-mcp-servers-safety-labels-honest-a-one-command-check-and-what-it-found-on-31-popular-1ml3", "published_at": "2026-07-26 07:34:04+00:00", "updated_at": "2026-07-26 07:59:12.086609+00:00", "lang": "en", "topics": ["ai-safety", "developer-tools", "ai-infrastructure"], "entities": ["Stripe", "GitHub", "Microsoft", "Google", "MongoDB", "PayPal", "actlint", "formael"], "alternates": {"html": "https://wpnews.pro/news/are-your-mcp-server-s-safety-labels-honest-a-one-command-check-and-what-it-found", "markdown": "https://wpnews.pro/news/are-your-mcp-server-s-safety-labels-honest-a-one-command-check-and-what-it-found.md", "text": "https://wpnews.pro/news/are-your-mcp-server-s-safety-labels-honest-a-one-command-check-and-what-it-found.txt", "jsonld": "https://wpnews.pro/news/are-your-mcp-server-s-safety-labels-honest-a-one-command-check-and-what-it-found.jsonld"}}