{"slug": "we-scanned-12-popular-mcp-servers-the-most-interesting-finding-was-our-own-false", "title": "We scanned 12 popular MCP servers. The most interesting finding was our own false positives.", "summary": "A developer built mcp-customs, a free offline CLI that checks MCP servers for security risks before installation. Scanning 12 popular MCP servers revealed that 11 of 12 had zero permission or scope declarations in their manifest, and the tool initially produced false positives by flagging test code and comments as runtime issues. After filtering, two real-code patterns were flagged as worth maintainer attention.", "body_md": "We built `mcp-customs`\n\n, a free, offline CLI that checks an MCP server for\n\ncommon security risks before you install it — think `npm audit`\n\n, but\n\nfor the servers your AI agent connects to. Before asking anyone to use\n\nit, we pointed it at 12 real, popular MCP servers and read every single\n\nfinding by hand. Here's what actually held up.\n\nWe pulled the current top MCP-related repos on GitHub by star count and\n\nscanned each one as-is, no cherry-picking:\n\n| Server | Stars | Score | Stamp |\n|---|---|---|---|\n| github/github-mcp-server | 30.8k | 97/100 | CLEARED |\n| BeehiveInnovations/pal-mcp-server | 11.6k | 0/100 | FLAGGED* |\n| firecrawl/firecrawl-mcp-server | 6.6k | 97/100 | CLEARED |\n| exa-labs/exa-mcp-server | 4.6k | 97/100 | CLEARED |\n| makenotion/notion-mcp-server | 4.4k | 29/100 | FLAGGED |\n| antvis/mcp-server-chart | 4.2k | 94/100 | CLEARED |\n| haris-musa/excel-mcp-server | 3.9k | 97/100 | CLEARED |\n| cloudflare/mcp-server-cloudflare | 3.9k | 0/100 | FLAGGED |\n| browserbase/mcp-server-browserbase | 3.4k | 22/100 | FLAGGED |\n| blazickjp/arxiv-mcp-server | 2.9k | 94/100 | CLEARED |\n| Jpisnice/shadcn-ui-mcp-server | 2.8k | 0/100 | FLAGGED* |\n| stickerdaniel/linkedin-mcp-server | 2.4k | 94/100 | CLEARED |\n\n*see below — these two scores don't mean what they look like they mean.\n\nEleven of twelve servers had **zero** permission or scope declaration in\n\ntheir manifest. Not \"weak\" declarations — none at all. This isn't a\n\nranking judgment on any one project; right now there's no real convention\n\nfor it. If you're building an MCP server, declaring what it actually\n\nneeds (filesystem? network? shell?) is the single easiest thing you can\n\ndo to make a client's \"do you want to allow this?\" prompt mean something.\n\nBefore publishing anything, we split every finding into \"runtime code\"\n\nvs. \"test/dev/script code\" — because a `subprocess.run(shell=True)`\n\nin a\n\ntest fixture is a very different thing from the same line in a request\n\nhandler. Once we did that:\n\n`tests/`\n\nor `simulator_tests/`\n\n— fake API keys used to test a `shell=True`\n\ncall in a test for a security-audit\nfeature. Runtime-code findings: `execSync()`\n\ncalls in a release-versioning script\n(`scripts/bump-version.js`\n\n) — not reachable by an agent, just a\nmaintainer running `npm version`\n\n.`eval()`\n\ncall in notion-mcp-server as critical.\nIt was inside a `//`\n\ncomment. We fixed comment-stripping before\nre-running anything in this post — an earlier draft of this table\nwould have been wrong.A heuristic scanner that can't tell test code from runtime code, or a\n\ncomment from a statement, isn't very useful. We'd rather show you where\n\nit broke than publish the inflated numbers.\n\nAfter filtering out test/dev noise, two real-code patterns remained that\n\nwe think are legitimately worth the maintainers' attention — not\n\nconfirmed vulnerabilities, just the exact shape of thing this category\n\nof tool exists to surface:\n\n`sandbox.container.app.ts`\n\n: a\nfile read and a file write both take a variable named `reqPath`\n\ndirectly into `fs.readFile`\n\n/ `fs.writeFile`\n\n. We didn't trace the full\ncall path to confirm whether it's constrained upstream — that's a\nfive-minute check for someone who knows the codebase, which is exactly\nthe point of flagging it rather than asserting it.`src/init-server.ts`\n\n: reads a spec file from a\npath resolved at startup. Lower stakes — looks like a local config\npath, not something an agent's tool call controls — but same category.Everything else that scored a CLEARED or a high number had, at most, the\n\nmissing-permissions finding from #1.\n\nDon't read the scores in the table above as a safety ranking — read\n\nfinding #2 first. A FLAGGED stamp from a heuristic tool like this means\n\n\"go look,\" not \"don't install.\" Several of today's FLAGGED results\n\nwould be CLEARED if the tool only understood that a test directory isn't\n\na runtime path, which is a limitation of the tool, not the project.\n\n```\nnpx mcp-customs scan ./path-to-some-mcp-server\n```\n\nFully offline, zero telemetry, free, Apache-2.0. The rules and the\n\nscanner itself are about 250 lines — read all of them in five minutes,\n\nwhich is more than you can say for most security tools.\n\nIf you maintain one of the servers above and want help interpreting (or\n\narguing with) a finding, open an issue. If you maintain a different MCP\n\nserver and want to run this yourself before we do, that's the whole\n\npoint — we'd rather you find your own false positives than us find\n\nthem for you in public.", "url": "https://wpnews.pro/news/we-scanned-12-popular-mcp-servers-the-most-interesting-finding-was-our-own-false", "canonical_source": "https://dev.to/mcpcustoms/we-scanned-12-popular-mcp-servers-the-most-interesting-finding-was-our-own-false-positives-kcf", "published_at": "2026-06-18 13:04:26+00:00", "updated_at": "2026-06-18 13:22:03.523674+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "ai-safety"], "entities": ["mcp-customs", "GitHub", "BeehiveInnovations", "firecrawl", "exa-labs", "makenotion", "antvis", "cloudflare"], "alternates": {"html": "https://wpnews.pro/news/we-scanned-12-popular-mcp-servers-the-most-interesting-finding-was-our-own-false", "markdown": "https://wpnews.pro/news/we-scanned-12-popular-mcp-servers-the-most-interesting-finding-was-our-own-false.md", "text": "https://wpnews.pro/news/we-scanned-12-popular-mcp-servers-the-most-interesting-finding-was-our-own-false.txt", "jsonld": "https://wpnews.pro/news/we-scanned-12-popular-mcp-servers-the-most-interesting-finding-was-our-own-false.jsonld"}}