{"slug": "i-built-a-security-scorer-for-213-mcp-servers-here-s-what-made-the-difference", "title": "I Built a Security Scorer for 213 MCP Servers — Here's What Made the Difference", "summary": "A developer built RepoAI, a security scoring system for 213 MCP servers, after finding that existing directories only mirror GitHub metadata without assessing risk. The tool runs 15 checks across repo health and MCP-specific criteria, producing a 0–100 score, and deliberately avoids using an LLM for scoring to prevent marketing bias. The project is open at repoai.io and plans to expand to Claude Skills and other AI-tool ecosystems.", "body_md": "Every time I wanted to add an MCP server to Claude Desktop or Cursor, I ended up\n\nwith a dozen browser tabs open, trying to answer one question the README never\n\nanswers: **is this actually safe to give tool-call access to my machine?**\n\nMCP (Model Context Protocol) directories exist, but most of them just mirror\n\nGitHub — stars, a README excerpt, a copy-paste config. None of them score\n\n*risk*. So I built [RepoAI](https://repoai.io), and I want to share the exact\n\nmethodology, because I think it's more interesting than the product pitch.\n\nStars measure popularity, not safety. A server with 5,000 stars can still:\n\nNone of that shows up in a star count.\n\n`computeSecurityScore()`\n\nruns 15 checks against each server, sums signed point\n\nvalues, and clamps the result to 0–100. Two categories:\n\n**Repo health** (max +101):\n\n| Check | Points | Why |\n|---|---|---|\n| Official maintainer | +18 / 0 | Vendor-published > random fork |\n| Editorially verified | +5 / 0 | A human actually reviewed it |\n| Actively maintained | +12 / +6 / 0 | Last commit ≤90d / ≤365d / older |\n| Established contributor base | +10 / +5 / 0 | Bus factor: ≥5 / ≥2 / 1 |\n| Community adoption | +10 / +5 / 0 | Stars: ≥1000 / ≥100 / fewer |\n| Has license | +8 / 0 | Unlicensed = unclear usage terms |\n| Reasonable dependency footprint | +5 / +2 / 0 | Smaller supply-chain surface |\n| Has homepage/docs | +3 / 0 | |\n| Substantial README | +5 / 0 | ≥500 chars |\nArchived |\n0 / −40\n|\nNo future fixes. Ever. |\n\n**MCP-specific:**\n\n| Check | Points | Why |\n|---|---|---|\n| Read-only mode available | +15 / 0 | A safer way to run it should exist |\n| Supports OAuth | +10 / 0 | Stronger than static tokens |\n| Dangerous tools present | 0 / −10 / −25 | Share of tools flagged execute/delete/write |\n| Dangerous tools, no safe mode | 0 / −20\n|\nThe double penalty — risky and no escape hatch |\n| Weak or missing auth | 0 / −15 | Anyone with the config file can use it |\n\nScores land in three bands: 80–100 `safe`\n\n, 60–79 `medium`\n\n, 0–59 `high`\n\nrisk.\n\nMy first instinct was to let an LLM write the verdict *and* set the score.\n\nThat's a mistake — an AI can be convinced by good marketing copy in a README\n\njust as easily as a human can. So the pipeline splits it:\n\nFor a server like `qdrant/mcp-server-qdrant`\n\n, the pipeline:\n\n`medium`\n\nrisk, mostly\nheld back by no OAuth support and a moderate dependency footprint)213 servers, 36 categories, all re-scraped and re-scored on a rolling basis\n\nso a score reflects last week's commit activity, not last year's.\n\nIt's [open at repoai.io](https://repoai.io) — MCP is the first layer, Claude\n\nSkills and the rest of the AI-tool ecosystem are next, same trust model.\n\nCurious what other people would want a security scorer to check for —\n\nwhat's bitten you when installing an MCP server?", "url": "https://wpnews.pro/news/i-built-a-security-scorer-for-213-mcp-servers-here-s-what-made-the-difference", "canonical_source": "https://dev.to/ismail_khouya_4643b00c238/i-built-a-security-scorer-for-213-mcp-servers-heres-what-made-the-difference-2hi4", "published_at": "2026-07-26 16:52:35+00:00", "updated_at": "2026-07-26 17:30:46.889323+00:00", "lang": "en", "topics": ["ai-safety", "developer-tools", "ai-agents", "ai-infrastructure"], "entities": ["RepoAI", "Claude Desktop", "Cursor", "Model Context Protocol", "GitHub", "qdrant/mcp-server-qdrant"], "alternates": {"html": "https://wpnews.pro/news/i-built-a-security-scorer-for-213-mcp-servers-here-s-what-made-the-difference", "markdown": "https://wpnews.pro/news/i-built-a-security-scorer-for-213-mcp-servers-here-s-what-made-the-difference.md", "text": "https://wpnews.pro/news/i-built-a-security-scorer-for-213-mcp-servers-here-s-what-made-the-difference.txt", "jsonld": "https://wpnews.pro/news/i-built-a-security-scorer-for-213-mcp-servers-here-s-what-made-the-difference.jsonld"}}