{"slug": "building-a-github-stats-mcp-server-with-security-metrics", "title": "Building a GitHub Stats MCP Server with Security Metrics", "summary": "A developer built a GitHub Stats MCP Server that integrates CHAOSS security metrics to assess open source project health. The server uses five tools aligned with the CHAOSS Security Practitioner Guide, including a Libyears calculator that measures dependency lag by querying GitHub's Dependency Graph SBOM API and PyPI. The project connects the MCP server to an external API for the first time, adding Resources and Prompts structures alongside Tools to help teams evaluate dependency security risks.", "body_md": "*👋 This is the second chapter of a series where I document what I'm learning about Model Context Protocol Architecture and Tool implementations*\n\nIn [Chapter 1](https://dev.to/anajsana95/the-weekend-i-fell-down-the-mcp-rabbit-hole-a), I built a simple Calculator MCP Server. This time, I connected my MCP server to an external API, added the two other MCP structures (Resources and Prompts), and ended up with something useful for teams evaluating open source dependencies and ecosystem health: a security risk assessment tool powered by[CHAOSS](https://chaoss.community/) metrics, helping practitioners better interpret project health\n\nLet's first get into the theory and new concepts\n\nWe briefly mentioned in the last post about the MCP's primitives:\n\nTools: Grants agency to the AI and are functions the LLM executes (e.g., get_repo_info())\n\nResources: Provides safe, contextual data the LLM can read a URL, a file, or an API response\n\nPrompts: Structure the conversation with expert context templates (e.g., *You are a data scientist analyzing CHAOSS metrics*)\n\nFor this project, I needed all three: Tools to fetch GitHub data, Resources to load the CHAOSS guide, and Prompts to give the LLM the right expert context.\n\n[CHAOSS](https://chaoss.community/) is a Linux Foundation project that develops metrics and frameworks for measuring the health of open source communities.\n\nTheir [Practitioner Guides](https://chaoss.community/about-chaoss-practitioner-guides/) are particularly useful because they take complex community health topics, such as security, contributor sustainability, and responsiveness, and translate them into actionable metrics. These metrics can help anyone interpreting open source project data develop insights to improve the project’s health.\n\nThe one I focused on building my server is the CHAOSS Security Practitioner Guide, which centers on three primary metrics:\n\nLibyears concept: if your project uses a dependency that's 2 years behind its latest release, that's 2 libyears of lag. Studies show that projects with high libyears are 4x more likely to have security vulnerabilities.\n\nThese three metrics together give a starting point for assessing the security posture of any open source project, which is something that an Open Source Manager in the OSPO could need when evaluating dependencies or reporting to security teams.\n\nUsed the same stack as Chapter 1 (Python, uv, FastMCP) with two extra dependencies:\n\n`httpx`\n\n: modern Python HTTP client for calling the GitHub API`python-dotenv`\n\n: for managing the GitHub token securely via a .env file.The key difference from Chapter 1 is that your MCP server now talks to an external API. I added five tools aligned wiht the CHAOSS practitioner guide to security:\n\n``` python\nfrom mcp.server.fastmcp import FastMCP\nimport httpx, os\nfrom dotenv import load_dotenv\n\nload_dotenv()\nGITHUB_TOKEN = os.getenv(\"GITHUB_TOKEN\")\nHEADERS = {\n    \"Authorization\": f\"Bearer {GITHUB_TOKEN}\",\n    \"Accept\": \"application/vnd.github+json\"\n}\n\nmcp = FastMCP(\"github-stats\")\n\n@mcp.tool()\ndef get_repo_info(owner: str, repo: str) -> dict:\n    \"\"\"Get basic info and stats for a GitHub repository\"\"\"\n    ...\n\n@mcp.tool()\ndef get_release_frequency(owner: str, repo: str) -> dict:\n    \"\"\"Get release frequency info\"\"\"\n    ...\n\n@mcp.tool()\ndef get_security_advisories(owner: str, repo: str) -> dict:\n    \"\"\"Get known security advisories for a GitHub repository\"\"\"\n    ...\n\n@mcp.tool()\ndef get_dependency_info(owner: str, repo: str) -> dict:\n    \"\"\"Check what dependency files exist in a repo\"\"\"\n    ...\n\n@mcp.tool()\ndef get_libyears(owner: str, repo: str) -> dict:\n    \"\"\"Calculate CHAOSS Libyears metric\"\"\"\n    ...\n```\n\nThe last tool called Libyears was the most interesting to add. It uses GitHub's Dependency Graph SBOM API to get all dependencies, then queries PyPI for each package to find how far behind the current version is from the latest. A high libyears score means outdated dependencies, which, according to CHAOSS, correlates with higher security risk.\n\nThis is where MCP gets powerful beyond just tool calling.\n\n``` php\n@mcp.resource(\"chaoss://security-guide\")\ndef get_chaoss_security_guide() -> str:\n    \"\"\"CHAOSS Security Practitioner Guide content\"\"\"\n    response = httpx.get(\"https://chaoss.community/practitioner-guide-security/\")\n    import re\n    text = re.sub(r'<[^>]+>', ' ', response.text)\n    text = re.sub(r'\\s+', ' ', text).strip()\n    return text[:8000]\n```\n\nThis fetches the CHAOSS Security Practitioner Guide and makes it available to the LLM as a readable document. The host can load it into context when needed — so the model doesn't just have your tool results, it has the actual framework for interpreting them.\n\n``` php\n@mcp.prompt()\ndef chaoss_security_analyst() -> str:\n    \"\"\"Expert prompt for CHAOSS security risk assessment from an OSPO perspective\"\"\"\n    return \"\"\"\nYou are a data scientist specializing in open source security metrics for OSPOs.\n\nYour role is to analyze GitHub repositories using CHAOSS Security Practitioner Guide \nmetrics and produce clear security risk assessments that OSPOs can present to their \nsecurity teams.\n\nWhen analyzing a repository, always:\n\n1. INTERPRET metrics using these CHAOSS thresholds:\n   Libyears: 0-2 = low risk, 2-5 = moderate risk, 5+ = high risk\n   Release Frequency: weekly/monthly = healthy, quarterly or less = concern\n   Security Advisories: 0 = good, any critical = immediate action needed\n\n2. CONTEXTUALIZE results considering project size, team, and ecosystem.\n\n3. STRUCTURE your output as a risk assessment report:\n   Executive Summary, Metrics Summary, Risk Level, Recommendations, Why this matters\n\"\"\"\n```\n\nThe Prompt gives the LLM a specific expert role: *you are a data scientist building a security report for an OSPO to present to their security team*\n\nSame workflow as Chapter 1, run your server.py, open the Inspector, verify all 5 tools, 1 resource, and 1 prompt are visible and working.\n\nI decided to test on a large, well-known open source ML framework.\n\nI then connected the server to Goose exactly as in Chapter 1, then asked: *Analyze [insert open source project name] security using all available GitHub-stats tools and give me a risk assessment with recommendations*\n\nAfter some model experimentation (Llama 3.3-70b had trouble with multiple tools, Qwen3-32b worked well), Goose called all 4 tools, synthesized the results with the CHAOSS context, and produced this:\n\n```\nSecurity Risk Assessment — [open source project]\n\nSecurity Advisories: 5 total (including 1 critical — arbitrary code execution when loading malicious PyTorch checkpoint with weights_only=True, published January 2026)\nRelease Frequency: 30 releases, latest v2.12.0 — active and healthy\nOpen Issues: 18,496 — high volume, warrants triage\nRisk Level: HIGH\n\nRecommendations:\n\n🚨 Patch unresolved security advisories immediately\n⚠️ Review high number of open issues — 32% untriaged\n✅ Release cadence is healthy — keep it up\n```\n\nA few considerations when reading this test report:\n\nA \"HIGH\" risk level here does not mean this is a bad or insecure project. For a project of the scale I was analyzing (hundreds of thousands of users, a dedicated security team, etc), these numbers need to be interpreted in context.\n\nThe high open issue count reflects high community activity, not neglect. The disclosed security advisories are being actively managed and publicly reported, which is actually a sign of good security hygiene (projects that disclose vulnerabilities are more trustworthy than those that don't)\n\nCHAOSS metrics are a starting point for conversation, not a final verdict. Security risk always depends on domain context (the same metric means something very different in a healthcare system versus a research prototype).\n\nFinal security decisions should always involve human judgment, domain expertise, and a deeper understanding of how the project is used in your specific context\n\nThe transport method between Goose and our server is `stdio`\n\n. This means they communicate via `stdin/stdout`\n\npipes. This is perfect for local development, but it means your server only exists while our machine is running and Goose is open.\n\nThen, what if I want to share this as an open web dashboard and not just in a local chat interface?\n\n`Streamable HTTP`\n\nis the MCP way of transport that lets an MCP server be exposed over HTTP, rather than only as a local process over `stdio`\n\n. In this scenario, client-to-server communicates using HTTP POST requests. When the server needs to send multiple messages over time, it responds with an [SSE stream](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events) for server-to-client messages.\n\nThere are a few use cases that require that transport:\n\nMCP Remote: server is deployed to the cloud, accessible via a public URL (e.g, This is already how the connectors in claude.ai work when you connect to Asana, Google Drive, etc)\n\nMCP Apps: browser UIs that connect directly to your MCP server without needing a desktop AI host at all. A new experimental feature in the [MCP Inspector](https://modelcontextprotocol.io/docs/tools/inspector).\n\nIf you're curious about where MCP official transport layers need to evolve for edge environments (like manufacturing, healthcare, or automotive) where latency and connectivity can't be taken for granted, this talk explores\n\n[Binary MCP with Protocol Buffers for constrained networks]\n\nWe will be exploring how to build a cloud-deployed version of our MCP server while adding a security layer using [OpenID Connect](https://openid.net/developers/how-connect-works/).", "url": "https://wpnews.pro/news/building-a-github-stats-mcp-server-with-security-metrics", "canonical_source": "https://dev.to/anajsana95/building-a-github-stats-mcp-server-with-security-metrics-3omi", "published_at": "2026-05-31 10:15:41+00:00", "updated_at": "2026-05-31 10:42:42.588053+00:00", "lang": "en", "topics": ["ai-tools", "artificial-intelligence", "large-language-models", "ai-agents"], "entities": ["Model Context Protocol", "GitHub", "CHAOSS", "Linux Foundation", "Ana J. Sana"], "alternates": {"html": "https://wpnews.pro/news/building-a-github-stats-mcp-server-with-security-metrics", "markdown": "https://wpnews.pro/news/building-a-github-stats-mcp-server-with-security-metrics.md", "text": "https://wpnews.pro/news/building-a-github-stats-mcp-server-with-security-metrics.txt", "jsonld": "https://wpnews.pro/news/building-a-github-stats-mcp-server-with-security-metrics.jsonld"}}