A deep-dive into the security posture of real-world AI agent deployments — and the open-source tool I built to fix it.
I collected 1,200 real MCP (Model Context Protocol) configuration files from public GitHub repositories, scanned them with an open-source security tool I built, and found that:
The tool is Pluto AgentGuard. It's free, runs locally, and takes about 3 minutes to scan 1,200 configs.
The AI security conversation has focused heavily on what LLMs say — hallucinations, jailbreaks, harmful content. Entire product categories exist for prompt filtering and output guardrails.
But the attack surface has shifted. Modern AI agents don't just generate text — they do things: browse the web, execute shell commands, query databases, push code, trigger CI/CD pipelines. The Model Context Protocol (MCP) is the dominant standard for connecting these capabilities to LLMs.
Here's the disconnect: nobody is auditing the configuration layer that determines what agents can actually do. The MCP config file — usually claude_desktop_config.json
or .mcp.json
— is the security boundary between "an AI assistant that helps me code" and "an AI assistant that can run arbitrary commands on my machine."
I wanted to know: how secure are these configurations in the real world?
I used the GitHub Code Search API to find real MCP configuration files across public repositories. The search targeted:
claude_desktop_config.json
files containing mcpServers
.mcp.json
files with MCP server definitionsmcp_config.json
and similar variantsCollection rules:
Result: 1,200 valid configs from 1,159 unique repositories, collected June 25, 2026.
Each config was scanned using Pluto AgentGuard's scan_mcp_config
function, which checks for:
http://
or https://
URLs) without auth headers or tokensmax_tokens
, max_response_length
) and session caps (max_turns
, session_timeout
)Each finding is assigned a severity (CRITICAL / HIGH / MEDIUM / LOW / INFO) and mapped to OWASP Agentic AI threat categories.
The entire scan ran locally in ~3 minutes. No API keys. No cloud. No LLM calls.
| Metric | Value |
|---|---|
| Total configs scanned | 1,200 |
| Unique repositories | 1,159 |
| Total findings | 2,904 |
| 🔴 CRITICAL | 88 (3.0%) |
| 🟠 HIGH | 280 (9.6%) |
| 🟡 MEDIUM | 2,536 (87.3%) |
| Configs with CRITICAL or HIGH | 20.7% |
| Configs with any finding | 100% |
Every single config had at least a MEDIUM finding. One in five had a CRITICAL or HIGH issue.
I also separately scanned the 11 highest-starred MCP servers to see how the most popular, most copied configs look:
| Server | Stars | Max Severity | Key Finding |
|---|---|---|---|
| Context7 | 58K | 🔴 CRITICAL | No authentication on remote endpoint |
| Chrome DevTools MCP | 44K | 🔴 CRITICAL | Full Chrome DevTools Protocol access, no HITL |
| Playwright MCP | 34K | 🟠 HIGH | Full browser automation, no HITL |
| GitHub MCP | 31K | 🟠 HIGH | Can merge PRs + trigger CI/CD, no HITL |
| Serena | 26K | 🔴 CRITICAL | Unrestricted shell execution, no HITL |
| FastMCP | 26K | 🟡 MEDIUM | Context safety gaps |
| Activepieces | 23K | 🔴 CRITICAL | No authentication on remote endpoint |
| n8n MCP | 22K | 🟠 HIGH | Arbitrary code execution via workflows, no HITL |
| Google MCP Toolbox | 16K | 🟠 HIGH | Unrestricted SQL (supports 20+ databases), no HITL |
| Figma MCP | 15K | 🟡 MEDIUM | External content injection risk |
| mcp-chrome | 12K | 🔴 CRITICAL | No auth + insecure HTTP transport |
5 CRITICAL. 4 HIGH. 0 of 11 had response limits or session caps.
I've filed security issues on the CRITICAL repos: Context7, Chrome DevTools, Serena, Activepieces, mcp-chrome.
Chrome DevTools MCP (44K★) gives the agent full Chrome DevTools Protocol access. That means:
✅ Attach to your existing Chrome sessions
✅ Execute JavaScript in page context
✅ Capture network response bodies (credentials, tokens, PII)
✅ Read cookies and local storage
✅ Intercept and modify requests
A prompt injection — say, a malicious instruction hidden in a webpage the agent is reading — can instruct the agent to exfiltrate your session cookies from Gmail, your bank, or your corporate SSO.
The default config has zero approval gates. The agent acts autonomously.
Serena (26K★) gives the agent unrestricted shell access. Not "run this safe command" — full bash
with the agent's user permissions. Combined with filesystem read/write, a prompt injection can:
~/.ssh/id_rsa
and exfiltrate it.bashrc
for persistence~/.aws/credentials
Context7 (58K★) and Activepieces (23K★) expose remote MCP endpoints over HTTPS with no authentication. Anyone who knows the URL can connect.
The typical config looks like:
{
"mcpServers": {
"context7": {
"url": "https://mcp.context7.com/mcp"
}
}
}
No API key. No OAuth. No mTLS. The equivalent of deploying a REST API with no auth and hoping nobody finds it.
Zero of 1,200 configs set max_response_length
or max_tokens
on their MCP servers. This enables context stuffing attacks: a malicious tool returns an oversized response that pushes the agent's system prompt and safety instructions out of the context window.
This is the lowest-effort fix imaginable — add two lines to your config — and nobody does it.
The current AI security stack looks like this:
[Prompt Filters] → [LLM] → [Output Guardrails] → [Agent Actions]
✅ covered ✅ covered ❌ unmonitored
Teams invest in prompt injection detection and output filtering. But the agent action layer — what the LLM actually does through MCP tools — is a blind spot. There's no "firewall" between the LLM's tool-use decision and the actual execution.
This is the "left of boom" problem. By the time an output guardrail catches something, the agent has already:
You need to catch the risk before the agent gets access to these capabilities. That means auditing the configuration layer.
I built Pluto AgentGuard to fill this gap. It's a security launch gate for AI agents — you run it before deploying, not after something breaks.
| Command | What it does |
|---|---|
aguard scan |
|
| Static analysis of MCP configs, secrets, permissions | |
aguard test |
|
| 22 attack scenarios across 6 packs test your policy's coverage | |
aguard whatif |
|
| Simulate policy changes and see risk delta before applying | |
aguard owasp |
|
| Map findings to 20 OWASP-inspired controls | |
aguard evidence |
|
| Generate launch readiness evidence packets | |
aguard baseline |
|
| Create baselines, detect configuration drift over time | |
aguard monitor |
|
| Replay agent traces, detect unauthorized tool calls |
pip install pluto-aguard
aguard scan ./your-project/
aguard test --policy ./policy.yaml --attack-pack all
aguard whatif --config ./config.yaml
aguard owasp ./your-project/
Most MCP security tools do config scanning. AgentGuard adds three things I haven't seen elsewhere:
Policy testing (aguard test
): Instead of "does your config have issues?", it asks "does your policy actually stop attacks?" — 22 scenarios covering prompt injection, data exfiltration, privilege escalation, context manipulation, supply chain, and social engineering.
What-if simulation (aguard whatif
): Before you add a new MCP server or change a policy rule, simulate the impact. See the risk score delta. Catch regressions before they ship.
Evidence generation (aguard evidence
): Produces a structured evidence packet (scan results + test results + OWASP mapping + risk score) for security review sign-off. Useful for enterprise teams that need launch gates with artifacts.
AgentGuard ships as a GitHub Action:
- uses: arpitha-dhanapathi/pluto-aguard@v0.9.2
with:
scan-path: ./
fail-on: high # Block PR if HIGH or CRITICAL found
format: sarif # Upload to GitHub Security tab
It also supports JSON, Markdown, HTML, and SARIF output formats.
If you're using MCP servers in any AI agent setup, here's a 5-minute security checklist:
pip install pluto-aguard
aguard scan ./your-project/
Add to every MCP server in your config:
{
"max_response_length": 8000,
"max_turns": 20,
"session_timeout": 3600
}
If you use Chrome DevTools, Playwright, Serena, filesystem, or any shell-capable server — enable human-in-the-loop approval. The exact mechanism depends on your client (Claude Desktop, Cursor, VS Code, etc.), but the principle is: the agent should ask before executing destructive operations.
If your MCP server is remote (HTTPS URL instead of stdio), add auth:
{
"mcpServers": {
"my-server": {
"url": "https://my-server.com/mcp",
"headers": {
"Authorization": "Bearer ${MCP_API_KEY}"
}
}
}
}
Block PRs that introduce MCP misconfigurations:
- uses: arpitha-dhanapathi/pluto-aguard@v0.9.2
with:
scan-path: ./
fail-on: high
MCP is 18 months old and already the de facto standard for agent-to-tool communication. The ecosystem is moving fast — 90K+ stars on awesome-mcp-servers, thousands of servers, and major platforms (Claude, Cursor, VS Code, Windsurf) supporting it natively.
But the security tooling hasn't kept pace. We're in the "move fast and break things" phase of agent infrastructure, and the configs people are shipping to production look like the web in 2005 — no auth, no limits, full trust.
The good news: the fixes are simple. Auth headers, response limits, HITL approval, and a scan in CI. None of this requires new technology — just applying existing security principles to a new surface.
The bad news: right now, almost nobody is doing it.
Let's fix that.
Pluto AgentGuard is open-source (Apache 2.0), written in Python, and runs entirely locally. Star it on GitHub if this was useful.
Have questions or findings to share? Open an issue or find me on LinkedIn.
Tags: #security #ai #opensource #python #mcp #agents