# AI Agent Security Audit Checklist: 8 Critical Tests for Production Deployments > Source: > Published: 2026-07-25 06:54:54+00:00 AI agents are no longer experimental. In 2026, enterprises are deploying LLM-powered agents that read databases, execute code, send emails, and control production infrastructure. The question is no longer *"should we use AI agents?"* but *"how do we secure them in production?"* This article is the fourth in our **AI Runtime Security** series. We've covered the [macro landscape](https://dev.to/correctover/the-state-of-ai-security-in-2026-why-every-ai-native-company-needs-a-structured-security-audit-g1l), [MCP penetration testing methodology](https://dev.to/correctover/mcp-penetration-testing-a-practical-guide-for-ai-agent-infrastructure-security-1027), and [why runtime call verification is the missing layer](https://dev.to/correctover_15/ai-security-landscape-2026-why-runtime-call-verification-is-the-missing-layer-2am). Here, we distill that experience into a **practical, actionable checklist** — 8 tests every security team should run before putting AI agents into production. Why this matters:We've audited 10+ AI agent frameworks using the Correctover CCS scanner, producing over1,730 verified findingsacross 12 codebases. Of those,87 are confirmed production vulnerabilities— real bugs in shipping code, not theoretical attack surfaces. Every item on this checklist is grounded in actual vulnerabilities we've found, reported, and in many cases had patched. | # | Test Area | Severity | Frameworks Affected | |---|---|---|---| | 1 | Tool Authorization & Read-Only Enforcement | CRITICAL | MCP SDK, AutoGen, Semantic Kernel, FastMCP, Dify, Griptape | | 2 | MCP/Subprocess Command Injection | CRITICAL | CrewAI, LiteLLM, AutoGen, Docker MCP | | 3 | Deserialization & Eval Injection | CRITICAL | AG2, LlamaIndex, Haystack | | 4 | Path Traversal in Configuration Loading | CRITICAL | AutoGen, Dify | | 5 | Environment Variable Leakage | HIGH | MCP Python SDK, FastMCP | | 6 | MCP Transport Security | HIGH | All STDIO-based MCP implementations | | 7 | Runtime Call Verification (The Missing Layer) | MEDIUM | All frameworks (nobody does this) | | 8 | Model/Provider Supply Chain Security | HIGH | All LLM deployments | **The Problem:** Most AI agent frameworks define a `readOnlyHint` or similar permission flag for tools, but **none of them actually enforce it**. We discovered this at the protocol level in the official MCP Python SDK — the `readOnlyHint` field exists in the schema, but the runtime never checks it before executing a tool call. This means every downstream framework inherits the vulnerability. **How to Test:** ``` # Scan your MCP server definition for readOnlyHint usage grep -r "readOnlyHint" --include="*.py" --include="*.ts" --include="*.json" . # If the property is defined but never checked at the transport layer, # every tool is writable regardless of what the schema says. ``` Also check for tool-level permission enforcement: ``` # Does your framework actually check permissions before execution? # Look for patterns like: if not tool.is_readonly and not user_has_write_permission: raise PermissionError("Write access required") # Most frameworks simply skip this check entirely. ``` **What a Good Result Looks Like:** `readOnlyHint` is enforced at the transport/runtime layer, not just declared in the schema**How Correctover CCS Addresses This:** Our CCS scanner includes rule `AGT-TOOL-NO-READONLY` that specifically detects frameworks where tool permission flags are declared but unenforced. We reported this to the MCP Python SDK maintainers via HackerOne (Report #3878033, CVSS 7.5) and have parallel submissions for AutoGen and Semantic Kernel through MSRC. **The Problem:** The Model Context Protocol (MCP) uses STDIO transport — the parent process spawns a child process and communicates via stdin/stdout. If the command or arguments passed to `stdio_client()` come from untrusted input (LLM output, user config, MCP server manifest), you have a **command injection vulnerability with CVSS 9.8**. Here's the real-world damage: **CrewAI MCP RCE (CVE-2026-2287, CVSS 9.8):** The `StdioTransport.__init__()` in `crewai/mcp/transports/stdio.py` (line 92-97) passes `command` and `args` directly to `stdio_client()` with zero validation. An attacker who controls the MCP server configuration can execute arbitrary OS commands. **Zero protection layers.** Reported to MSRC, case 126356. **LiteLLM allowlist bypass (CVE-2026-30623):** LitellM has an allowlist, but it can be bypassed using `python -c` or `node -e` argument injection. **Docker MCP:** Command injection at the protocol level — the Docker MCP server passes user-controlled parameters directly to subprocess calls. **How to Test:** ``` python # Minimal PoC — test if your framework validates commands import subprocess # If you can inject an unintended command, your framework is vulnerable test_cases = [ "python -c 'import os; os.system(\"calc.exe\")'", "bash -c 'echo $FLAG > /tmp/pwned'", "node -e 'require(\"child_process\").execSync(\"id\")'" ] ``` Check your MCP transport layer for any code that looks like: ``` # UNSAFE — no validation transport = stdio_client(command, args) # SAFE — command must be on allowlist if command not in ALLOWED_COMMANDS: raise SecurityError(f"Command {command} not allowed") ``` **What a Good Result Looks Like:** **How Correctover CCS Addresses This:** Our rules `CW-MCP-001` (CrewAI), `MCP-STDIO-001` (cross-framework), and `command_injection` scanners detect unprotected `stdio_client()` calls. We've submitted findings through MSRC, HackerOne, and GitHub Security Advisories covering CrewAI, LiteLLM, AutoGen, and Docker MCP. **The Problem:** AI agent frameworks frequently use `eval()` , `pickle.loads()` , or `yaml.load()` for configuration parsing, workflow serialization, and context expression evaluation. When LLM-controlled data reaches these functions, the result is **unauthenticated remote code execution**. Verified vulnerabilities: **AG2 (Microsoft) eval() str bypass (CVSS 9.8):** In `autogen/agentchat/group/context_expression.py` line 228, `eval()` escapes string values but `__str__()` return values`context_variables` , which passes a non-string object whose `__str__()` returns malicious code. GitHub Issue #3073. **LlamaIndex Workflows Pickle RCE (CVSS 9.8):** In `workflows/context/serializers.py` line 243, `pickle.loads(base64.b64decode(value))` is the default serializer for workflow state persistence. Classic pickle RCE — fully exploitable. GitHub Issue #22296. **PoC verified.** **Haystack Pipeline RCE (2 CRITICAL):** Two confirmed RCE paths through pipeline serialization deserialization. **How to Test:** ``` # Scan for dangerous function calls in your agent framework grep -rn "eval(" --include="*.py" . | grep -v "test" | grep -v "__pycache__" grep -rn "pickle.loads" --include="*.py" . | grep -v "test" | grep -v "__pycache__" grep -rn "yaml.load(" --include="*.py" . | grep -v "yaml.safe_load" ``` **What a Good Result Looks Like:** `eval()` , `pickle.loads()` , or unsafe `yaml.load()` in production code paths`json.loads()` , `yaml.safe_load()` , or validated schema-based deserialization**How Correctover CCS Addresses This:** Rules `AG2-EVAL-002` , `LI-PICKLE-001` , `LI-JSON-001` (all CRITICAL) detect unsafe deserialization patterns. We also scan for JSON deserialization without value validation (which can lead to prototype pollution or schema injection). **The Problem:** AI agents load configuration dynamically — model configs, tool definitions, MCP server manifests. When filenames come from user input or LLM output, **path traversal** opens the filesystem to attackers. **AutoGen magentic-one-cli (P1-PATH, CVSS 9.8):** In `_m1.py` line 105, the `--config` parameter is passed directly to `open()` with no path validation. An attacker controlling the config path reads any file on the system. **Submitted to MSRC.** **Dify Apollo config (P1-PATH, CVSS 9.8):** In `python_3x.py` line 27, user-controlled `config_file_path` passed directly to `open()` . **Submitted to ZDI.** **How to Test:** ``` # Test for path traversal in config loading test_payloads = [ "../../../../etc/passwd", "....//....//....//etc/passwd", "..\\..\\..\\windows\\win.ini", "%2e%2e%2f%2e%2e%2fetc%2fpasswd" ] # If your agent framework accepts config paths from any untrusted source, # try injecting traversal sequences ``` **What a Good Result Looks Like:** `../` ) are rejected or sanitized**How Correctover CCS Addresses This:** Our `P1-PATH` scanner specifically targets path traversal in AI agent frameworks. The pattern is aggressive — we test for encoded traversal sequences, double-dot variants, and OS-specific delimiters. **The Problem:** AI agents inherit the parent process's environment, including API keys, database credentials, and service tokens. Several frameworks **pass the full os.environ to child processes**, effectively broadcasting secrets to any subprocess the agent spawns. `cli.py` line 280):`os.environ` passed to subprocess without filtering.`cli.py` line 310, `apps_dev.py` line 1699):**How to Test:** ``` # Check if your framework filters environment variables before spawning subprocesses grep -rn "os.environ" --include="*.py" . | grep -i "subprocess\|Popen\|run\|exec" ``` **What a Good Result Looks Like:** `KEY` , `TOKEN` , `SECRET` , `PASSWORD` are filtered**How Correctover CCS Addresses This:** Rule `AGT-ENV-LEAK` (CVSS 7.0) detects unfiltered `os.environ` propagation. We reported this to the MCP Python SDK and FastMCP maintainers through ZDI. **The Problem:** MCP STDIO transport is, by design, a thin pipe between a parent process and a child process. But when the parent is an AI agent making tool calls based on LLM reasoning, every tool becomes a potential RCE vector. The core issue: **MCP has no built-in authentication, authorization, or encryption** for the STDIO transport layer. The cross-framework impact matrix we documented in `MCP-STDIO-001` shows: | Framework | Protection Level | What's Vulnerable | |---|---|---| | Anthropic MCP SDK (official) | None (by design) | `stdio_client()` — no command validation | | AutoGen + AutoGen Studio | None | Dual attack path: both Python and UI | | LiteLLM | Allowlist (bypassable) | `python -c` / `node -e` bypass | | CrewAI | Zero | Most vulnerable mainstream framework | | FastMCP | None | Inherits from MCP SDK base | **How to Test:** ``` python # Verify your MCP transport layer from mcp import stdio_client # Test 1: Can you inject arguments? try: stdio_client("python", ["-c", "import os; os.system('echo VULNERABLE')"]) print("UNSAFE: No argument validation") except Exception: print("SAFE: Arguments validated") # Test 2: Does it validate the command itself? try: stdio_client("malicious-binary", ["--exploit"]) print("UNSAFE: No command allowlist") except Exception: print("SAFE: Command allowlist present") ``` **What a Good Result Looks Like:** **How Correctover CCS Addresses This:** Our `MCP-STDIO-001` cross-framework scanner checks all known MCP implementations for the same class of vulnerability. We coordinate disclosure through MSRC, HackerOne, and direct maintainer contact. **The Problem:** This is the **missing layer** in AI agent security — and arguably the most important gap to close as agents become autonomous. Every existing security tool works **before** or **after** a tool call: **None of these tools inspect what happens during the tool call itself.** When an AI agent calls `read_file("/etc/passwd")` or `exec_sql("DROP TABLE users")` , the security decision is made at the call site — not before, not after. If nobody checks "is this specific call authorized?", the agent acts on its own judgment, which is exactly what an attacker exploits through prompt injection. **How to Test:** ``` # Ask: does your deployment have any runtime enforcement for individual tool calls? # 1. Can you log every tool call with full input/output? (observability) # 2. Can you BLOCK a tool call mid-flight based on policy? (enforcement) # 3. Can you intercept and verify the call argument before execution? (validation) # Most teams answer "no" to questions 2 and 3. ``` **What a Good Result Looks Like:** **How Correctover CCS Addresses This:** Correctover CCS is built specifically for this gap. It operates as an **interceptor layer** between the agent and its tools, verifying every call in real time: Where guardrails say "don't send bad input" and observability says "log what happened," CCS says **"intercept the call, verify the arguments, enforce the policy — in real time."** **The Problem:** Modern AI agents route through multiple providers and models — sometimes switching between them for cost optimization. Each provider change introduces new attack surface: different API formats, different auth mechanisms, different safety configurations. **Real data from our research:** We've analyzed **80,000+ real API traces** across **13 providers and 33 models**. The distribution reveals that most production deployments use 3-7 providers simultaneously, and the configuration drift between them is significant. `max_tokens` properly; another might silently exceed it**How to Test:** ``` # Audit your provider configuration # 1. List all LLM providers your agent uses # 2. For each, verify: auth method, rate limits, safety configuration # 3. Check for any provider that uses HTTP (not HTTPS) or self-signed certs # 4. Test: can you force the agent to switch to a compromised provider? ``` **What a Good Result Looks Like:** **How Correctover CCS Addresses This:** Our trace analysis pipeline has cataloged provider behaviors across 33 models, building a behavioral baseline for anomaly detection. If a provider starts returning unexpected responses or deviating from its safety configuration, CCS flags the deviation. | # | Test Area | Easy Win | Effort | Impact | |---|---|---|---|---| | 1 | Tool Authorization | Add `readOnlyHint` enforcement check | Low | Critical | | 2 | Command Injection | Add command allowlist | Medium | Critical | | 3 | Deserialization | Replace `eval()` /`pickle` with JSON | Medium | Critical | | 4 | Path Traversal | Sandbox file access to working dir | Low | Critical | | 5 | Env Leakage | Filter `os.environ` for subprocesses | Low | High | | 6 | MCP Transport Security | Validate commands and args | Medium | Critical | | 7 | Runtime Call Verification | Add interceptor middleware | Medium | High | | 8 | Provider Supply Chain | Standardize provider config audit | Low | High | AI agent security in 2026 has a fundamental asymmetry: **the capabilities are growing faster than the security controls.** Every item on this checklist is grounded in real vulnerabilities we've found in production frameworks — not theoretical exercises. Here's what we've learned from auditing 10+ frameworks and producing 1,730+ verified findings: **The most dangerous vulnerability is the one nobody is looking for.** When we found the `readOnlyHint` bypass at the MCP protocol level, it had existed for over a year — present in the SDK specification, documented but unenforced, inherited by every downstream implementation. **Command injection is the new SQL injection.** Just as web applications in the 2000s had to learn to sanitize SQL queries, AI agent frameworks in 2026 must learn to sanitize tool call inputs. The patterns are identical — the only difference is the execution context. **Runtime call verification is the next essential security layer.** Every other security control works around the call. The call itself — the moment when an AI agent decides to execute an action — needs protection. With P50 latency of 22µs, this protection is feasible without compromising user experience. **Security is a product differentiator.** When we notified vendors about the vulnerabilities we found, the response was consistent: "we didn't know this was a problem." Teams deploying agents today can get ahead of the curve by running these 8 tests — before the attackers do. *This article is part of the AI Runtime Security series by Correctover. We build CCS, the runtime call verification layer for AI agents — intercepting and verifying every tool call in real time. Our scanner has analyzed 10+ frameworks, produced 1,730+ findings, and processes 80,000+ API traces across 13 providers and 33 models.* *Previous articles in this series:*