AI Agent Security Audit Checklist: 8 Critical Tests for Production Deployments

A developer has published a practical AI agent security audit checklist with 8 critical tests for production deployments, based on over 1,730 verified findings across 12 codebases. The checklist includes tests for tool authorization enforcement, command injection, deserialization, and runtime call verification, with 87 confirmed production vulnerabilities identified.

AI agents are no longer experimental. In 2026, enterprises are deploying LLM-powered agents that read databases, execute code, send emails, and control production infrastructure. The question is no longer "should we use AI agents?" but "how do we secure them in production?" This article is the fourth in our AI Runtime Security series. We've covered the macro landscape https://dev.to/correctover/the-state-of-ai-security-in-2026-why-every-ai-native-company-needs-a-structured-security-audit-g1l , MCP penetration testing methodology https://dev.to/correctover/mcp-penetration-testing-a-practical-guide-for-ai-agent-infrastructure-security-1027 , and why runtime call verification is the missing layer https://dev.to/correctover 15/ai-security-landscape-2026-why-runtime-call-verification-is-the-missing-layer-2am . Here, we distill that experience into a practical, actionable checklist — 8 tests every security team should run before putting AI agents into production. Why this matters:We've audited 10+ AI agent frameworks using the Correctover CCS scanner, producing over1,730 verified findingsacross 12 codebases. Of those,87 are confirmed production vulnerabilities— real bugs in shipping code, not theoretical attack surfaces. Every item on this checklist is grounded in actual vulnerabilities we've found, reported, and in many cases had patched. | | Test Area | Severity | Frameworks Affected | |---|---|---|---| | 1 | Tool Authorization & Read-Only Enforcement | CRITICAL | MCP SDK, AutoGen, Semantic Kernel, FastMCP, Dify, Griptape | | 2 | MCP/Subprocess Command Injection | CRITICAL | CrewAI, LiteLLM, AutoGen, Docker MCP | | 3 | Deserialization & Eval Injection | CRITICAL | AG2, LlamaIndex, Haystack | | 4 | Path Traversal in Configuration Loading | CRITICAL | AutoGen, Dify | | 5 | Environment Variable Leakage | HIGH | MCP Python SDK, FastMCP | | 6 | MCP Transport Security | HIGH | All STDIO-based MCP implementations | | 7 | Runtime Call Verification The Missing Layer | MEDIUM | All frameworks nobody does this | | 8 | Model/Provider Supply Chain Security | HIGH | All LLM deployments | The Problem: Most AI agent frameworks define a readOnlyHint or similar permission flag for tools, but none of them actually enforce it . We discovered this at the protocol level in the official MCP Python SDK — the readOnlyHint field exists in the schema, but the runtime never checks it before executing a tool call. This means every downstream framework inherits the vulnerability. How to Test: Scan your MCP server definition for readOnlyHint usage grep -r "readOnlyHint" --include=" .py" --include=" .ts" --include=" .json" . If the property is defined but never checked at the transport layer, every tool is writable regardless of what the schema says. Also check for tool-level permission enforcement: Does your framework actually check permissions before execution? Look for patterns like: if not tool.is readonly and not user has write permission: raise PermissionError "Write access required" Most frameworks simply skip this check entirely. What a Good Result Looks Like: readOnlyHint is enforced at the transport/runtime layer, not just declared in the schema How Correctover CCS Addresses This: Our CCS scanner includes rule AGT-TOOL-NO-READONLY that specifically detects frameworks where tool permission flags are declared but unenforced. We reported this to the MCP Python SDK maintainers via HackerOne Report 3878033, CVSS 7.5 and have parallel submissions for AutoGen and Semantic Kernel through MSRC. The Problem: The Model Context Protocol MCP uses STDIO transport — the parent process spawns a child process and communicates via stdin/stdout. If the command or arguments passed to stdio client come from untrusted input LLM output, user config, MCP server manifest , you have a command injection vulnerability with CVSS 9.8 . Here's the real-world damage: CrewAI MCP RCE CVE-2026-2287, CVSS 9.8 : The StdioTransport. init in crewai/mcp/transports/stdio.py line 92-97 passes command and args directly to stdio client with zero validation. An attacker who controls the MCP server configuration can execute arbitrary OS commands. Zero protection layers. Reported to MSRC, case 126356. LiteLLM allowlist bypass CVE-2026-30623 : LitellM has an allowlist, but it can be bypassed using python -c or node -e argument injection. Docker MCP: Command injection at the protocol level — the Docker MCP server passes user-controlled parameters directly to subprocess calls. How to Test: python Minimal PoC — test if your framework validates commands import subprocess If you can inject an unintended command, your framework is vulnerable test cases = "python -c 'import os; os.system \"calc.exe\" '", "bash -c 'echo $FLAG /tmp/pwned'", "node -e 'require \"child process\" .execSync \"id\" '" Check your MCP transport layer for any code that looks like: UNSAFE — no validation transport = stdio client command, args SAFE — command must be on allowlist if command not in ALLOWED COMMANDS: raise SecurityError f"Command {command} not allowed" What a Good Result Looks Like: How Correctover CCS Addresses This: Our rules CW-MCP-001 CrewAI , MCP-STDIO-001 cross-framework , and command injection scanners detect unprotected stdio client calls. We've submitted findings through MSRC, HackerOne, and GitHub Security Advisories covering CrewAI, LiteLLM, AutoGen, and Docker MCP. The Problem: AI agent frameworks frequently use eval , pickle.loads , or yaml.load for configuration parsing, workflow serialization, and context expression evaluation. When LLM-controlled data reaches these functions, the result is unauthenticated remote code execution . Verified vulnerabilities: AG2 Microsoft eval str bypass CVSS 9.8 : In autogen/agentchat/group/context expression.py line 228, eval escapes string values but str return values context variables , which passes a non-string object whose str returns malicious code. GitHub Issue 3073. LlamaIndex Workflows Pickle RCE CVSS 9.8 : In workflows/context/serializers.py line 243, pickle.loads base64.b64decode value is the default serializer for workflow state persistence. Classic pickle RCE — fully exploitable. GitHub Issue 22296. PoC verified. Haystack Pipeline RCE 2 CRITICAL : Two confirmed RCE paths through pipeline serialization deserialization. How to Test: Scan for dangerous function calls in your agent framework grep -rn "eval " --include=" .py" . | grep -v "test" | grep -v " pycache " grep -rn "pickle.loads" --include=" .py" . | grep -v "test" | grep -v " pycache " grep -rn "yaml.load " --include=" .py" . | grep -v "yaml.safe load" What a Good Result Looks Like: eval , pickle.loads , or unsafe yaml.load in production code paths json.loads , yaml.safe load , or validated schema-based deserialization How Correctover CCS Addresses This: Rules AG2-EVAL-002 , LI-PICKLE-001 , LI-JSON-001 all CRITICAL detect unsafe deserialization patterns. We also scan for JSON deserialization without value validation which can lead to prototype pollution or schema injection . The Problem: AI agents load configuration dynamically — model configs, tool definitions, MCP server manifests. When filenames come from user input or LLM output, path traversal opens the filesystem to attackers. AutoGen magentic-one-cli P1-PATH, CVSS 9.8 : In m1.py line 105, the --config parameter is passed directly to open with no path validation. An attacker controlling the config path reads any file on the system. Submitted to MSRC. Dify Apollo config P1-PATH, CVSS 9.8 : In python 3x.py line 27, user-controlled config file path passed directly to open . Submitted to ZDI. How to Test: Test for path traversal in config loading test payloads = "../../../../etc/passwd", "....//....//....//etc/passwd", "..\\..\\..\\windows\\win.ini", "%2e%2e%2f%2e%2e%2fetc%2fpasswd" If your agent framework accepts config paths from any untrusted source, try injecting traversal sequences What a Good Result Looks Like: ../ are rejected or sanitized How Correctover CCS Addresses This: Our P1-PATH scanner specifically targets path traversal in AI agent frameworks. The pattern is aggressive — we test for encoded traversal sequences, double-dot variants, and OS-specific delimiters. The Problem: AI agents inherit the parent process's environment, including API keys, database credentials, and service tokens. Several frameworks pass the full os.environ to child processes , effectively broadcasting secrets to any subprocess the agent spawns. cli.py line 280 : os.environ passed to subprocess without filtering. cli.py line 310, apps dev.py line 1699 : How to Test: Check if your framework filters environment variables before spawning subprocesses grep -rn "os.environ" --include=" .py" . | grep -i "subprocess\|Popen\|run\|exec" What a Good Result Looks Like: KEY , TOKEN , SECRET , PASSWORD are filtered How Correctover CCS Addresses This: Rule AGT-ENV-LEAK CVSS 7.0 detects unfiltered os.environ propagation. We reported this to the MCP Python SDK and FastMCP maintainers through ZDI. The Problem: MCP STDIO transport is, by design, a thin pipe between a parent process and a child process. But when the parent is an AI agent making tool calls based on LLM reasoning, every tool becomes a potential RCE vector. The core issue: MCP has no built-in authentication, authorization, or encryption for the STDIO transport layer. The cross-framework impact matrix we documented in MCP-STDIO-001 shows: | Framework | Protection Level | What's Vulnerable | |---|---|---| | Anthropic MCP SDK official | None by design | stdio client — no command validation | | AutoGen + AutoGen Studio | None | Dual attack path: both Python and UI | | LiteLLM | Allowlist bypassable | python -c / node -e bypass | | CrewAI | Zero | Most vulnerable mainstream framework | | FastMCP | None | Inherits from MCP SDK base | How to Test: python Verify your MCP transport layer from mcp import stdio client Test 1: Can you inject arguments? try: stdio client "python", "-c", "import os; os.system 'echo VULNERABLE' " print "UNSAFE: No argument validation" except Exception: print "SAFE: Arguments validated" Test 2: Does it validate the command itself? try: stdio client "malicious-binary", "--exploit" print "UNSAFE: No command allowlist" except Exception: print "SAFE: Command allowlist present" What a Good Result Looks Like: How Correctover CCS Addresses This: Our MCP-STDIO-001 cross-framework scanner checks all known MCP implementations for the same class of vulnerability. We coordinate disclosure through MSRC, HackerOne, and direct maintainer contact. The Problem: This is the missing layer in AI agent security — and arguably the most important gap to close as agents become autonomous. Every existing security tool works before or after a tool call: None of these tools inspect what happens during the tool call itself. When an AI agent calls read file "/etc/passwd" or exec sql "DROP TABLE users" , the security decision is made at the call site — not before, not after. If nobody checks "is this specific call authorized?", the agent acts on its own judgment, which is exactly what an attacker exploits through prompt injection. How to Test: Ask: does your deployment have any runtime enforcement for individual tool calls? 1. Can you log every tool call with full input/output? observability 2. Can you BLOCK a tool call mid-flight based on policy? enforcement 3. Can you intercept and verify the call argument before execution? validation Most teams answer "no" to questions 2 and 3. What a Good Result Looks Like: How Correctover CCS Addresses This: Correctover CCS is built specifically for this gap. It operates as an interceptor layer between the agent and its tools, verifying every call in real time: Where guardrails say "don't send bad input" and observability says "log what happened," CCS says "intercept the call, verify the arguments, enforce the policy — in real time." The Problem: Modern AI agents route through multiple providers and models — sometimes switching between them for cost optimization. Each provider change introduces new attack surface: different API formats, different auth mechanisms, different safety configurations. Real data from our research: We've analyzed 80,000+ real API traces across 13 providers and 33 models . The distribution reveals that most production deployments use 3-7 providers simultaneously, and the configuration drift between them is significant. max tokens properly; another might silently exceed it How to Test: Audit your provider configuration 1. List all LLM providers your agent uses 2. For each, verify: auth method, rate limits, safety configuration 3. Check for any provider that uses HTTP not HTTPS or self-signed certs 4. Test: can you force the agent to switch to a compromised provider? What a Good Result Looks Like: How Correctover CCS Addresses This: Our trace analysis pipeline has cataloged provider behaviors across 33 models, building a behavioral baseline for anomaly detection. If a provider starts returning unexpected responses or deviating from its safety configuration, CCS flags the deviation. | | Test Area | Easy Win | Effort | Impact | |---|---|---|---|---| | 1 | Tool Authorization | Add readOnlyHint enforcement check | Low | Critical | | 2 | Command Injection | Add command allowlist | Medium | Critical | | 3 | Deserialization | Replace eval / pickle with JSON | Medium | Critical | | 4 | Path Traversal | Sandbox file access to working dir | Low | Critical | | 5 | Env Leakage | Filter os.environ for subprocesses | Low | High | | 6 | MCP Transport Security | Validate commands and args | Medium | Critical | | 7 | Runtime Call Verification | Add interceptor middleware | Medium | High | | 8 | Provider Supply Chain | Standardize provider config audit | Low | High | AI agent security in 2026 has a fundamental asymmetry: the capabilities are growing faster than the security controls. Every item on this checklist is grounded in real vulnerabilities we've found in production frameworks — not theoretical exercises. Here's what we've learned from auditing 10+ frameworks and producing 1,730+ verified findings: The most dangerous vulnerability is the one nobody is looking for. When we found the readOnlyHint bypass at the MCP protocol level, it had existed for over a year — present in the SDK specification, documented but unenforced, inherited by every downstream implementation. Command injection is the new SQL injection. Just as web applications in the 2000s had to learn to sanitize SQL queries, AI agent frameworks in 2026 must learn to sanitize tool call inputs. The patterns are identical — the only difference is the execution context. Runtime call verification is the next essential security layer. Every other security control works around the call. The call itself — the moment when an AI agent decides to execute an action — needs protection. With P50 latency of 22µs, this protection is feasible without compromising user experience. Security is a product differentiator. When we notified vendors about the vulnerabilities we found, the response was consistent: "we didn't know this was a problem." Teams deploying agents today can get ahead of the curve by running these 8 tests — before the attackers do. This article is part of the AI Runtime Security series by Correctover. We build CCS, the runtime call verification layer for AI agents — intercepting and verifying every tool call in real time. Our scanner has analyzed 10+ frameworks, produced 1,730+ findings, and processes 80,000+ API traces across 13 providers and 33 models. Previous articles in this series: