{"slug": "before-you-run-the-code-your-ai-agent-wrote-check-these-five-things", "title": "Before you run the code your AI agent wrote, check these five things", "summary": "A developer published a five-point checklist for reviewing AI-agent-generated code before running it, covering hallucinated package names, unreviewed install scripts, agent hooks and MCP configs, hardcoded keys and unsafe eval, and SSRF in agent-written URL features. The same author released two open-source MIT-licensed tools, am-i-hacked and secure-semgrep, which scan projects for malicious code and AI-agent-specific risks and exit non-zero on findings for CI use.", "body_md": "Coding agents are good enough that it's tempting to accept the diff, run `npm install`, and start the dev server. Most of the time that's fine. The problem is the time it isn't, because an agent has your permissions and reads text you never saw.\n\nThese are the five places I look before running anything an agent produced.\n\nAssistants sometimes suggest package names that don't exist, or that are one letter off a real one. Attackers register those names. For every new entry in `package.json` or `requirements.txt`, check that it's the package you meant and that it has a real history.\n\nAdvisory scanners (`npm audit`, OSV-Scanner) are still worth running, but they only know about *reported* problems. A brand-new malicious package has no advisory yet.\n\n`curl ... | sh`, a new `postinstall` script, an editor task. An agent that browses can repeat whatever an untrusted page told it to run. Read every script entry the agent added or changed.\n\nAgent hooks, MCP server definitions, `.claude/settings.json`, `.mcp.json`, `.vscode/tasks.json`, `*.config.js`. All of these start processes, and none of them look like \"code\" in a diff review.\n\nHardcoded API keys. Model output passed to `exec` or `eval`. User input concatenated into a system prompt. No `max_tokens`. Agents write this code readily because it's all over their training data.\n\nIf the agent wrote a link preview, a webhook, or an \"import from URL\" feature, check whether it validates where the URL points. Otherwise someone can aim your server at `169.254.169.254` and read your cloud credentials. That's SSRF.\n\nI maintain two open-source tools for this. Both run with `npx` and need no account.\n\n**[am-i-hacked](https://isaacbell.github.io/secure-devtools/tools/am-i-hacked/)** reads the project for signs of malicious code: auto-run editor tasks, install scripts that download or decode things, obfuscated payloads, executables disguised as assets, capture code paired with an exfiltration endpoint, and the project's own AI-tool config.\n\n```\nnpx am-i-hacked\n```\n\nPut it in front of your dev server so it runs every time:\n\n```\n{ \"scripts\": { \"dev\": \"am-i-hacked && next dev\" }\n```\n\n**[secure-semgrep](https://isaacbell.github.io/secure-devtools/tools/secure-semgrep/)** runs Semgrep with bundled rules for AI-agent code: hardcoded provider keys, model output to exec, user input in system prompts, MCP command injection and tool poisoning, risky agent hooks, prompt injection in `SKILL.md` files. It also adds Semgrep's own security packs for your stack. It needs `semgrep` installed.\n\n```\nnpx secure-semgrep -L ts -L node .\nnpx secure-semgrep -L ssrf .   # opt-in SSRF rules\n```\n\nBoth exit `1` on findings, so they drop into CI.\n\nNeither tool knows what you *asked* the agent to do. A scan finds known patterns; it doesn't prove the code is correct or safe. Neither scans installed `node_modules`. Neither is antivirus. They're a first pass that tells you where to look, and reading the diff is still the check that matters.\n\nThe full guide, including a table of what the AI rules cover, is here: [Is AI-generated code safe to run? How to check it first](https://isaacbell.github.io/secure-devtools/guides/ai-generated-code-security/).\n\nEverything is MIT licensed: **[github.com/IsaacBell/secure-devtools](https://github.com/IsaacBell/secure-devtools)**.", "url": "https://wpnews.pro/news/before-you-run-the-code-your-ai-agent-wrote-check-these-five-things", "canonical_source": "https://dev.to/ikeisahacker/before-you-run-the-code-your-ai-agent-wrote-check-these-five-things-4g8g", "published_at": "2026-10-09 12:43:00+00:00", "updated_at": "2026-10-09 12:51:38.056309+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "developer-tools", "ai-tools", "agent-protocols"], "entities": ["am-i-hacked", "secure-semgrep", "Semgrep", "IsaacBell/secure-devtools", "MCP", "npm audit", "OSV-Scanner", "Claude"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/before-you-run-the-code-your-ai-agent-wrote-check-these-five-things", "markdown": "https://wpnews.pro/news/before-you-run-the-code-your-ai-agent-wrote-check-these-five-things.md", "text": "https://wpnews.pro/news/before-you-run-the-code-your-ai-agent-wrote-check-these-five-things.txt", "jsonld": "https://wpnews.pro/news/before-you-run-the-code-your-ai-agent-wrote-check-these-five-things.jsonld"}}