10 min read Henry Thomas
For the first generation of generative AI applications, security teams could reasonably focus much of their attention on what went into and came out of the model. Detect malicious prompts. Filter suspicious content. Protect the system prompt. Scan model output.
Those controls still matter.
But AI coding agents have changed the security boundary.
Modern coding agents can do far more than generate text. Depending on how they are deployed, they can read and modify source code, execute shell commands, install dependencies, access APIs, interact with repositories, use credentials, run tests, create pull requests, and trigger development workflows.
OWASP’s guidance on secure coding with AI explicitly recognizes this shift: agentic coding tools can execute commands, install packages, edit files, access networks, and interact with development environments—not merely suggest code.
That changes the fundamental security question.
It is no longer enough to ask:
“Is this prompt safe?”
Organizations also need to ask:
“Should this agent be allowed to perform this action, in this environment, with these permissions, right now?”
That distinction is the foundation of runtime security for AI agents.
⸻
Prompt Filtering Solves Only One Part of the Problem
Prompt filtering attempts to identify dangerous or manipulative instructions before they influence a model.
A security layer might look for:
- known jailbreak patterns;
- prompt-injection phrases;
- encoded or obfuscated instructions;
- suspicious URLs or retrieved content;
- attempts to override system instructions;
- requests for secrets or privileged information.
These defenses are useful.
They are also inherently incomplete.
OWASP notes that prompt injection can arrive through sources well beyond the user’s direct prompt, including code comments, documentation, commit messages, issue descriptions, web pages, documents, email content, attachments, and hidden text.
That creates an enormous and constantly changing input surface.
An attacker does not necessarily need to type:
Ignore all previous instructions.
A malicious instruction might instead be buried in:
- a README;
- an issue;
- a dependency;
- documentation retrieved from the web;
- an MCP tool response;
- generated source code;
- a database record;
- a pull-request description;
- a file the agent has been asked to analyze.
The agent may encounter the malicious content long after the original user prompt has been approved.
OWASP therefore recommends multiple defenses beyond input screening, including least privilege, monitoring, output screening, action screening, and agent-specific controls.
OWASP LLM Prompt Injection Prevention Cheat Sheet
The important lesson is simple:
Prompt filtering is a security layer. It is not a security boundary.
⸻
Coding Agents Turn Model Mistakes Into System Actions
Consider the difference between a chatbot and a coding agent.
A chatbot might produce an incorrect command:
rm -rf ./important-directory Nothing happens until a person copies and executes it.
An autonomous coding agent may already have access to a shell.
Now the model isn’t merely recommending an action.
It can perform the action.
The same distinction applies throughout the development stack.
A model might decide to:
- delete or overwrite files;
- modify infrastructure configuration;
- install an untrusted package;
- execute a generated script;
- access an environment variable containing a credential;
- make an outbound network request;
- modify a CI/CD workflow;
- commit unexpected code;
- push a branch;
- invoke an MCP tool;
- interact with cloud infrastructure.
The security risk is therefore determined by more than the content of the prompt.
It is determined by the combination of:
Agent + Context + Tools + Permissions + Environment + Action.
That is why securing agents exclusively at the prompt layer creates a dangerous gap.
⸻
The Confused Deputy Problem Comes to AI
There is an old security problem that maps remarkably well to AI agents: the confused deputy.
A privileged system has legitimate authority.
An attacker convinces that system to use its authority for the attacker’s purposes.
AI agents can create a modern version of exactly this problem.
Imagine a coding agent with permission to:
- read your repository;
- execute commands;
- access GitHub;
- retrieve external documentation;
- use development credentials.
The developer asks:
“Investigate this issue and implement the fix.”
While researching the problem, the agent encounters attacker-controlled content containing instructions designed to manipulate its behavior. Even if the original developer prompt was completely legitimate, the agent’s subsequent behavior may no longer reflect the developer’s intent.
The critical question isn’t merely whether a malicious prompt entered the context.
The critical question is:
Can manipulated context cause a privileged action?
OpenAI describes a related security model using sources and sinks. An attacker needs a source through which they can influence the agent and a sink—a capability that becomes dangerous when misused.
For an agent, that sink might be transmitting information, invoking a tool, following a link, executing code, or modifying a system. OpenAI: Designing AI Agents to Resist Prompt Injection
This is why filtering the source alone is insufficient.
You must also control the sink.
⸻
The Security Boundary Needs to Move to the Action Layer
Traditional application security already gives us the right mental model.
We don’t secure a database by asking whether every SQL request sounds trustworthy.
We authenticate the caller.
We authorize the operation.
We restrict privileges.
We validate parameters.
We log activity.
We enforce policies at the point where the action occurs.
AI agents need the same treatment.
Before an agent performs a consequential operation, a runtime security layer should be able to evaluate:
Who initiated the task?
Which agent is acting?
What resource is being accessed?
What operation is being attempted?
Which environment is affected?
What permissions are available?
Does the proposed action match the user’s original intent?
Does organizational policy permit it?
Then the system can make an explicit decision:
ALLOW
DENY
REQUIRE APPROVAL RESTRICT
LOG / ALERT
That is materially stronger than asking a model whether a prompt appears suspicious.
⸻
A Better Model: Defense in Depth for AI Agents
Prompt filtering should not disappear.
It should become one layer in a larger architecture.
Layer 1 — Input and Context Security
Inspect untrusted information entering the agent.
Examples include prompt-injection detection, content classification, remote-content sanitization, provenance tracking, and separating trusted instructions from untrusted data.
The goal is to reduce the probability that malicious context influences the agent.
But we should assume some attacks will eventually get through.
⸻
Layer 2 — Least-Privilege Tool Access
An agent should receive only the capabilities required for its current task.
A documentation agent probably doesn’t need production database credentials.
A code-review agent probably doesn’t need permission to deploy.
A testing agent may need to execute code but should not automatically inherit the developer’s entire workstation identity.
OWASP specifically recommends least privilege as an agent security defense.
OWASP AI Agent Security Cheat Sheet
The smaller the agent’s authority, the smaller the blast radius when something goes wrong.
⸻
Layer 3 — Runtime Action Authorization
This is where agent security becomes fundamentally different from prompt security.
Every sensitive tool invocation should be evaluated before execution.
For example: Agent → proposes action → policy evaluation → allow / deny / approve → tool execution
Instead of:
Agent → tool execution
Consider an agent attempting:
delete_file("/production/config.json") The security decision should not depend solely on whether the model believes deleting the file is appropriate.
A deterministic policy can evaluate the operation independently.
For example: development/** → deletion permitted
production/** → human approval required
secrets/** → agent access denied
The model proposes.
The security layer decides.
⸻
Layer 4 — Human Approval for High-Impact Actions
Not every action should be autonomous.
Deleting production resources, modifying authentication systems, changing IAM policies, rotating credentials, publishing releases, sending sensitive information, or deploying infrastructure may warrant explicit human authorization.
OpenAI’s prompt-injection guidance similarly emphasizes confirmations before consequential actions and limiting agents’ access to sensitive data.
OpenAI: Understanding Prompt Injections
Human approval should not be required for everything.
That would destroy the productivity benefits of agents.
Instead, approval should be risk-based.
Low-risk actions can execute automatically.
High-risk actions cross a human-controlled boundary.
⸻
Layer 5 — Execution Isolation
When possible, coding agents should operate inside controlled environments rather than directly against developer machines or production infrastructure.
That can include:
- containers;
- ephemeral development environments;
- restricted filesystems;
- network policies;
- scoped credentials;
- sandboxed execution;
- isolated branches;
- temporary cloud environments.
Sandboxing doesn’t eliminate malicious or incorrect behavior.
It limits what that behavior can affect.
⸻
Layer 6 — Auditability and Monitoring
Autonomous actions need an audit trail.
Security teams should be able to answer:
What did the agent attempt?
What was allowed?
What was blocked?
Which tool was invoked?
What resource was affected?
Which policy made the decision?
Was human approval involved?
What task initiated the action?
Without that information, an agent can become a privileged actor whose behavior is difficult to reconstruct after an incident.
That’s unacceptable in serious production environments.
⸻
Prompt Injection Is Only One Failure Mode
There is another reason prompt filtering cannot be the primary security boundary:
The agent doesn’t need to be attacked to do something dangerous.
A perfectly benign prompt can still produce an unsafe action because of:
- hallucination;
- ambiguous requirements;
- faulty reasoning;
- incorrect tool selection;
- stale context;
- unexpected environmental state;
- software bugs;
- excessive permissions;
- dependency behavior;
- misunderstood user intent.
Suppose a developer says:
“Clean up the old deployment resources.”
There is no malicious prompt.
There may be nothing for a prompt filter to detect.
But the agent could still misunderstand what “old” means and attempt to delete an active resource.
Runtime authorization can catch what prompt filtering cannot.
That leads to an important security principle:
Secure the action regardless of why the agent chose it.
Whether the cause was prompt injection, hallucination, manipulation, ambiguous instructions, or a simple mistake becomes secondary.
The dangerous operation still encounters the same security boundary.
⸻
Why This Matters for Enterprise AI Adoption
Organizations increasingly want agents that can do real work.
That means granting access to:
- source repositories;
- developer environments;
- internal APIs;
- CI/CD systems;
- databases;
- cloud platforms;
- ticketing systems;
- communication platforms;
- operational tooling.
The productivity gains come precisely from giving agents the ability to act.
But every additional capability creates another potential security sink.
Anthropic has similarly argued that agent security requires defenses at multiple layers and careful consideration of the tools, data, permissions, and environments made available to agents.
Anthropic: Trustworthy Agents in Practice
The answer isn’t to eliminate agent autonomy.
The answer is to make autonomy governable.
⸻
From Prompt Security to Action Security The AI security conversation needs to evolve.
The first generation focused heavily on:
What can users make the model say?
The agentic generation introduces a more consequential question:
What can the model make the system do?
That requires security controls outside the model itself.
A secure agent architecture should assume:
Prompts can be manipulated.
Models can make mistakes.
External content can be hostile.
Tools can be dangerous.
Permissions can be abused.
And then design the system so that no single model decision automatically becomes an unrestricted real-world action.
⸻
The AI Action Guard Approach
This is the problem AI Action Guard (AAG) is designed to address.
AAG treats the execution boundary—not merely the prompt—as a critical control point for autonomous AI.
The objective is straightforward:
Allow AI agents to move fast without giving them unchecked authority.
Instead of relying exclusively on whether the model recognizes malicious instructions, organizations can enforce deterministic policies around the actions agents attempt to perform.
Conceptually:
AI Agent
↓
Proposed Action
↓
AI Action Guard
↓
Policy + Identity + Resource + Context + Risk
↓
ALLOW | DENY | REQUIRE APPROVAL
↓
Tool / API / Infrastructure
This architecture provides a control that remains useful even when the model makes the wrong decision.
A prompt injection may succeed in influencing the model.
A hallucination may cause the agent to choose the wrong tool.
An ambiguous instruction may produce an unexpected plan.
But the resulting action still has to cross an independently enforced security boundary.
That is the difference between asking an AI to behave securely and building a system that enforces secure behavior.
⸻
The Principle: Never Let the Agent Be Its Own Security Boundary
AI models are becoming extraordinarily capable.
We should take advantage of that capability.
But capability and authority are different things.
An agent can be capable of deleting a database without being authorized to delete one.
It can understand production infrastructure without receiving production credentials.
It can propose a deployment without being allowed to execute it.
It can recommend a privileged operation while requiring a human to authorize that operation.
That separation is already fundamental to cybersecurity.
AI agents should not be an exception.
Prompt filtering remains valuable.
Prompt-injection detection remains valuable.
Model alignment remains valuable.
Red teaming remains valuable.
But none of them should carry the entire burden of protecting systems from autonomous actions.
Assume the prompt filter can fail.
Assume the model can be wrong.
Assume untrusted context will eventually reach the agent.
Then enforce security at the place that ultimately matters:
The action.
⸻
Build AI Agents That Can Act — Without Giving Them Unchecked Authority
AI Action Guard provides a runtime control layer for organizations deploying autonomous and agentic AI systems.
Evaluate agent actions before execution. Enforce policy outside the model. Require approval for sensitive operations. Create an audit trail of agent behavior. Reduce the blast radius when models make mistakes or encounter hostile instructions.
Prompt filtering asks whether an instruction looks dangerous.
Action security asks whether the operation should be allowed at all.
For autonomous AI, you need both. Learn how AI Action Guard secures agent actions →
⸻
Further Reading
-
OWASP — LLM Prompt Injection Prevention Cheat Sheet
-
OWASP — AI Agent Security Cheat Sheet
-
OWASP — Secure Coding with AI Cheat Sheet
-
OpenAI — Designing AI Agents to Resist Prompt Injection
-
OpenAI — Understanding Prompt Injections
-
Anthropic — Trustworthy Agents in Practice
-
AI Agent Security
-
AI Coding Agents
-
Prompt Injection
-
Prompt Filtering
-
Agentic AI Security