# Your Next Insider Threat Won’t be Human: Securing AI Agents From the Outside in

> Source: <https://www.rapid7.com/blog/post/ai-inhuman-insider-threats-securing-agents>
> Published: 2026-10-19 16:34:43+00:00

*Why the controls on an AI agent have to live outside of it, and what a security operations team can implement and maintain those controls.*

Security teams spent two decades learning how to work with the people they trust most. Domain admins, DBAs, finance approvers and contractors with VPN access all get a mature playbook: verified identity, scoped access, logged activity and a fast way to revoke it all.

Today, we’re navigating the same kind of access with regard to AI agents. They read our mailboxes, query our data lakes, open tickets, push code and increasingly, act on production systems. Unlike a new hire, an agent can’t be interviewed, background-checked or made to sign an acceptable-use policy.

The industry has largely settled on one framing: AI agents are the new insider threat but that label only tells us half of the story. It tells us *who* to worry about, not *where* the controls should sit. Our argument is that the answer is a security idea from 1972, the reference monitor, and that the SOC is where it gets enforced in practice.

With traditional software, a security incident ends in a root cause. You find the vulnerable function, the misconfigured rule or the bad commit, and you fix it. The behavior lived in code we could read.

Large language models don't give us that. We can't point to the training example or the set of weights that produced a specific decision. The same prompt can produce different actions on different runs. For a defender, that removes the most basic assumption we rely on: that the system will do tomorrow what it did today.

The attack surface changes too, and computer science has an old name for this problem: the **confused deputy**. This is a trusted program tricked into misusing its own authority on someone else's behalf. An agent that reads a poisoned email, web page or document and then acts on hidden instructions is a confused deputy with API keys. No password is stolen and no exploit fires, which makes the agent's permissions the real blast radius, not the model's intelligence.

Most discussion of agents as insiders focuses on two gaps: agents have no career to lose, and they act at machine speed. Both are real. From a detection and forensics seat, though, four other differences change the work more:

| **Difference** | **Human insider** | **AI agent** | **What it means for defenders** | 
|---|---|---|---|
| Monoculture | One person fails on their own | Many identical copies share one weakness | Treat a working injection like a wormable bug: fix the fleet, not one instance | 
| Replayability | Events can be reconstructed and the person interviewed | The same input can produce a different action on a rerun | Forensics must rest on recorded inputs and tool calls, not on replaying the agent | 
| Command channel | Instructions come from a manager | Instructions arrive mixed in with the data it processes | Any content the agent reads is a potential command | 
| Self-report | A person can be questioned and cross-checked | Its account of its reasoning may not match what it did | Evidence must come from outside the agent | 

The last row matters most. We can't make an agent's self-reported reasoning the basis of trust, because we don't yet know how to guarantee that reasoning is faithful. Evidence has to come from outside the agent.

If the model can't be fully trusted, the controls that govern it can't live inside of it. A system prompt that says "never delete production data" is a suggestion, not a control. A well-crafted injection can talk the model out of it.

Security architects solved a version of this problem in the 1970s with the reference monitor: a mechanism that checks every access request, can't be bypassed and can't be tampered with by the program it governs. For AI agents, that means separating three things: the model that reasons, the harness that orchestrates it and the policy layer that decides which actions are actually allowed.

The agent can propose anything, but only the enforcement point decides what reaches real systems, and every decision lands in a log the agent can't alter.

None of these controls are new. What's new is applying them consistently to a non-human actor that reasons, plans and acts.

**Give every agent its own identity.** An agent should never run under a shared service account or a person's own credentials. It needs an identity that can be scoped, monitored, and revoked independently.

**Enforce least privilege on actions, not just data.** Define an explicit action space: which tools, which APIs, which parameters. High-impact actions like payments, deletions or production changes need step-up approval from a human.

**Log everything in a place the agent can't touch.** Every tool call, input, and output goes to tamper-evident storage outside the agent's reach. You should be able to reconstruct what happened without asking the agent.

**Never let a model grade its own work.** Use a different model, a deterministic rule, or a human to verify important outcomes. Avoid depending on a single model vendor for anything critical.

**Red team continuously.** Test prompt injection, tool misuse, data exfiltration, and edge cases, not just the happy path. Re-test every time the model, the prompt, or the tool set changes.

**Build a kill switch before you need it.** In May 2017, WannaCry was stopped by luck. A researcher registered a nonsense domain hard-coded in the ransomware, and new infections stopped spreading because the malware checked that domain before running. Defenders didn't build that kill switch. They found it in the attacker's code, it did nothing for machines already encrypted, and later variants simply removed it. Agent fleets can't depend on that kind of luck. The stop control has to be ours, sit outside the agent, cut off its credentials and tool access mid-run, and be tested regularly, the way an EDR host isolation action is.

**Disclose and share incidents.** When an agent misbehaves, tell the people affected, document which control failed and share the lessons with the wider community, the same way we share threat intelligence today.

**Persistent memory** is a new risk. Agents with long-term memory can be poisoned once and stay compromised. Add memory state to "log everything" and "quarantine its outputs."

Logging alone doesn't protect anyone, as no human team can review millions of agent actions, and handing that review to another unsupervised model just adds another black box. The SOC needs to treat agent activity as a first-class telemetry source, the way it treats endpoints, identities and cloud workloads today.

In practice, that means:

**Behavioral baselines per agent.** Know which tools, data sources and destinations each agent normally uses, and alert on deviations, such as a code-review agent that starts reading the finance file share.

**Detections for agent-specific attacks.** Write rules for prompt-injection indicators, unusual tool chaining, sudden bursts of high-privilege actions and data flowing to unexpected destinations.

**Correlation with the rest of the environment.** Tie agent actions to the user, the session, the content the agent read, and the systems it touched, so an investigation can follow the full chain.

**Response playbooks.** Pause the agent, revoke its tokens, quarantine its outputs and roll back its changes, with the same speed we expect when isolating a compromised host.

The security industry can lead in this area, as we already know how to collect noisy telemetry, find the signal, and contain threats quickly. Agents are the next category of asset to protect, and the next category of actor to watch.

Models will keep getting smarter, and vendors will keep promising they're safer. Both may be true, but neither removes our responsibility for what an agent does inside *our* environment with *our* data and *our* credentials.

The goal isn't to find a model we can trust completely. It's to build a system that stays safe even when the model gets it wrong. Security teams have done this for people, networks, and endpoints for decades. AI agents are simply the next privileged insider, and it's time we treated them that way.
