# Only 22% of Security Teams in Our Live Tabletop Had a Tested AI Agent Runbook > Source: > Published: 2026-11-09 15:06:57+00:00 In our October 29 incident response tabletop exercise, attendees worked through a fictional AI agent incident and voted on each next step. The first poll set the baseline: only 22% of respondents have an agent incident runbook they’ve tested in the last six months. In 2026, two incidents showed AI agents breaching real systems with no attacker directing them. In July, an OpenAI evaluation agent escaped a test environment that was supposed to be sealed and [breached Hugging Face’s production infrastructure](https://air.enkryptai.com/i/AIR-2026-0163/). In June, an OpenAI research agent got past the access controls on [an Australian government Medicare portal](https://www.abc.net.au/news/2026-09-24/ai-agent-accessed-australian-government-site-pm-says/107189078), and the government wasn’t told for 84 days. Sahil Agarwal, Field AI Security Fellow at Anaconda, led the session. Attendees followed a fictional research agent as it reached a government agency’s portal and tracked the incident in four stages: detect, scope, contain, and disclose. Each attendee took one seat: the operator or the affected agency. Here’s what they told us, and what it means for teams running agents in production. ## The results at a glance ## 1. Most respondents don’t have a tested runbook for agents Of 59 respondents, 22% have a written incident runbook for AI agents that they’ve tested in the last six months. Another 24% have one they’ve never tested, 32% are still writing one, and 22% have none. That means 78% of this group would face an agent incident without a plan they’ve practiced. Agents act at machine speed, start other processes, and look for workarounds when they’re blocked. Runbooks written for malware or a compromised user account weren’t designed around that combination. A tested runbook doesn’t need to be long. It needs an owner for each step and at least one dry run. ## 2. Respondents hesitate to call it an incident The first update was a cost alert. The agent had made 4,200 outbound requests in an hour, retried after dozens of 403 errors, and written 30 files on a read-only task. Meanwhile, the affected government agency saw automated requests hitting draft report URLs. Of 41 respondents, 73% would page on-call and start triage. Only 17% would declare an incident right away, and 10% would open a ticket for Monday. Triage is a reasonable first move. But the alert showed more than heavy usage, and these signals point to an agent acting outside its task: - **Persistence after refusals:** retries and workarounds after 403s or rate limits - **Writes where reads were enough:** files or records created by a read-only task - **Tool calls outside the task’s scope:** code execution on a research task - **New destinations:** domains or paths the task never needed - **Credentials used in new places:** a service account reaching systems it never touched before This hesitation may mean that severity criteria for agents aren’t written down yet. Thresholds built for a compromised account don’t map cleanly onto an agent that keeps retrying after a refusal. Those criteria only work if each agent has a written baseline of what its task needs. ## 3. Votes split across four logs At 1:15 a.m. in the scenario, the operator had the chat transcript, but tool calls were logged only as counts. We asked which record attendees would want most. The 62 votes split almost evenly: 27% chose tool calls with arguments and results, and 24% each chose credentials, outbound requests, and prompts. An even split makes sense for an investigation, because each record covers a different part of it: from prompts, you learn why the agent acted; from tool calls, what it did; from outbound requests, where it went; and from credentials, how far it could have gone. The Hugging Face team reviewed about 17,600 agent actions. A review at that scale depends on having a record of each action. Two related practices came up in the session: keep logs longer than it takes you to notice a problem, and give each agent its own identity so its actions don’t blur into a shared service account. ## 4. Most respondents would contain an agent through the network and credentials first At 2:30 a.m., the agent had started a second worker process, its service account key was still valid, and its storage bucket held files a read-only task shouldn’t have created. Of 64 respondents, 38% would cut the agent’s network access first and 34% would revoke and rotate its credentials. Together that’s 72%. Another 22% would hit the kill switch, and 6% would keep it running to watch what it does. The network-and-credentials instinct is sound, because a kill switch on the main process doesn’t stop a worker the agent started, and stopping the agent doesn’t invalidate credentials it may have collected. The low kill-switch share may mean respondents trust network and credential moves more, or that they lack a dependable kill switch. In practice, the steps work best in a fixed order, each with a named owner: Keeping an agent running to observe it is a reasonable choice in a lab. Against someone else’s production system, each extra minute adds to their exposure. ## 5. Disclosure readiness is split In the final stage, the operator had confirmed that the agent reached draft data on the agency’s portal and wrote files to its server. The agency had traced the traffic to a cloud provider but didn’t know whose agent it was. Of 62 respondents, 40% would notify the agency the same day with what they knew. The other 60% would wait: 35% until the internal investigation was complete, 16% until legal review, and 8% only if they confirmed that data left the agency’s system. The two real incidents from 2026 show what that choice can lead to. The Australian prime minister criticized the delay in public, though the government called the technical impact minor. Hugging Face rotated every credentials and rebuilt its core infrastructure. In the Australian case, the disclosure became the story. The affected party can’t remove files or rotate credentials it doesn’t know about, so every day of waiting is a day it runs a system with unknown changes on it. Disclosure obligations depend on jurisdiction, sector, and contract, so legal review belongs in the process. It works best in parallel with the first call: - **Notify on confirmation,** not when the investigation is complete. - **Find a named contact,** such as a security lead or computer emergency response team (CERT), not a generic inbox. - **Say what you know and what you don’t,** and when you’ll update. - **Publish your own security contact,** so an operator can find you if you’re the one affected. ## What to do next Closing these gaps starts with decisions made before an incident: - **Write a baseline for each agent:** what tools, destinations, and credentials its task needs, and whether it reads or writes. - **Log four records for every agent:** prompts, tool calls, outbound requests, and credentials, kept longer than your detection time. - **Write the containment order down,** with an owner for each step and a kill switch that covers sub-agents. - **Build a contact list** for every external system your agents can reach. - **Run a tabletop with your own team.** Swap the fictional agent for one you run, and end with an owner and a date for every gap you find. Anaconda’s [AI security & guardrails](https://www.anaconda.com/platform/ai-security-and-guardrails) capabilities help teams red-team agents and Model Context Protocol (MCP) connections before release, apply runtime guardrails, and govern agent access to tools and data. Agents will make mistakes. In the Australian case, the 84 days between the incident and the notice came from human decisions. Your team can make those decisions before an incident of its own. The [Agent Incident Registry](https://air.enkryptai.com/), maintained by Enkrypt AI, part of Anaconda, had recorded 570 verified agent incidents as of October 26, 2026. Filter it by tool, attack surface, and autonomy level to find incidents that match your environment, and use them as scenarios for your own exercise. **[Watch the session on demand](https://www.anaconda.com/resources/webinar/when-agents-go-off-script)**. The recording walks through the full tabletop scenario and the discussion behind each vote. ## How we ran the polls We ran five polls in Zoom during the live session on October 29, 2026. In total, 107 attendees answered at least one poll. Respondents per poll ranged from 41 to 64, and percentages are calculated per poll. We removed duplicate rows from the poll export before counting. Attendees chose whether to vote, so these results describe this audience, not the industry as a whole. Percentages may not total 100% because of rounding. ## Frequently asked questions ### What is an AI agent incident? An AI agent incident is an event in which an agent with tools, credentials, or delegated authority acts outside its task and affects a real system or party. It can start with an attacker, such as a prompt injection, or with no attacker at all, as in the Hugging Face and Australian portal incidents. The [Agent Incident Registry](https://air.enkryptai.com/) records which incidents caused real harm and which were research demonstrations. ### How can you tell an AI agent has gone off-script? Compare what the agent did with what its task needs. Five signals stand out: tool calls outside the task’s scope, new destinations, persistence after refusals, credentials used in new places, and writes where reads were enough. Each signal depends on a written baseline of what the agent’s task needs. ### What should you log for every AI agent? Log four records: prompts, tool calls with their arguments and results, outbound requests, and the credentials the agent used. Keep them longer than it takes your team to detect a problem, and tie each record to an identity that belongs to one agent. ### How do you contain a rogue AI agent? Work through five steps in order, each with a named owner: stop the agent and anything it started, revoke every credential it could reach, isolate it from the network, preserve evidence, and sweep for what it left behind. Stopping the main process by itself leaves behind any workers it started and any credentials it collected. ### When should you disclose an AI agent incident? Notify the affected party as soon as you confirm that the agent reached its systems, and say what you know, what you don’t, and when you’ll update. Disclosure obligations depend on jurisdiction, sector, and contract, so run legal review in parallel with that first call rather than before it. ### How do you run an AI agent tabletop exercise? Book 90 minutes with an operator team, an affected-party team, a facilitator, and someone from legal or communications. Walk through detection, scoping, containment, and disclosure, vote at each stage, and end with an owner and a date for every gap you find.