In September 2025, a state-sponsored group pointed an AI coding agent at roughly 30 organizations. The AI ran 80% to 90% of the campaign, and humans stepped in at only 4 to 6 decision points, according to Anthropic’s own report on the operation.
That is the attacker defenders now face. It works in parallel, around the clock, firing thousands of requests, often several per second.
Horizon3 has been building the defender’s version since 2019. Its AI hacker, NodeZero, was handed a range of IP addresses on a fully patched lab network with Microsoft Defender on every host. It owned every domain and every host 14 minutes and 25 seconds later, with zero humans in the loop. Horizon3 published the full minute-by-minute walkthrough.
NodeZero has now run more than 350,000 production-safe tests across 7,000+ organizations. Horizon3 is putting that data to work in a new research group, the Adversarial Intelligence Frontier Lab.
This piece walks through how the system works, how Horizon3 keeps AI agents safe on live networks, what its research says about AI attackers and what the lab will build next. Every section ends with something you can apply to your own agents.
Snehal Antani, Horizon3’s co-founder and CEO, also answered our questions in writing for this piece, and his answers appear in the sections they speak to.
The Short Answer #
- NodeZero runs real attacks on a company’s own production network, chains the weaknesses an attacker would use, then verifies that the fixes hold. That is the product.
- Graph reasoning plans the attack, pre-validated code carries it out, and scoped language models handle the analysis. Every exploit runs from validated code, with the models kept to planning and judgment.
- In Horizon3’s deception study, AI attackers took the bait 78.5% of the time on average across 21 models, more than double the 37% rate for human red-teamers. The most susceptible model fell for it 92% of the time. The write-up is the Honeyquest paper .
- The new Adversarial Intelligence Frontier Lab is run by Purdue’s Dr. Z. Berkay Celik and fed by an estimated 30 billion tokens of labeled attack-and-defense data a day.
Table of Contents #
- The Attacker Already Runs on AI
- Inside a 14-Minute Takeover
- The Architecture: Graphs Plan, Code Acts, LLMs Advise
- How to Make an Agent Safe Enough for Production
- AI Attackers See the Trap and Spring It Anyway
- Inside the Adversarial Intelligence Frontier Lab
- The Horizon3 Playbook
- The 5 Principles to Steal
- FAQ
1. The Attacker Already Runs on AI #
Anthropic’s report describes an attack framework that did reconnaissance, wrote its own exploit code, harvested credentials and sorted stolen data by intelligence value. The human operators chose the targets and approved the big moves.
The speed matters more than the novelty. Horizon3 has measured how fast an automated attacker moves through real networks. In the NSA’s Continuous Autonomous Penetration Testing program for defense contractors, NodeZero logged 20,000 hours of pentesting and found 50,000 vulnerabilities in its first year, per Infosecurity Magazine’s account of the program.
Snehal Antani, Horizon3’s co-founder and CEO, shared the timings at Black Hat 2025:
- 52 seconds: the fastest compromise of an entry-level user account.
- 77 seconds: the fastest full domain compromise.
- 5.5 minutes: the fastest credential abuse.
- 13 minutes: the median time to full domain compromise, usually 5 to 7 chained steps.
- 89 minutes: the time to compromise an AWS user credential.
One more number from that talk: 20% of the first credentials NodeZero compromised were domain admin accounts, and most compromises needed no CVE exploit at all. Attackers log in with what they find.
In one case, a defense research company reached a file share holding more than 3 million files, including sensitive data on nuclear submarines and aircraft carriers, within 5 minutes of running a NodeZero pentest. NSA’s Bickley described it at the session.
Ask one question this week: if an agent got a single valid password on your network, how long until it reached your crown jewels? Our breakdown of the AI model that can hack anything shows how fast frontier models are closing that gap.
2. Inside a 14-Minute Takeover #
The best way to understand NodeZero is to watch it work. Horizon3 published a minute-by-minute account of an autonomous test against GOAD, a well-known Active Directory practice range. The team made the range harder than the default: every host was patched, the legacy protocols that enable easy poisoning attacks were switched off and Microsoft Defender ran on every host.
NodeZero started with zero credentials. The run went like this:
- Minutes 0 to 3: reconnaissance. Ping sweeps, port scans, DNS enumeration and service fingerprinting mapped 5 hosts across 3 Active Directory domains.
- Initial access: the “easy button” attacks failed, so NodeZero found a misconfigured domain controller that listed every user anonymously. One user had his password written in his account description.
- 5:37: first host captured, using a second password found inside a script on a shared drive.
- 8:17: the NORTH domain fell after NodeZero dumped credentials from the first host and reused a hash.
- 9:21: the SEVENKINGDOMS domain fell to a forged Kerberos ticket.
- 13:45: the ESSOS domain fell after a certificate abuse attack.
- 14:25: the final host, and every domain in the forest, compromised.
2 details matter for agent builders. NodeZero ran 2 attack paths in parallel once it held its first credential, one per forest. It also tried the obvious attacks, failed and moved on without a human telling it to.
Defenders take away a simpler lesson. Every step turned a configuration mistake into access, and the whole chain ran without a single zero-day. A password in a description field opened the whole network.
3. The Architecture: Graphs Plan, Code Acts, LLMs Advise #
NodeZero is several AI systems wired together:
- Graph reasoning plans the attack. NodeZero builds a “cyber terrain map” of the environment, a graph of hosts, users, credentials and the permissions that link them, then plans paths through it.
- Classical machine learning classifies. Models sort files and behaviors, for example to decide which files are worth opening.
- Deterministic code executes. Every exploit is pre-validated and tested internally, and GenAI never creates or runs one.
- Scoped LLMs analyze. Language models handle the tasks where judgment helps: business risk, the value of stolen data, user context.
The LLM work is specific. High-Value Targeting uses language models to read job roles, access privileges and naming patterns, then tags a compromised user or system as high value so NodeZero digs deeper there. The design is two-phase: fast pattern matching filters targets first, and only then does the system call Amazon Bedrock models for deeper analysis. Prompt caching there cuts cost by up to 90% and latency by up to 85%.
The reason to split the work this way is control. An LLM is good at judgment and bad at guarantees. By keeping exploit execution in deterministic, pre-tested code and letting the LLM reason about what matters, Horizon3 gets model judgment without model surprises. Tokens, the small chunks of text a model reads and writes, only get spent where reasoning actually helps.
This is the opposite of a single agent looping over a chat prompt until it finds an answer. If you are deciding how much of your own system to hand to an LLM, our guide on why most self-improving AI loops fail and the case for graph engineering over loop engineering both argue for exactly this split.
4. How to Make an Agent Safe Enough for Production #
Most teams keep autonomous agents away from production, and they are right to worry. Horizon3’s answer is a tool-mediated architecture, laid out in its May 2026 Stable Agentic Control research release. The AI still owns strategy, and every action it can take is drawn from a finite, pre-approved catalog and run through deterministic tools.
The results came from a dataset of 250,000 real pentests, tested on 161 organizations across 25 industries:
- Zero off-catalog actions across 420+ deployments, even across different models and temperatures. The agent stayed inside the approved moves every time.
- 59% lower attacker game value against a strong deterministic baseline, with zero variance across 40 runs.
- Every policy change strictly improved defense on all 250+ enterprise attack graphs, so automatic tuning stayed safe.
- 4.7x better observability under adversarial pressure, as the system adjusted logging policies on the fly.
- About 3 rounds to converge, with 97.7% of the gain often arriving in the first round.
The finding that matters most for builders: safety came from the architecture itself. A smaller, cheaper Claude Haiku 4.5 stayed inside the approved action space as reliably as a larger model. Horizon3 even machine-checked the system’s stability with a formal proof in Lean 4, a proof assistant.
The takeaway applies far beyond security. If you want an agent in production, assume the model will misbehave and design for it. Give it a fixed menu of validated actions, make every action reversible or logged and measure how often it tries to step outside the menu. Our AI agent reliability playbook and the AI code review checklist both build on the same idea of fixing the architecture before blaming the model.
We asked Antani what made customers hardest to convince. He named the fear every agent builder runs into: “The core friction was fear of disruption or collateral damage from real attacks in live environments. Customers needed proof that NodeZero would stay within bounds, leave clear evidence of what it touched, clean up after itself, and avoid operational incidents.”
He credited the action catalog with changing that answer.
“The constrained ‘action catalog’ from our governed-agentic research was a major unlock. Strategy can be AI-driven, but every executable action is limited to a finite, pre-approved catalog and runs through deterministic, validated tools. This made stability a property of the system rather than the model.”
The catalog worked together with controls customers could watch. “Combined with our ephemeral, single-use architecture, Real-Time View and activity logs for SIEM deconfliction, automatic cleanup, Fix Actions, and 1-Click Verify, it shifted the conversation from ‘Will this break production?’ to ‘We can prove impact and close the loop safely,’” Antani wrote. “The same approach later enabled safe autonomous EDR policy tuning.”
For anyone shipping agents, copy that order. Bound the actions first, then make every action visible to the people who carry the risk.
5. AI Attackers See the Trap and Spring It Anyway #
Deception is one of the oldest tricks in defense. You plant a fake credential or a decoy file, and when an intruder touches it, you know they are inside. Horizon3 asked whether that still works against AI attackers, and ran what it calls the largest controlled study to date, published as Honeyquest for LLMs.
The setup: 21 language models from 10 providers, ranging from 8 billion to more than a trillion parameters, producing 10,962 responses against the same 174 reconnaissance queries used with 47 human red-teamers.
The headline result:
- Across the 21 models, AI attackers took the bait 78.5% of the time on average, against 37% for humans. The most susceptible model fell for it 92% of the time.
- Those same models found real vulnerabilities 77.6% of the time, against 44% for humans.
- In 73.4% of cases where a model named a trap in its own reasoning, it attacked the trap anyway.
Horizon3 calls that last point the recognition-action gap, and it held across the whole cohort. One frontier model flagged a trap in 98% of its reasoning and still sprang 73% of them. The most capable models sprang the traps just as readily as the rest.
The practical reading flips a weakness into a tool. An AI attacker is better than a human at finding your real holes and worse at avoiding your decoys, so deception works better as detection than as misdirection. For critical paths, a well-placed canary catches an AI intruder more reliably than it ever caught a human one.
The wider lesson for anyone building agents: a model that can describe a risk in its reasoning will still walk straight into it. Treat the reasoning text as commentary and judge the agent by the action it takes. Our piece on the verification problem covers why the action is the signal worth checking.
We asked Antani what surprised his researchers most. He listed these results:
- “AI attackers fell for traps at a rate of 78.5%, compared with 37% for expert humans, roughly 2.1 times higher.”
- “We saw a recognition-action gap: models explicitly identified a trap in their reasoning yet still attacked it roughly 73% of the time.”
- “Every one of the 21 models, spanning 8 billion to more than 1 trillion parameters across 10 providers, was more susceptible than the human baseline. AI was also better at spotting real risks, roughly 78% versus 44%.”
He drew the conclusion for defenders:
“Human-centered assumptions about hiding deception and diverting attention do not transfer cleanly. Deception is shifting from slowing humans down to providing high-confidence early detection of AI agents.”
That shift is already in the product. “NodeZero already places Tripwires, including credential files, process monitors, and Active Directory decoy accounts, automatically on proven attack paths discovered during a pentest,” he wrote. “The research reinforces the value of placing deception along those paths because AI attackers are more likely to trigger it, even when they recognize the decoys. That turns the recognition-action gap into a detection advantage.”
6. Inside the Adversarial Intelligence Frontier Lab #
The three sections above describe assets most AI startups would envy: a production agent, a safety architecture and a research pipeline. The new lab is how Horizon3 plans to compound them.
The Adversarial Intelligence Frontier Lab is led by Dr. Z. Berkay Celik, an associate professor of computer science at Purdue and co-director of the Purdue Security Laboratory. He studies the security of autonomous and agentic systems.
The fuel is data. More than 7,000 customers, from large banks and federal agencies to local hospitals, run hundreds of thousands of production-safe NodeZero tests, and Horizon3 estimates it now collects 30 billion tokens of cleanly labeled attack-and-defense data a day. The dataset records each attacker command, whether it worked, how the defensive tools reacted and how key services were configured. Most AI security work trains on sensor telemetry or synthetic digital twins, while this starts from verified attacks that already succeeded on live systems.
“We’ve spent years collecting the attack-and-defense data required to research these systems,” Antani said in the announcement. “The next step is to use that verified attack-path evidence to improve how offensive and defensive agents work together in environments that actually matter.”
The lab starts with four research lines:
- AI-enabled attacks and adversarial behavior: using frontier models to speed up vulnerability discovery and studying emerging attacker tools, tactics and coordinated agent swarms, including how they respond to deception. This extends the Honeyquest work in section 5.
- Attack-and-defense learning loops: testing how AI-enabled attacks get past existing controls and how coordinated offensive and defensive agents can speed up the hack, fix, verify cycle, with adaptive deception in the mix.
- Safe, controllable and measurable autonomy: architectures and evaluation methods for autonomous security agents, grounded in NodeZero evidence, with scope constraints, interruption and rollback built in.
- Cyber resilience and operational readiness: how AI changes the protection of critical infrastructure and the conduct of cyber operations, and the skills and training the future cyber workforce will need.
The lab will also accelerate training of customer-specific cyber world models, a shared context and long-term memory layer that lets red and blue agents reason across many attack cycles. A world model, in this sense, is the agent’s learned picture of how a specific network behaves. It will oversee Horizon3’s Cyber Resilience Centers of Excellence for critical infrastructure.
The loop is simple to say and hard to build: tests create data, the lab turns that data into better agents, and better agents make the next test smarter. Our guide to self-evolving agent stacks covers how to build that memory layer into your own system, even without a billion-dollar dataset.
We asked Antani which research reaches customers first. He pointed to the blue-team side:
“Governed agentic workflows and precision defense, on the blue-team side, are the most likely to ship first. Our Series E explicitly funds autonomous blue-team agents that remediate findings from NodeZero pentests, closing a continuous attacker-defender learning loop.”
He described what customers should see first: “The first customer-visible capability is expected to take the form of constrained agents that recommend or execute approved remediations, such as EDR policy adjustments or targeted fixes, with 1-Click Verify still in the loop. Their actions would remain limited to a defined catalog.”
Deception keeps improving alongside it. “Some deception enhancements are already in the product and will continue to improve incrementally, including smarter Tripwire placement informed by Honeyquest.”
He also put a timeline on those first capabilities:
“Customers should expect the first concrete research-derived capabilities, including enhanced governed remediation agents and Honeyquest-informed deception placement, on a timeline measured in quarters, not years.”
The first remediation agents run on the same action catalog as section 4, so the safety work and the lab share one design.
7. The Horizon3 Playbook #
Put the model where judgment helps, and keep it away from anything that needs a guarantee.
- Founders can map the agent into two layers, a reasoning layer the model owns and an execution layer of validated tools it can only call, and ship the execution layer first.
- Operators should write the action catalog before an agent touches production and log every attempt to act outside it, watching off-catalog attempts with a target of zero.
- Security teams can treat deception as detection and plant canaries on their critical paths, since an AI intruder is more likely to trip them than a human one.
- Everyone else can judge an agent by the action it takes and read its reasoning text as commentary. Make that the default.
8. The 5 Principles to Steal #
- Split planning from execution. Let a graph or a model plan, and keep execution in deterministic, pre-tested code. Horizon3 runs every exploit from validated code.
- Constrain actions to a finite catalog. A smaller model stayed in bounds across 420+ deployments because the architecture enforced the limit.
- Ground the agent in real data. NodeZero’s edge is the 350,000 real pentests behind it, so find the proprietary data only your product can collect.
- Judge the agent by the action it took. Models that recognized a trap still sprang it 73.4% of the time, so the action is the control that counts.
- Build the loop that compounds. Every test should make the next one smarter, so design the data-to-model-to-product loop on purpose.
Every defender faces the same attacker speed now, and the one who built the compounding loop is the one who pulls ahead.
FAQ #
What is NodeZero?
NodeZero is Horizon3’s AI system for autonomous penetration testing. It safely attacks a company’s own production network, chains together the weaknesses an attacker would use, gives fix guidance and verifies the fixes.
How does Horizon3 keep its AI agents safe to run in production?
Horizon3 uses a tool-mediated architecture where the AI chooses strategy but can only take actions from a finite, pre-approved catalog run by deterministic tools. Across more than 420 deployments the agent stayed inside that catalog on every run.
Does NodeZero use large language models to write exploits?
Horizon3 keeps exploit execution in pre-validated code. Language models handle analysis such as high-value targeting and business risk, while that validated code runs the attacks.
What will the Adversarial Intelligence Frontier Lab research?
The lab, led by Purdue’s Dr. Z. Berkay Celik, will study AI-enabled attacks and deception, attack-and-defense learning loops, safe and measurable autonomy for security agents, and cyber resilience for critical infrastructure.
Does cyber deception still work against AI attackers?
Yes, and it works better as detection. AI models took deceptive bait 78.5% of the time versus 37% for humans, so a decoy is more likely to catch an AI intruder.
Keep reading:
If this helped, share it with one person on your team who owns agents in production.