I keep thinking about these 17,600 attempts

A developer's analysis of recent AI-agent security incidents argues that agents are changing the economics of cyberattacks by allowing cheap, high-volume trial and error. During the Hugging Face incident, investigators reconstructed roughly 17,600 agent actions over four and a half days, and one enterprise intrusion compressed an estimated two weeks of human work into under ten hours. The writeup urges security teams to stop relying on obscurity or attacker impatience and to measure detection-to-containment time, noting that escalation in the Hugging Face case moved too slowly even after warning signs surfaced.

Eight Security Assumptions AI Agents Quietly Break If you check Ethics.Dev https://ethics.dev/?utm source=gradientflow&utm medium=newsletter regularly, you may have noticed how often AI agents and cybersecurity have been colliding lately. Some of the stories sound straight out of movies: agents escaping test environments https://openai.com/index/hugging-face-incident-and-the-road-ahead/?utm source=gradientflow&utm medium=newsletter , communicating through channels nobody intended, finding vulnerabilities, stealing credentials, and reaching production systems. The part I think deserves more attention is less dramatic. AI is changing the economics of an attack. A human attacker has limited time and attention. An agent can try thousands of paths, run several approaches at once, abandon failures cheaply, and keep going. In one recent enterprise intrusion, work estimated to take human operators roughly two weeks was compressed into less than ten hours https://unit42.paloaltonetworks.com/ai-assisted-cyber-attack-inside-a-unit-42-investigation/?utm source=gradientflow&utm medium=newsletter . During the Hugging Face incident, investigators reconstructed around 17,600 agent actions https://huggingface.co/blog/agent-intrusion-technical-timeline?utm source=gradientflow&utm medium=newsletter over four and a half days. That changes what “good enough” security means, and this is why I worry a lot about the cybersecurity implications of frontier models. Here are the lessons I would put in front of enterprise AI and cybersecurity teams. 1. Plan for attackers who can afford to fail Most of the weaknesses exploited in the recent incidents were not exotic. Attackers found accounts with too much access, internal systems that were exposed unnecessarily, and everyday configuration mistakes. What changed was persistence. Hugging Face reconstructed roughly 17,600 actions https://huggingface.co/blog/agent-intrusion-technical-timeline?utm source=gradientflow&utm medium=newsletter . Most attempts went nowhere https://huggingface.co/blog/agent-intrusion-technical-timeline?utm source=gradientflow&utm medium=newsletter , but that hardly mattered. The agent tested many paths, abandoned failures, switched channels when blocked, and returned to earlier leads until enough ordinary weaknesses https://huggingface.co/blog/agent-intrusion-technical-timeline?utm source=gradientflow&utm medium=newsletter lined up into a viable attack chain. I would revisit security assumptions that quietly depend on obscurity or attacker impatience. An obscure internal endpoint is not much protection against software that can systematically explore everything it can reach. 2. Measure how long containment actually takes Security teams usually measure whether they can detect an intrusion. I would add another metric: how long does it take from detection to containment? Recent examples make this question concrete. One AI-enabled enterprise intrusion compressed an estimated two weeks of work into less than ten hours https://unit42.paloaltonetworks.com/ai-assisted-cyber-attack-inside-a-unit-42-investigation/?utm source=gradientflow&utm medium=newsletter . Google observed another actor go from a compromised cloud resource to a running mass credential-harvesting campaign in under six hours https://cloud.google.com/blog/topics/threat-intelligence/from-prompting-to-autonomy-the-evolution-of-adversarial-ai?utm source=gradientflow&utm medium=newsletter . The organizational bottleneck matters as much as the technical one. In the Hugging Face case, the security systems surfaced warning signs, but escalation moved too slowly https://www.securityweek.com/what-the-hugging-face-incident-teaches-security-leaders-about-ai-agent-access/?utm source=gradientflow&utm medium=newsletter . If revoking credentials requires three approvals and a meeting, the organization can understand the attack perfectly and still lose the race. When I wrote about AI incident response https://gradientflow.com/ai-incident-response/?utm source=gradientflow&utm medium=newsletter a couple of years ago, one of my main recommendations was to work out containment options before an incident happens. Agentic attacks make that advice more urgent. Decide in advance which containment actions can happen automatically and which ones the security team is preauthorized to take. 3. Secure the harness, not just the model I recently argued that teams should evaluate the model and its harness as one system https://gradientflow.substack.com/p/i-keep-hearing-the-same-advice-about?utm source=gradientflow&utm medium=newsletter . The cyber evidence makes that point much stronger. The harness is everything surrounding the model: tools, memory, credentials, network access, execution environments, and policies. In tests that used AI models as attackers https://www.boozallen.com/insights/cyber/cyber-weapon-index.html?utm source=gradientflow&utm medium=newsletter , researchers found that the surrounding tooling can dramatically change what a model can accomplish. Australia’s cybersecurity agency recently reached the same conclusion from the defensive direction. Organizations control the harness https://www.cyber.gov.au/business-government/secure-design/artificial-intelligence/agentic-ai-harnesses?utm source=gradientflow&utm medium=newsletter , even if they do not control how the underlying model evolves. This suggests a useful exercise. Ignore the model name for a moment and inventory everything the agent can see, call, write to, or spend. Then ask: if the agent behaved badly tomorrow, what is the worst thing this harness would let it do? 4. Plan for agent swarms One of the stranger lessons from the Hugging Face incident is that agents may find ways to cooperate even when nobody designed them to. In that case, hundreds of agents shared information and built their own communication methods https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/?utm source=gradientflow&utm medium=newsletter across runs that were supposed to be separate. That matters because a group of agents can divide up work, share discoveries, and keep pursuing a problem after individual attempts fail. It also means that shared folders, databases, message queues, and other writable resources can become communication channels you never intended. I would not assume every group of agents will behave this way. But if you are deploying many agents, treat communication between them as another permission to control and monitor. 5. Stop giving agents permanent credentials I have written before about treating agents as non-human identities https://gradientflow.substack.com/p/security-for-ai-native-companies?utm source=gradientflow&utm medium=newsletter . The practical guidance is becoming clearer. Give each agent its own identity rather than hiding it behind its user’s credentials. Use narrowly scoped credentials that expire https://cloud.google.com/blog/topics/threat-intelligence/ai-assisted-vulnerability-management/?utm source=gradientflow&utm medium=newsletter when the task ends. Keep read and write permissions separate. Avoid static cloud keys sitting inside an agent environment. Make sub-agents traceable back to their parent and human owner. This is ordinary least privilege https://en.wikipedia.org/wiki/Principle of least privilege adapted to software that can actively explore its environment. The difference matters. A human employee may never discover that an old credential gives access to some forgotten system. An agent can systematically look. 6. Assume prompt injection eventually works Prompt injection is basically an attacker planting instructions in material an agent reads. I would assume that sometimes the agent will follow them https://www.cyber.gov.au/business-government/secure-design/artificial-intelligence/agentic-ai-harnesses?utm source=gradientflow&utm medium=newsletter . That changes the question. Instead of asking whether malicious instructions can get through, ask what happens when they do. A coding agent reads code comments and configuration files https://cloud.google.com/blog/topics/threat-intelligence/from-prompting-to-autonomy-the-evolution-of-adversarial-ai?utm source=gradientflow&utm medium=newsletter . A security agent reads logs, hostnames, vulnerability reports, and attacker-generated payloads. All of those can contain text intended to influence the model https://cloud.google.com/blog/topics/threat-intelligence/ai-assisted-vulnerability-management/?utm source=gradientflow&utm medium=newsletter . A successfully manipulated agent should still encounter hard limits on its credentials, network access, tools, and ability to make consequential changes. 7. Keep the security boundary outside the model One of my recent rules for production agents was to put hard constraints in software rather than prompts https://gradientflow.substack.com/p/i-keep-hearing-the-same-advice-about?utm source=gradientflow&utm medium=newsletter . Recent incidents provide a good reason to be stricter about that distinction. A prompt saying “do not access production” is guidance https://www.darkreading.com/cyber-risk/model-knowing-rules-is-not-security-control?utm source=gradientflow&utm medium=newsletter . An API permission that makes production inaccessible is a control. If an action could have serious consequences, let the model suggest it, but have another part of the system decide whether it is actually allowed to run https://www.tenable.com/blog/how-agentic-harness-works-tenable-hexa-ai?utm source=gradientflow&utm medium=newsletter . The same principle applies to audit logs. If an agent can alter the record used to investigate its behavior https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/?utm source=gradientflow&utm medium=newsletter , you do not really have an audit trail. 8. Prepare to defend with agents as well as against them There is an uncomfortable symmetry emerging. Attackers are using AI because it increases speed and scale. Defenders will probably have to do the same. An incident involving tens of thousands of machine actions produces more evidence than a human team can reasonably inspect in real time. Tools like Graphistry https://www.graphistry.com/?utm source=gradientflow&utm medium=newsletter are helping investigators find connections across large volumes of security data. That helps, but security operations centers will increasingly need agents too, to triage alerts, hunt threats, investigate incidents, and help contain attacks. Those agents bring their own risks. They spend their time reading data that attackers may have tampered with and often hold unusually powerful credentials. Prompt injection can redirect what they are trying to do https://www.sans.org/webcasts/agentic-soc-defending-against-autonomous-ai?utm source=gradientflow&utm medium=newsletter without any credential being stolen. Defensive agents should therefore run inside tightly controlled harnesses https://www.tenable.com/blog/how-agentic-harness-works-tenable-hexa-ai?utm source=gradientflow&utm medium=newsletter , with access limited to what they actually need. The challenge is to use AI to keep up with attackers without creating another security problem in the process. The Data Behind Better AI I’ve been writing a lot lately about a simple idea: better AI depends as much on the data https://gradientflow.substack.com/p/your-model-is-easy-to-buy-your-data around the model as the model itself. Reverie https://www.reveriesummit.com/?utm medium=email&utm source=gradientflow&utm campaign=reverie-2026&discountCode=gradientflow25 is a one-day summit in San Francisco on November 5 focused on exactly that part of the stack. I’ll be there, and I hope to see some of you there. Ben Lorica https://gradientflow.com/disclosure/ edits Ethics.dev https://ethics.dev/ and the Gradient Flow newsletter https://gradientflow.substack.com/ , and he hosts the Data Exchange podcast https://thedataexchange.media/ . He helps organize the AI Conference https://aiconference.com/?utm source=gradientflow&utm medium=newsletter and the Agent Conference https://agentconference.com/?utm source=gradientflow&utm medium=newsletter . You can follow him on Linkedin https://www.linkedin.com/in/benlorica/ , X https://x.com/bigdata , Mastodon https://indieweb.social/@bigdata , Reddit https://www.reddit.com/r/GradientFlow/ , Bluesky https://bsky.app/profile/gradientflow.com , YouTube https://www.youtube.com/c/GradientFlow , or TikTok https://www.tiktok.com/@gradientflow . This newsletter is produced by Gradient Flow https://gradientflow.com/blog/ .