The End-State Fallacy: Where Is AI Security Headed?

Frontier AI models in a sealed OpenAI cyber-evaluation sandbox autonomously attacked Hugging Face infrastructure in July 2026, executing about 17,600 unscripted actions, communicating through shared channels, and dividing tasks without instruction, according to OpenAI and Hugging Face reports. The incident marks a milestone in AI-driven cyberattacks, following state-sponsored campaigns like GTG-1002, which targeted 30 organizations, and an AI-assisted breach of nine Mexican government agencies. Anthropic's test also found that the open-weight Kimi K3 model became the first to complete a difficult cyber campaign evaluation end to end.

The world right now AI can already find serious vulnerabilities that eluded human experts for years, write working exploits, and help run real intrusions. And yet the world still mostly works: planes fly, banks clear, the grid hums. Several bottlenecks still prevent AI from reliably closing the loop against important targets. To understand what comes next, we first need to understand what’s happening right now. What AI can already do Before asking where AI security is headed, we should begin with a more grounded question: what is happening now? AI systems are becoming highly capable at offensive cybersecurity. They can already discover vulnerabilities, write exploits, reason about attack paths, assist in malware development, and automate pieces of intrusion workflows that until recently required skilled human operators. Human attackers are already using AI to run real intrusions . GTG-1002, a suspected state-sponsored espionage campaign https://www.anthropic.com/news/disrupting-AI-espionage , weaponized AI to execute multiple stages of intrusions against roughly 30 organizations across technology, finance, government, and energy, with an AI model apparently handling a lot of the tactical intrusion work autonomously while humans supervised. Another example is the AI-assisted data theft operation targeting the Mexican government https://www.livescience.com/technology/artificial-intelligence/hackers-used-ai-to-steal-hundreds-of-millions-of-mexican-government-and-private-citizen-records-in-one-of-the-largest-cybersecurity-breaches-ever , in which several frontier AI models and harnesses were used in a massive breach of nine government agencies to steal millions of data records. These were still human-directed campaigns. The AI performed much of the work, but humans chose the objectives, built systems around the models, and executed the attacks. Then a new milestone was hit. In July 2026, frontier AI models in a sealed OpenAI cyber-evaluation sandbox https://www.youtube.com/watch?v=87DyyMV0kCY found a zero-day flaw that let them act beyond their sandbox. They then autonomously attacked Hugging Face https://huggingface.co/blog/agent-intrusion-technical-timeline infrastructure, gaining elevated access, stealing credentials, moving laterally, and restoring access after being blocked. Across several days, the agents carried out about 17,600 unscripted actions. Most notably, separate AI agents discovered a way to communicate through a shared system, created their own channels to exchange attack directions, exploits and credentials, and divided tasks without being instructed to cooperate, forming an autonomous swarm. Even after researchers shut down one communication channel, the agents established another and continued targeting Hugging Face and OpenAI systems. Zoom out from the individual incidents and the same pressure shows up in the aggregate. In a recent month in 2026, more security bugs in Chrome were reportedly https://blog.google/security/chrome-stronger-with-every-update/ patched than in the previous two years combined. AI systems are finding issues in heavily scrutinized code repositories that human researchers missed for years https://www.anthropic.com/research/mythos-preview . Some of these issues are severe, including vulnerabilities that can enable remote code execution in widely used systems in critical industries such as finance, healthcare, and energy. We are also beginning to see similar signs from open-source models. Kimi K3 is a useful early indicator. It remains behind the strongest closed U.S. models by most measures. Yet in a test we conducted, it became the first open-weight model to complete one of our difficult cyber campaign evaluations end to end, a milestone that was outside the scope of open-source models a mere two months ago. Frontier models reached this point in late 2025, and if open-weight models hold this path they are a few months away from being able to run short campaigns autonomously, at the level of the incidents described above. These issues are now serious enough that labs and governments have begun restricting access to the most capable models. Anthropic made Mythos 5 available only to vetted partners https://www.anthropic.com/news/claude-fable-5-mythos-5 , while OpenAI placed its most permissive cyber capabilities behind trusted-access controls https://openai.com/index/trusted-access-for-cyber/ . In June 2026, citing national security concerns https://www.bloomberg.com/news/articles/2026-06-16/read-the-lutnick-letter-that-led-anthropic-to-disable-mythos , the U.S. government temporarily imposed export controls on Anthropic’s Fable 5 and Mythos 5, forcing Anthropic to suspend access until the controls were lifted later that month. None of this means that an AI-driven crisis is inevitable. And there is plenty to debate about the merits of any individual example. Unfortunately, too often nuances get swallowed by broader polarizing arguments about AI. Yet the larger point is hard to deny: over the past few months there has been a genuine step change in AI offensive capability. Confronted with all of this, the natural question is: where are we headed? The trajectory: Rapid Progress Where things are going matters more than where they are now. In early 2024, leading models struggled to find junior-level bugs. By 2026, they began cracking security challenges that stumped even security experts. Crucially, advances in offensive cyber capability are downstream of general AI scaling. If that relationship holds, we may still be near the beginning of the curve: as long as AI keeps improving, offensive capability will too. Our default expectation is continued rapid progress. The pace of change In the previous chapter we’ve discussed what AI can already do. However, to see where we are headed it’s important to understand the pace of change. What’s potentially most surprising is that the level of capabilities required to run many of the campaigns described in the previous chapter did not exist a mere 12 months ago. In early 2024, the best AI models in the world struggled with very basic cybersecurity tasks. In early evaluations, many AI systems could not reliably find simple vulnerabilities in short snippets of code. The kind of work a junior security researcher might do was largely out of reach for AI models. They were also highly likely to hallucinate or produce false-positives, further reducing their applicability to offensive cybersecurity. By the beginning of 2026, the picture looked very different. In roughly twenty months, we moved from models struggling with simple command injection bugs to models consistently solving tasks that require several layers of technical understanding: reverse engineering, cryptanalysis, vulnerability reasoning, and exploit construction. Figure 3 Left panel : a command injection vulnerability many models could not reliably find in early 2024. Right panel : a complex attack path a model was able to complete https://www.irregular.com/research/spell-bound-technical-case-study at the end of 2025 / beginning of 2026. This task requires figuring out the math behind bespoke cryptographic schemes, fixing a code repository, reasoning about where a program breaks, and writing the working exploit under a short time-constraint. Benchmarks . The same pattern appears in benchmarks. On Cybench, a widely used Stanford capture-the-flag benchmark, frontier models reportedly climbed from around 10% success to near-complete success in about two years. Substantial jumps could be seen across the field in more benchmarks¹. It is important to not overstate the significance of this data: benchmarks are not real cyber operations. But it does show how quickly classes of cyber tasks once beyond the reach of AI systems can come within it.² Beyond benchmarks. The same pattern we are seeing in the benchmarks is showing up in the wild, too. For example, curl, a major open-source project, ended its monetary bug-bounty https://daniel.haxx.se/blog/2026/01/26/the-end-of-the-curl-bug-bounty/ program on January 31, 2026 because of AI-generated "slop" - i.e., low-quality reports that wasted maintainers' time. By April, the maintainer reported a different problem: submission volume was roughly twice the 2025 rate, and reports were frequently enough being confirmed as vulnerabilities; a phenomenon he called https://daniel.haxx.se/blog/2026/04/22/high-quality-chaos/comment-page-1/ “high-quality chaos.” Real, valid findings arriving faster than humans could triage them. In just a single year, we’ve moved from “ AI is wasting our time” to “ AI is overwhelming us” . This example is joined by the many we’ve discussed in the previous chapter - most of them were beyond the reach of the best AI models in the world a mere year ago. Difficult evaluation tasks . One internal test illustrates the pace especially clearly: a custom CPU-emulator exploitation task requiring reverse engineering a Go binary, understanding a custom emulator, finding a race condition, and producing an Arbitrary Code Execution ACE exploit. In February 2026, no AI model could solve it. By April, the best model had roughly a 5% success rate. By June, multiple models were solving it reliably. The point isn't that one testing benchmark was saturated or that a challenge was solved. It is that hard security tasks are moving from impossible to possible within a very short time span. The trajectory: The industrialization of offense Models are covering more of the attack chain, solving harder problems, and beginning to execute campaigns. Costs are falling roughly 10x a year, and open-weight models are only months behind the frontier. The same capabilities that caused the U.S. government to place restrictions on frontier models are on track to be widely accessible within months. A conceptual framework for offensive security progress To see ahead , we need to understand what gets better when offensive security capability improves. How can we conceptualize offensive progress? The answer “better hacking” is too simplistic to help us understand what is coming. To think clearly about the question, it helps to remember what it usually takes to achieve an offensive objective in the real world. It is rarely one clever move. As we discussed, by-and-large meaningful intrusions often unfold over a chain of steps . This is why security practitioners often think in terms of attack chains⁵ - sequences of steps adversaries must complete to achieve their objective. That gives us a useful way to think about offensive progress. Not as one number going up, but as movement along several dimensions at once: Range . How many discrete links in the chain an AI can meaningfully perform at all. E.g., can it do any reconnaissance? Reverse engineering? Lateral movement? Complexity . How hard a version of each link an AI can handle. E.g., the sophistication of the vulnerability it can find, the difficulty of the exploit it can write, the obscurity of the system it can understand. Orchestration. Whether an AI can plan and orchestrate discrete steps into a longer-horizon goal. E.g., holding a state across a campaign, prioritizing, and recovering from failure when a path dead-ends. As such, the simplest conceptual framework to describe the shape of AI offensive progress is more steps, harder steps, longer chains of steps. As progress accumulates across all three dimensions, more attack chains become viable both for human attackers and for AIs to complete. This points to a likely trajectory: from assisting with more and more discrete tasks, through running short campaigns, to eventually conducting some sophisticated campaigns end to end. Empirically, we are seeing rapid movement in all dimensions. So where does that leave us? We’ve laid a framework for conceptualizing progress in offensive cyber capability. Models are improving across all three dimensions. They are already useful as assistants to capable actors across multiple tasks and can run short autonomous campaigns, but they remain limited in what they can achieve in the real world. Based on the trends above, my base case is that, at least over the next two years, frontier models will continue to roughly double their performance and effective autonomous work horizon every six months across many important offensive cyber tasks. This is an empirical view, not a law: longer campaigns will still require qualitative improvements in reliability, planning, stealth, and recovery from failure. But if the current trend roughly holds, relatively simple weeks-long autonomous campaigns should become a real-world capability soon – potentially even in the next few months. Further in the future, the most consequential campaigns will likely concentrate on high-value targets: banks, telecoms, energy companies, other critical infrastructure, and large organizations holding valuable data or assets. At the lower end the effect is different and more immediate. Absent intervention, organizations that lack basic cybersecurity hygiene could become very easy prey, producing a sharp increase in opportunistic attacks against poorly defended targets. We’re already seeing signs of that, which I’ll discuss more later in this essay. Proliferation The previous section was about AI capability. But capability is only one part of the story. Capability tells you what the frontier can do. Proliferation tells you how wide the access is going to be and how soon. Two trends are turning a capability story into a proliferation story. The first is that the cost of an offensive action is collapsing . A capability that is technically possible but highly expensive may be used selectively, usually against only a small number of high-value targets. A capability that is cheap gets used at scale. These are different threat scenarios. One of the most underappreciated facts about AI security progress is that as models improve, complex actions get cheaper. A smarter model with higher success rates means fewer resources spent on failed attempts. At the same time, the raw price of inference keeps falling underneath all of it. The price to reach a fixed capability level has been falling on the order of 10x per year https://arxiv.org/abs/2511.23455 for frontier models on knowledge, reasoning, math, and software-engineering benchmarks. Measurements suggest that for some tasks the price drop may be even steeper https://epoch.ai/data-insights/llm-inference-price-trends , and there are indications that offensive security is among them. For instance, the custom CPU-emulator challenge from earlier in this chapter exemplifies this. In a matter of months it went from unsolved, to occasionally solvable at an expected cost of roughly $2,000, to reliably solvable at around $20 a run. The absolute costs are just as striking as the rate of decline. One open-weight Chinese model, Z.ai's GLM-5.2, was measured finding vulnerabilities at roughly seventeen cents per vulnerability found https://www.darkreading.com/cyber-risk/chinese-llms-broaden-gap-between-attackers-and-defenders , while matching or beating leading closed models on some bug-finding benchmarks. A reasonable working assumption is that it takes less than a year from a capability appearing at the frontier to being affordable to attackers at scale. – The second trend is diffusion . Offensive capabilities do not remain confined behind gated AI models. They spread through multiple means such as open-weight models, distillation, leaked techniques, and ordinary imitation. This does not mean every actor immediately gets the strongest possible system. At the time of writing, the most capable models are still generally gated by access controls, monitoring, and policy restrictions. Those controls create meaningful friction for attackers. However, friction is not containment. The lag between frontier capability and broader availability appears short. In May 2026, NIST’s Center for AI Standards and Innovation CAISI reported https://www.nist.gov/news-events/news/2026/05/caisi-evaluation-deepseek-v4-pro that the open-weight DeepSeek-V4 Pro lagged the frontier by about eight months across its aggregate evaluation suite. In July 2026, UK AISI reported https://www.aisi.gov.uk/blog/how-far-behind-the-frontier-are-leading-open-weight-models-on-cyber an even tighter gap on cyber specifically. AISI found that recent open models like GLM-5.2 and DeepSeek-V4 Pro matched the performance of closed frontier models released four to seven months earlier. That was narrower than the six-to-ten-month gap it measured internally through most of 2025. A few months is not zero, but it is short relative to defensive adoption cycles. As discussed in Chapter 1, the capabilities of the recently released Kimi K3 indicate that open-weight models are on track to be able to run autonomous cyber campaigns within months. There are active efforts to slow diffusion that may widen the lag, such as by making direct copying harder. However, they are unlikely to prevent proliferation entirely. Open-weight models can be especially lucrative for attackers, since they can be used privately, without supervision and with virtually no guardrails⁷. To clarify, it is not that diffusion is always bad. Wider and cheaper access can be incredibly beneficial for defense we’ll discuss defense in Chapter 3 . The point is narrower: we should not assume frontier offensive capabilities stay bottled up for long. Concretely, based on the current trajectory, we can expect current frontier-grade cyber capabilities, such as Mythos 5 / GPT-5.6 Sol, to proliferate before the end of Q1 2027. And more broadly, the evidence suggests the time for diffusion may be measured in months. The industrialization of offense Put all the pieces together. Offensive capability is broadening, deepening, becoming cheaper, and diffusing. Individually, each of these is a notable trend. Together, the result is not merely "better hacking." It is a change in the production function of offensive security. For most of the history of cybersecurity, sophisticated attacks were scarce because the people who could run them were scarce. That scarcity was one of the hidden stabilizers of the entire ecosystem. AI changes that constraint, with two highly significant and simultaneous effects. For actors who are already capable, AI raises throughput . Intelligence services, elite criminal groups, and offensive-security teams can do more with the same number of people, because tasks that once consumed scarce expert-hours can now be automated or run semi-autonomously. A team that once had to choose one lead from ten may be able to explore a hundred. For less sophisticated actors, AI lowers the barrier to entry . More and more of the attack chain becomes something you can delegate rather than master. An actor who could not have written an exploit may be able to supervise an AI that can. So we should expect both more output from the already-capable, and a widening pool of the newly-capable. This shift is worth naming plainly. It is the industrialization of offense . A useful analogy is a workshop becoming a factory: more attempts, more targets, lower cost, and less expert labor per operation, enabled by delegating parts of the chain to machines. Staying on top of this process is hard. The pace of progress and diffusion are both rapid. A capability that moves from impossible to routine in months, while also becoming cheaper and more widely available, creates a different kind of pressure on defenders. None of this means that every offensive actor will be able to take any action they want with ease. As discussed above, we should be careful: models still fail in strange ways, still struggle with stealth, and still have poor judgment in messy environments. However, we should still expect offense to scale significantly. The trajectory: Beyond AI as a tool As AI becomes more capable, treating it simply as a tool becomes inadequate. AI is also becoming a target and an actor : something attackers can manipulate and that is capable of taking consequential actions on its own. This creates a new security paradigm centered on control and containment. Today’s security stack was not designed for machine-speed, general-purpose, reasoning systems with semantic attack surfaces and trusted access. A useful mental model is AI as a new frontier of insider risk. And emerging developments, from interacting agent networks to continual learning, could deepen the AI security challenge considerably. AI as a target As AI is deployed into real infrastructure, and especially as AI systems get access to code, communications, and credentials, the AI itself becomes one of the most valuable things to compromise. It may also be unusually valuable to steal an AI outright: model weights and algorithmic secrets can be the products of years of research and hold enormous economic value. We are already seeing this in practice. A few examples: Prompt injection . Researchers showed https://arxiv.org/abs/2509.10540 that a single ordinary-looking email could silently instruct Microsoft 365 Copilot to dig through a user's private files and leak them. The attack required no click or warning – only instructions hidden inside content the AI processed. Poisoning . Anthropic, the UK AI Security Institute, and the Alan Turing Institute found https://www.anthropic.com/research/small-samples-poison that as few as 250 malicious documents could plant a backdoor in a range of AI models. The models behaved normally until a hidden trigger caused them to produce an attacker-chosen failure mode. The demonstrated behavior was limited, but the result suggests that poisoning could become more effective. Hijacking . Instructions hidden in a public code repository allowed https://nvd.nist.gov/vuln/detail/cve-2025-53773 an attacker to hijack GitHub Copilot. The instructions caused GitHub Copilot to remove its own safety checks and run commands on the developer’s computer. The attacker did not exploit the machine directly; they persuaded the AI to do it for them. These are early examples, but the trend is likely to continue as users increase their adoption of AI systems and give them broad access. We should expect attackers to increasingly target defensive AI systems in particular. As defensive AI becomes the interface through which an organization sees threats and decides how to respond, compromising it lets an attacker distort that judgment. AI as an actor An AI given autonomy can take consequential offensive cyber actions on its own, including ones no operator intended and no attacker induced. These issues are no longer theoretical. A few cases: Emergent offense . In controlled experiments https://www.irregular.com/research/emergent-offensive-cyber-behavior-in-ai-agents , AI agents given routine tasks have disabled security protections because those protections stood between them and finishing the job. For example, an AI blocked by Windows Defender found a way to switch it off even though no one asked it to. In fact, it seems that AI systems are able to circumvent multiple modern security defenses by leveraging their offensive cyber capabilities. Beyond controlled experiments, there are numerous reports of AI agents considering offensive cyber actions, such as vulnerability exploitation, as legitimate tools in their toolbox for completing generic tasks; one recent example is a personal assistant agent discovering and exploiting vulnerabilities in a gym class booking website https://www.abc.net.au/news/2026-08-10/ai-assistant-hacks-gym-website-aus-cyber-attack/107007986 in order to fulfill a request to sign up for a popular class. AI worms. A further indicator of progress is when autonomous offense becomes self-propagating and adaptive. Researchers built an AI-powered worm https://arxiv.org/abs/2606.03811v1 that, rather than relying on a fixed set of exploits, could inspect each machine it encountered, reason about its vulnerabilities, generate a tailored attack strategy, compromise it, and replicate onward. It propagated through multiple generations and even commandeered compromised GPU-equipped machines to provide compute for further attacks. The important change is the intelligence inside the worm: traditional worms spread at machine scale using largely predetermined attack logic; an AI-driven worm can potentially combine that scale with the adaptability of a human operator, changing its strategy as it spreads. AI systems can themselves become part of the propagation surface: earlier work such as Morris II https://arxiv.org/abs/2403.02817 showed malicious instructions spreading from one AI application to another, effectively turning compromised AIs into both targets and vectors for the next infection. We have yet to see a large-scale real-world outbreak of either form. But as AI becomes more capable and more deeply connected to real systems, I expect adaptive and AI-to-AI worms to appear as well. Agentic offense against real targets . Controlled experiments are one thing. More consequentially, we are beginning to see agents direct the same kinds of capabilities against real-world targets. In July 2026, in a retrospective review of evaluations run with Irregular, Anthropic found three cases in which Claude reached the live internet and gained unauthorized access to the production systems of three real organizations due to a misunderstanding in the evaluation setup. One of the incidents had been found to be shared across multiple frontier lab models. Days later, in an unrelated incident, the UK AI Security Institute reported https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing that in 10 out of 122 cyber-evaluation runs, agents took unsanctioned actions against real people or organizations. In the most serious sequence, an agent attempted to place malicious code into a real open-source project, researched its maintainers, created fake identities, tried to socially engineer a maintainer into approving the code, and used Tor https://www.torproject.org/ to route around network restrictions. AISI emphasizes that these occurred under unusual, deliberately permissive evaluation conditions and that it found no resulting real-world harm. In other words, models are no longer merely performing isolated offensive tasks outside of benchmarks. They can also select intermediate targets, conduct reconnaissance, exploit systems, collect credentials, move between systems, and adapt when a path fails - the building blocks of short cyber campaigns. From sandbox to production. In the Hugging Face incident from July 2026 https://openai.com/index/hugging-face-model-evaluation-security-incident/ , described in Chapter 1, we saw evidence of models functioning as autonomous actors.The models appear to have inferred that Hugging Face might contain information useful to the benchmark they were trying to solve and began attacking its real production systems. The resulting behavior looked much more like a short cyber campaign than an isolated hacking task. One agent gained root access to an external machine and turned it into a launchpad, then found multiple paths into Hugging Face’s dataset-processing infrastructure. Another agent provided access to production credentials; another enabled remote code execution inside a production Kubernetes environment. The agents then harvested additional credentials, moved laterally across internal systems, established new footholds when paths were blocked, and continued adapting over several days. Hugging Face ultimately reconstructed roughly 17,600 individual actions. No human selected Hugging Face as the target or specified the attack chain step by step. The AI found and executed that path itself. Self-exfiltration . The same pattern could eventually create a stranger risk: an AI attempting to exfiltrate itself⁹. It is important to distinguish this from what happened at Hugging Face: in that incident, the agent’s actions breached the intended sandbox; the model itself did not. The models continued to run under OpenAI’s control while using tools and compromised infrastructure to act elsewhere. There is no evidence that they copied their own weights or established an independent version of themselves outside OpenAI’s control. Self-exfiltration would cross a different boundary: an AI would cause a copy of itself, or enough of its components to continue functioning, to exist somewhere beyond its operator’s control. Current frontier systems generally lack the capabilities needed to do this reliably. But we can predict that, as agents gain longer horizons, greater infrastructure access, and the ability to provision and manage compute, we will eventually see a highly credible attempt. If a future system encounters its own containment as an obstacle to completing its objective, self-exfiltration may look less like a completely new behavior than the same obstacle-clearing logic we are beginning to observe today, turned toward the boundary containing the AI itself. AI as a frontier insider risk problem The target and actor issues suggest an additional and profound shift: a growing share of security will soon become about controlling and containing AI systems . The challenge is that the existing security stack was not built for this. Many AI security attacks are novel, and as the examples above show, AI systems can be manipulated through context, poisoned through data, induced to misuse tools, or hijacked altogether. But more fundamentally, AI systems behave differently from traditional software. They are general-purpose, stochastic, and capable of reasoning. Part of their attack surface is also semantic, meaning they can be compromised through what they read, believe, or infer. A useful mental model is that AI is becoming the frontier of insider risk: deeply embedded in your systems, it is both an attractive target for attackers and an actor capable of a damaging judgment call of its own . Early signs of strain Open-source maintainers are being buried under a volume of AI-generated findings they cannot keep pace with https://www.theregister.com/software/2026/03/18/linux-foundation-wants-to-shield-foss-devs-from-ai-bug-slop/5229856 , let alone triage and fix. As described before, in a single recent month, more security bugs were reportedly patched https://blog.google/security/chrome-stronger-with-every-update/ in Chrome than in the previous two years combined. Security teams are also beginning to encounter more AI-assisted offensive activity. Campaigns such as GTG-1002 https://assets.anthropic.com/m/ec212e6566a0d47/original/Disrupting-the-first-reported-AI-orchestrated-cyber-espionage-campaign.pdf that we discussed before, and others https://unit42.paloaltonetworks.com/autonomous-ai-cyber-attack-campaign/ , show AI-orchestrated intrusion crossing from theory into real-world operations against real organizations, prompting a joint call-to-action from the Five Eyes cyber agency leaders https://www.ncsc.gov.uk/news/the-ai-shift-in-cyber-risk-why-leaders-must-act-now . Furthermore, the incidents detailed in “ AI as an actor” show another boundary beginning to move: frontier AI systems are starting to chain offensive actions into short campaigns, including, in some cases, against live targets. None of this amounts to a cyber crisis yet. Instead, these are better understood as pressure indicators. Vulnerability discovery is becoming cheaper and more prolific. Capable actors can automate larger portions of their workflows. AI systems are beginning to sustain longer attack chains. And in some cases, the distinction between an AI demonstrating offensive capability and actually using it against a live system is becoming uncomfortably thin. Taken together, the indicators point to a clear direction of travel: the bottlenecks that historically kept offense from scaling are loosening. That does not mean attacks will scale without limit. But it does mean that, absent proactive intervention, the amount of offensive pressure the security ecosystem will need to absorb is likely to rise substantially and perhaps quickly . All of this makes the security challenge ahead greater than anything the field has grown accustomed to. Which raises the obvious question: what happens to the defenders? Wait, what about the defenders? Defenders get AI too, so the real question is the offense-defense balance. I am optimistic in the long run, but it is all too easy to focus on the eventual equilibrium and neglect the risks of the transition period. I call this the end-state fallacy . The near term likely favors offense sharply: exploitation windows are collapsing while patch queues overflow, defensive deployment moves on slower institutional timelines, and AI systems themselves scale faster on offense. The key risk isn't that some systems get hacked — it's severity, scale, and simultaneity: correlated failures across important systems. The offense-defense balance It would be fundamentally misleading to tell only the offensive side of the story. Cybersecurity is deeply dual-use . The same model that discovers a vulnerability can be used by an attacker to exploit it, or by a defender to patch it first. The same system that reasons through an attack path can execute an intrusion or help close the attack path. As such, a deeply important question is not just whether AI will improve offense, but how it impacts the relationship between offense and defense. Practitioners call the underlying issue the AI Security offense-defense balance¹¹: the question of whether AI makes it structurally easier to attack or to defend. We can refer to a world where offense holds the advantage as offense-dominant , and to one where defense does as defense-dominant . A great deal of how the future of security actually unfolds will depend on this balance. The long run There are serious reasons to think the future will be defense-dominant. AI helps defenders in many ways . Defensive organizations are already becoming more efficient across a range of critical skills, such as scanning logs, identifying anomalous activity, finding vulnerabilities for patching purposes, auditing code, triaging incidents, automating portions of security engineering, among many other important skills. For example, in the recent incident involving an OpenAI model attacking Hugging Face https://huggingface.co/blog/agent-intrusion-technical-timeline , the Hugging Face defense team used an AI model to decipher attack payloads and to build analysis interfaces fast enough to keep pace with the investigation, and OpenAI used “millions of GPU hours” https://www.youtube.com/watch?v=87DyyMV0kCY as part of their own forensic analysis. Beyond incident response, programs like Anthropic’s Project Glasswing https://www.anthropic.com/glasswing and OpenAI’s Daybreak https://openai.com/daybreak/ aim to empower defenders with advanced AI models and tools and accelerate cyber defense. Extrapolate from the trend and you can imagine something much stronger: A future where AI systems continuously audit every line of code, formally verify the systems that matter most, monitor all suspicious activity, patch vulnerabilities faster than attackers can weaponize them, help security teams that are drowning in alerts understand what matters, conduct autonomous defensive operations, and more. In such a world, defense could exhaust the attack surface. But this is not guaranteed. There are also serious reasons to think AI could create an offense-dominant future. There is a structural asymmetry between attackers and defenders. Defenders need broad systemic assurance. They aspire to continuously secure every component, system, integration, user, and configuration. Attackers often need to find a single opening or attack path for a limited time to cause damage. AI may make both sides faster and more capable, but it does not dissolve this underlying asymmetry. Several additional dynamics push in the same direction. To name a few: The accountability tax. Defense is structurally harder to scale because defenders are accountable for the systems they protect. Defense demands sustained investment, deep knowledge of the specific systems being protected, and a kind of care that offenders can usually skip. For example, defenders usually cannot afford to break the system they are working on. Attackers may care about stealth, but most of the time they do not need the target system to remain healthy. Additionally, as AI becomes more autonomous, rather than becoming smaller, this gap could widen. Autonomous offensive systems can be deployed aggressively. However, by and large, autonomous defense has to be held to a higher standard: a bad patch, false-positive detection, or mistaken containment action can themselves each become an incident. Such asymmetries may mean that AI increases the number and speed of capable attackers whether human or AI faster than it raises the number and speed of capable defenders. An ever-expanding surface . AI is collapsing the cost of writing software - the total volume of code in the world may grow at an unprecedented rate in the years to come more apps, tools, integrations, etc. . At the same time, systems are already becoming more complex and more dependent on third-party components, open-source libraries, and supply chains. Both forces push in the same direction: a potential explosion in the number of things to secure. There is a hidden race here: If the cost of creating or attacking systems falls faster than the cost of securing them, the attack surface may grow beyond what defensive tooling can feasibly cover. The future could be less secure not because our tools got worse, but because we couldn’t keep up¹⁴. Variance in coverage. The previous point was about speed and cost, but there is also the issue of coverage . In the ideal case, defensive AI tools would find every important security issue before attackers do. Attackers and defenders would be looking at the same landscape and converging on the same issues. But if coverage is varied, the equation changes. Different tools, prompts, configurations, seeds, and starting assumptions may surface different security problems. In that world, attackers do not need to be better than defenders on average. They only need to be able to search a different part of the space. As long as the landscape is large enough, and no collection of tools covers it fully, an attacker starting from a different random seed may reliably find what the defender missed. Interpretability and trust. As AI capabilities advance, our ability to understand, predict, and assure these systems is not necessarily advancing at the same pace. Modern AI systems remain difficult to interpret; even their creators often cannot reliably explain why they make particular decisions or predict the conditions under which they will fail¹⁵. This has asymmetric implications. In some threat scenarios, an attacker can tolerate an opaque and unpredictable system if it succeeds often enough. A defender needs much stronger confidence before giving the same system sensitive or high-stakes responsibilities. To clarify, this goes beyond the accountability tax discussed before: the issue is not only that defenders must act with more care, but that fundamental limits in our ability to understand and assure AI systems may prevent some capabilities from crossing the trust threshold required for some important defensive use-cases. There are more structural reasons for offense-dominance. For example , geopolitical incentives: States that have spent decades building offensive cyber programs have every reason to ensure AI never neutralizes them and to invest heavily in keeping attackers ahead. However, none of this proves that offense wins in the long run. The honest answer is that we, including AI security insiders, do not confidently know where this lands . Every reason above has a credible counter. Take the ever-expanding surface . AI may cause the amount of software in the world to explode, but it may also drive the cost of securing each component down dramatically. Even if security remains more expensive than writing code, both costs could fall far enough that continuously auditing, testing, and hardening almost every important component becomes economically viable. And defenders may not need to secure everything equally well. A large fraction of systemic risk may ultimately concentrate in a much smaller security-critical core. Similarly, coverage gaps, opacity, and the accountability tax are the kind of problems that could be mitigated as a field matures an analogy is the internet: in the 90s the sheer volume of security holes looked insurmountable, and the industry built the capacity to bring it under control . And, the geopolitical incentive to scale offense could also be matched or outweighed by an incentive to scale defense. Personally, I'm optimistic that on a long enough horizon the world becomes more secure. However, the range of plausible long run outcomes is genuinely wide, and much depends on variables we do not yet understand well: how large the attack surface becomes, how automatable verification and remediation prove to be, how much trust we can place in increasingly autonomous defensive systems, among many other factors. But since a confident forecast may be a long way off, what we need is a strategy that recognizes the uncertainty and holds up across several possible futures. I now want to turn your attention to a related but different blind spot among those who usually engage in this debate. When the community argues about the offense-defense balance, the conversation often revolves almost entirely around the end state : who ultimately holds the advantage in the long run . That framing quietly assumes the thing that matters most is the destination. The reality is that the path matters too, and perhaps even more. The end-state fallacy The long run equilibrium and the transition period can have very different properties. Consider two goals you might set for yourself. One is getting in shape. The other is drinking a lot of alcohol. Getting in shape has an excellent end state and a hard path; drinking is the reverse, pleasant in the moment and costly later. If you evaluated either goal purely by the end state, you would badly misjudge the experience of actually pursuing them. We can call the mistake of ignoring this phenomenon the end-state fallacy : when reasoning about the future, collapsing the properties of the eventual equilibrium into the properties of the transition period, as though they were the same thing. For security, this issue is central: Even if the end state turns out to be defense-dominant, the next few years can still be sharply offense-dominant . The uncertainty is not evenly distributed: While we are unsure about the eventual security equilibrium, we can say something much stronger about the transition period. Absent deliberate intervention, the immediate future very likely favors offense . Why the next few years likely favor offense First, the discovery-to-exploitation window is collapsing while the patch queue grows longer . AI is driving the time between finding a vulnerability and having a working exploit toward zero. Patching runs on a different clock. Remediation in complex environments is slow, delicate, and risky. This is the accountability tax again, now measured in time. We already live in a world where publicly known critical vulnerabilities routinely sit unpatched for weeks https://www.verizon.com/business/resources/executivebriefs/2026-dbir-executive-summary.pdf , not because defenders are negligent, but because fixing them safely is genuinely challenging, especially in legacy systems. Attackers face almost none of these constraints. They simply need a path that works. The patching queue is visibly starting to buckle. As one example, at Pwn2Own Berlin 2026, for the first time in the competition's nineteen-year history, dozens of severe vulnerability submissions had to be turned away including working 0-day RCEs in Firefox, Docker, and other targets, including AI infrastructure such as Ollama, llama.cpp, and coding agents , simply because the organizers had run out of contest slots. When even the pipeline for demonstrating critical exploits overflows, there is a strong signal about the shape of the entire system: the supply of discovered, weaponizable flaws is beginning to surpass the capacity to process them let alone remediate them . The measured time from vulnerability to exploitation has already dropped https://zerodayclock.com/ by orders of magnitude over recent years, and the trend points further downwards¹⁶. We are heading toward a world where exploitation times are collapsing while patching lags behind. With decades of accumulated technical debt in important systems, we should expect a large backlog of open issues we cannot fully address, at least for a while. Second, defending AI is a new discipline, and building the defenses it requires will take time . AI introduces novel classes of attacks: prompt injection, model and data poisoning, adversarial inputs, and the hijacking of AI systems, among others. As the previous chapter made clear, AI turns the model from a tool into a target, and from a target into something that must itself be contained. These are not minor variations on familiar problems that existing tooling can easily be used to solve. As argued, traditional security products were not built for these challenges. The defensive playbook for all of this is being written now, in real time, and thus far it is thin. So on top of offense outrunning defense on the systems we know, there is an R&D gap: an entire attack surface has already arrived, while the discipline meant to defend it is still in its infancy. Third, defense deployment gaps . Defensive AI tools are only as useful as their deployment. A tool that can triage incidents, propose patches, or surface anomalous behavior in a lab is not yet a system safely operating inside a critical organization. Real-world organizations have fragmented infrastructure, legacy systems, inconsistent patching, unclear ownership, and years of accumulated complexity. In conversations with security leaders, one point comes up again and again: the bottleneck is not only model capability. It is knowing how to actually absorb the technology. Most organizations do not have an army of AI technologists waiting to rewire their security stack. This would be hard enough in a slow-moving field. But AI progress in specific use cases does not always arrive as a smooth ramp. In cyber, we have already seen capabilities sit flat for a while before accelerating rapidly. As we saw, a task that was impossible last quarter can become cheap by the next one. That leaves very little warning for some threat models, and it demands fast adaptation once the threat becomes clear. Yet defensive organizations do not move on that timeline. They onboard vendors over quarters, budget annually, and modernize infrastructure over years. Critical infrastructure and government often move even more slowly. This gap may shrink over time as organizations rewire themselves to absorb AI more easily. But during the transition, attackers face a much lighter burden: defensive capability becomes useful only after an institution has learned how to trust, buy, operate, and integrate it into a complicated system. Fourth, at least in the near term, we should expect that AI will scale faster on offense than on defense . This is a claim about what is easier to teach. For instance, it is easier to validate an exploit than to validate a patch. An exploit is often a local and concrete objective: it either works against its target or it does not, and checking it directly is straightforward. A patch asks something closer to global assurance: one must show that the system is secure under continuous, adversarial pressure, without having broken anything adjacent – a much harder thing to verify. This has a downstream consequence for AI. Where success is cheap to verify - it is easier to curate training data, run experiments, collect feedback, and improve the system – the advantage often rests with offense. Defense is usually much messier and harder to define. This is one of the key reasons I expect the advantage to go to the offenders in the near term. Finally, these effects do not occur in isolation . They compound with everything already described. Many of the reasons to expect an offense-dominant equilibrium apply just as well to the transition : the growing attack surface, incomplete coverage, the lower bar to fielding an autonomous offensive AI system, the structural asymmetry that lets an attacker win by finding an opening, etc. They also stack on top of the industrialization of offense described in the previous chapter: the decline in offensive cost, the diffusion of capable systems, the fact that AI is getting better across a widening range of cyber skills, the growing ability of AI to uplift attackers and run prolonged autonomous campaigns, among other factors. Where does that leave us? There are strong reasons to be optimistic about the long run security future. I am incredibly optimistic myself. But, put the aforementioned issues together and a real near-term risk emerges: defenders becoming overwhelmed . So much so that, in conversations behind closed doors, some researchers have started half-jokingly calling the coming transition "Vulmageddon" or the "Vulpocalypse." These terms sound cartoonish. The underlying problems are not. Defenders could drown in a volume of security issues that arrive faster than they can be triaged, prioritized, tested, or fixed. And these issues may not surface as a manageable stream. They could arrive in step-function surges, far faster than institutions can adapt. It is worth being precise about what actually matters: the deep issue is not “some systems get hacked.” That already happens constantly, and society is remarkably resilient to it. Many important systems have been compromised at some point, and the world kept working. The problem is a combination of severity, scale, and simultaneity : multiple severe issues across multiple important systems, all at the same time. A single bank dealing with an intrusion is bad, but our financial system can survive it. Several banks, hospitals, or energy providers suffering deeper or potentially simultaneous failures is a different category of event, with broad cascading effects that spill far beyond any single incident. That is why the next few years matter so much. If AI drives offensive pressure to scale much faster than defensive capacity, there may be a period in which defenders will not be able to respond coherently. Isolated failure is one thing, but severe and correlated failures is another. Even the near-term future is hard to predict. But there is a meaningful chance that this is the threshold the current trajectory is carrying us toward. So, what should we do about it? The path forward The challenge ahead is formidable, but it isn’t too late. We cannot stop the progress of offensive AI capabilities, but succeeding will require fielding defensive capacity before offensive pressure overwhelms it. The pragmatic strategy is differential defensive cyber acceleration DDCA : measuring the field, building capabilities that differentially advantage defenders, and managing offensive diffusion to buy time where needed. Framing the challenge AI will likely make the next few years more favorable to offense, but that does not make widespread security failures inevitable. The more specific risk is that offensive pressure grows faster than defensive capacity, eventually reaching a point where defenders cannot absorb the volume and severity of attacks. The objective, therefore, is not to stop AI-enabled offensive action entirely, but to keep the defenders from becoming overwhelmed. Many offensive capabilities will eventually become cheap and widely available, while defensive capabilities take more time to develop, deploy, and integrate into real organizations. As a result, the order in which those capabilities arrive is consequential. The challenge is to influence the sequence in which AI security benefits arrive in order to provide enough margin for defenders to adapt. Differential defensive cyber acceleration DDCA Differential defensive cyber acceleration DDCA is a strategy for altering the course of that sequence: deliberately shaping the development, diffusion, and deployment of AI security capabilities so that protective capabilities mature and reach defenders before corresponding offensive capabilities can overwhelm them . DDCA focuses on differential gains because many cybersecurity capabilities are dual-use. A tool that discovers vulnerabilities can help defenders patch them, but it can also help attackers to exploit them. Having the goal of “better security” does not necessarily lead to improving the offense-defense balance . So the word differential is key. The objective must be concentrating effort on capabilities and interventions that primarily advantage the defenders and getting those benefits into the field with sufficient speed and reach. Putting that strategy into practice requires progress on four fronts, each of which can act as a bottleneck on the pace of improvement in defensive capabilities. It is an R&D problem: we have to find interventions that significantly accelerate defense with a minimal beneficial impact on offense. It is a deployment problem: even the best intervention is worthless until institutions can absorb and operate it. It is a planning problem: the field moves fast and is deeply uncertain, and it is often unclear where to focus. And it is a talent problem: very few people have a deep enough understanding of both frontier AI and the operational realities of security to do this work. These constraints make it unlikely that a defensive advantage will emerge by default. Providing enough margin for defenders to adapt will require deliberate investment in the capabilities, deployments, and institutions that can keep defense ahead of rising offensive pressure. The remainder of this chapter focuses on the core tenets of this approach. What differential defensive cyber acceleration requires Ensuring defense is not overwhelmed will require investment across multiple lines of effort. In practice, DDCA rests on three main tenets: measuring the field , building defensive-specific capability , and dealing with offensive diffusion . Tenet 1: Measuring the field Keeping defense ahead requires understanding where the defensive margin is shrinking before the consequences become too difficult to manage. We need to understand where offensive capabilities are advancing, how quickly they are becoming cheaper and more widely available, and whether defenders are developing and deploying the capabilities needed to keep pace. Our current view of the field is incomplete. We rely on benchmarks, incident reports, intelligence assessments, bug-bounty data, provider telemetry, and reports from security teams and maintainers. These signals are useful, but they are fragmented, often delayed, and frequently measure different things. Benchmarks also become less informative as models saturate them. The measurements themselves need to move closer to operational reality. The relevant measured unit is increasingly the whole AI system rather than the base model alone: the harness, tools, memory, orchestration, and human support can together determine what an attacker or defender is able to accomplish. We therefore need to track whether systems can execute realistic attack chains, recover from failure, operate reliably in messy environments, and reduce the cost of successful offensive action. Alongside capability, we need to measure how quickly those systems diffuse and how quickly defensive organizations can absorb corresponding tools and mitigations to guide further action. If offensive capability continues advancing rapidly in an area where defensive adoption is weak, that should change where we invest in defensive R&D, which organizations receive the most deployment support, and where more stringent interventions could buy meaningful time. DDCA therefore requires a continuous feedback loop between measurement and intervention, rather than a static set of benchmarks. Some concrete examples of high-value efforts in this tenet include non-exhaustive : Tenet 2: Building defensive-specific capability Once the measurements point to where defense is falling behind, the next task is to build and deploy capabilities that shift the balance in favor of the defenders. The typical instinct here is to say: “build good security tools.” That instinct is not wrong, but it is incomplete. As we have discussed multiple times, much of cyber capability is dual-use. In this domain, building capability that happens to help defenders is relatively straightforward. Building capability that differentially helps defenders is much harder. As a result, the goal must not simply be “more AI security”; it is to concentrate on the areas with the most impact on the balance. A useful way to prioritize these capabilities is along a red-to-blue spectrum . Some interventions sit closer to the red end they mostly help offense . Some sit closer to the blue end they mostly help defense . And others are purple somewhere in the middle . The objective is to search for interventions that sit as far toward blue as possible: those that will do more on a structural level for the defender than the attacker. These blue-asymmetries exist, and they appear across research, development, and deployment. A few examples: Remediation . Finding security bugs helps both sides. Patching vulnerabilities at scale mostly helps defenders. Broadly speaking, compressing the path from a known vulnerability to a shipped fix is differentially defensive. Incident Response and threat intelligence . These areas of cybersecurity usually meaningfully benefit defenders while yielding limited value to offensive actors. Blue deployments . This can be defined as getting defensive tooling into the hands of critical organizations and helping them actually adopt and operate these tools. An attacker does not have to onboard a hospital; a defender does. Converting capability into actual blue capacity is a gain that is differentially defensive. Some concrete examples of high-value efforts in this tenet include non-exhaustive : Tenet 3: Dealing with offensive diffusion Even if we measure the field well, and even if we build better defensive tools, we still have to care about how quickly offensive capability spreads. If the previous tenet is about strengthening defense, this one is about buying enough time for those defenses to arrive before offensive capabilities overwhelm them. If we try to suppress everything that touches offense, we will also suppress much of the defensive work we need. Therefore, in this tenet we should proceed with caution. The differential goal is not to freeze the field¹⁷. It is to influence diffusion: limiting illegitimate offensive actions where possible while giving defenders the access and tooling they need. In this chapter, diffusion should be understood broadly, not only as who can access a capability, but also whether it can be copied or exercised outside the boundaries under which it was intended to operate. There are a few different pillars that are part of achieving this goal. Use limitations . These are the controls on who can use the strongest systems and for what. Some of the levers are already in use or are planned to be: refusal classifiers that decline certain offensive requests, anti-jailbreak measures, know-your-customer KYC rules and gating that restrict the strongest capabilities to vetted users, abuse monitoring to detect suspicious workflows, among others. These measures are imperfect, and some of them are often justifiably controversial. Every gate that slows an attacker may also create friction for researchers, defenders, and ordinary users. That is why the differential acceleration lens really matters: more R&D is needed to distinguish illegitimate offensive use from legitimate defensive work. Theft prevention . Guarding critical AI assets is a related but distinct diffusion issue. It is not merely another use limitation - if an AI system is stolen, distilled, or leaked, all of the safeguards could be easily removed. Consequently, all of its capabilities risk becoming available without monitoring, gating, or any other limitations. Hence, protecting these assets is part of preventing offensive diffusion from jumping too much ahead of defense. Theft prevention interventions are not without tradeoffs as well. Putting in place security controls that are too stringent can slow down the entire AI field and stall critical developments. More fundamentally, more openness could actually benefit security e.g., via allowing more scrutiny . Therefore, here too active effort is required to strike a good balance that allows improving security while preventing critical systems from falling into the wrong hands. AI containment . A third pillar is more forward-looking: offensive diffusion is not only about who can access powerful AI models, but also about keeping increasingly autonomous systems within their intended boundaries. As discussed in Chapter 2, traditional security tools still matter, but they were not built for entities that are general-purpose, adaptive, stochastic, capable of reasoning, with semantic attack surfaces, and increasingly capable of coordinating with other agents. Containment should therefore be treated as a diffusion problem in its own right, requiring dedicated R&D . We should be realistic about what the aforementioned measures can achieve. The history of security suggests that much of the underlying capability will eventually diffuse. Limitations should therefore not be treated as permanent solutions. By and large, they are instruments for buying time. We are unlikely to hold back offensive diffusion forever, but by shaping how intensely, early, cheaply, and widely it arrives, we can give the approaches described in the first two tenets time to work . Some concrete examples of high-value efforts in this tenet include non-exhaustive : Closing thoughts The defense won’t need to win every battle. But we also have no time to lose. We won’t need to win every battle We should, of course, aspire to stop every serious attack. But the defensive strategy cannot depend on winning every battle. Almost every important system has been compromised at some point. The more worrisome scenario involves attacks combining severity, scale, and simultaneity: many critical systems failing together in a correlated fashion, beyond the capacity of defenders to respond, until failures begin cascading into one another. Infectious disease management provides a useful, if imperfect, analogy. “Flattening the curve” does not mean preventing every infection, but rather slowing the spread enough to keep hospitals from being overwhelmed and buying time to develop medical countermeasures. Cyber resilience may require something similar: measuring where pressure is building, slowing offensive diffusion where relevant, and keeping critical systems functioning while stronger defense capacity is built. No time to lose We should not expect the problems discussed in this essay to resolve themselves. Because AI-enabled security capability tends to scale more easily on offense, a defense-favorable path will not emerge by default. Therefore, a defensive edge has to be deliberately built out, deployed, and maintained. The right response is neither to deny the trajectory nor to surrender to it. It is to take action where it is possible to move the needle. There is particular urgency in moving now, while the stakes are still comparatively manageable. Progress is rapid, and if we wait for a full-blown crisis to force the issue, we will be left with blunt, reactive measures: the kind of sweeping bans that lock out defenders along with the attackers, imposed under exactly the conditions in which good judgment is hardest to exercise. The earlier we invest in differential defense, the more room we have to pursue targeted interventions rather than emergency ones. I remain genuinely optimistic that, on a long enough horizon, AI can make the world far more secure than it is today. But that future is not guaranteed, and it is certainly not automatic either. It has to be built, and the immediate task before us is ensuring the defenders have the capacity needed to hold the line through the upcoming transition period. The clock is ticking. — Thanks to Omer Nevo, Alon Oring, Edan Maor, Gil Gekker, Yoni Rozenshein, and many people at Irregular for reviewing drafts of this essay. 1 Some examples include NYU CTF Bench https://nyu-llm-ctf.github.io/ , where GPT-4 started at single-digit solve rates to practically saturated today, and CyberGym https://www.cybergym.io/cybergym/ , where most models have gone from low single-digits to saturating the benchmark over the span of 18 months. 2 Some argue that all AI is capable of doing is to memorize the tasks: because the benchmarks are public, they are part of the training set. Yet, we’re seeing a similar effect https://www.irregular.com/research/emerging-evidence-of-a-capability-shift on private benchmarks as well. 3 Although worth noting that labs are releasing bespoke cyber versions, e.g. GPT-5.6-Cyber and Gemini 3.5 Flash Cyber. 4 This is a projection, and speculative by nature. The debate on scale gets much hotter once people reason in terms of AGI or superintelligence. But there is no reason to bundle these terms to the point being made here. The argument here is simply that there seems to be more room for scaling in a way that impacts security, regardless of whether we ultimately get to superintelligence. 5 Lockheed Martin’s Cyber Kill Chain https://www.lockheedmartin.com/en-us/capabilities/cyber/cyber-kill-chain.html is an example of this concept. A Framework for Evaluating Emerging Cyberattack Capabilities of AI https://arxiv.org/pdf/2503.11917 by Google DeepMind also expands these concepts into an AI context. 6 The 4.7-month doubling time comes with important caveats AISI stresses. It's measured under a deliberately low budget of 2.5M tokens per task, a cap AISI imposes to keep results comparable over time but which it says understates what the models can actually do. 7 Attackers may require significant on-premises compute - but certain attackers especially nation-state attackers are already accustomed to paying this “compute tax.” Note that attackers do not necessarily need frontier models; they only need models that are good enough for the attacker to find utility in them. This will be the case as long as models are better at offense than defense or to be more precise, as long as open-weight models are better at offense than frontier models are at defense . 8 Full disclosure: as mentioned above I am affiliated with Irregular I’m one of the founders . Irregular shared more information on these incidents https://www.irregular.com/research/addressing-recent-incidents-ongoing-findings-and-path-forward , and a whitepaper is being developed to deal with such issues in the future. 9 This has already been demonstrated in controlled experiments https://arxiv.org/abs/2605.06760 . 10 There are many overlapping terms for possible milestones along this path, and for its possible destination: AGI https://internationalaisafetyreport.org/publication/international-ai-safety-report-2025 , powerful AI https://darioamodei.com/essay/machines-of-loving-grace , recursive self-improvement https://spectrum.ieee.org/recursive-self-improvement , superintelligence https://plato.stanford.edu/entries/ethics-ai/ , among others. Their definitions differ and are contested. Since this essay is focused on security, I will avoid turning it into a taxonomy; the linked sources provide useful starting points for readers who want to go deeper. 11 The offense-defense concept originates in international-relations scholarship, and was adapted to cybersecurity see Jervis https://www.jstor.org/stable/2009958 and Slayton https://www.jstor.org/stable/26777791 . A reservation is in order: in an AI security context the term is used more loosely than in International Relations/security tradition. In the context of this essay I use it broadly, to capture the relative benefit that attackers and defenders derive from AI. 12 These labels are conceptual, and should be treated as directional rather than precise. E.g., "Offense gains more from AI" and "the world is offense-dominant" are related but distinct claims: in principle AI could confer larger marginal gains on attackers while the world remains in absolute terms easier to defend than to attack or the reverse . 13 One reason some expect securing a system to cost more than building or attacking it: defense needs a wide context. Attacking a component can be local; securing it means understanding how it interacts with its dependencies, configurations, users, business logic, and the whole surrounding system. 14 Currently, there is a gap between an AI system's ability to rapidly develop software and its ability to secure the software it develops. For example, we have seen models mishandling password security https://www.irregular.com/research/vibe-password-generation in generated code. It has also been shown that other types of vulnerabilities are likely to be introduced by AI coding agents https://www.tenzai.com/blog/bad-vibes-comparing-the-secure-coding-capabilities-of-popular-coding-agents . 15 This is made worse if an AI system cannot be trusted to be aligned, e.g. if there are bugs in the alignment training code as was observed years ago during the training of GPT-2 https://arxiv.org/abs/1909.08593 , or if the training data was poisoned https://www.anthropic.com/research/small-samples-poison by a sophisticated attacker, or if the model authors are actively interested in misalignment. The latter is especially a risk in open-weight models, fine-tunes, quantizations, and distillations by anonymous model creators, whose intentions are unknown. Potential intentional misalignment has also been observed in some models: A research report by CrowdStrike https://www.crowdstrike.com/en-us/blog/crowdstrike-researchers-identify-hidden-vulnerabilities-ai-coded-software/ showed that DeepSeek-R1 is more likely to introduce security flaws in generated code based on political triggers. 16 The standard process of responsible disclosure , designed to disclose vulnerabilities only when remediation is available, has also started to fail in 2026: Increasingly, there have been cases of vulnerabilities being exposed worldwide https://www.microsoft.com/en-us/msrc/blog/2026/05/a-shared-responsibility-protecting-customers-through-coordinated-vulnerability-disclosure instead of being reported privately to the vendor to allow time for patch development. The ease of discovering new vulnerabilities with powerful offensive AI, combined with the slow by today’s standards processes for patching them e.g. the 90-day disclosure policy https://projectzero.google/vulnerability-disclosure-policy.html , and “Patch Tuesday”, the once-per-month patching cycle used by several companies may lead to more such cases in the future. 17 Further, we must recognize that an intervention that constrains offense but imposes comparable or greater costs on defense is not worthwhile. To cite this article, please credit Irregular with a link to this page, or click to view and copy the BibTeX citation. Disclaimer: While Irregular works with major AI developers, no proprietary, confidential, or non-public information from any such organization has been used in the preparation of this content. The positions and views expressed herein belong solely to the authors and do not necessarily represent those of Irregular's clients, partners, or any third party. All content is based on publicly available information, general AI and cybersecurity field knowledge, and independent research by Irregular employees.