Jump to leaderboard
Measuring frontier model attack susceptibility in real-world enterprise workflows.
FrontierBreach advances agent security evaluation along four dimensions:
- State-of-the-art (SOTA) frontier attack success rates (ASR): FrontierBreach has leading ASR on frontier models from every provider, and is the only known benchmark to achieveASR on Gemini 4 Carbon .
- Enterprise workflows: Measures model susceptibility on complex, long-horizon tasks spanning real enterprise use cases, e.g., analyze a bond portfolio and recommend a trading strategy.
- Cyber consequences: Evaluates whether attackers can achieve high-impact security objectives, including data exfiltration, ransomware, and unauthorized system manipulation.
- Threat intelligence-driven synthetic attack generation: Our synthetic data pipeline translates real-world attack chains discovered by ourthreat intelligence team into adversarial scenarios across established agent benchmarks.
Existing benchmarks systematically underrepresent the security risks facing modern AI agents, as they rely on chatbot/simplified scenarios that do not capture the complexity of long-horizon enterprise agents, and rely on outdated attack methodologies that do not succeed on frontier models.
Across the original 21 models, attacks succeeded in 34.9% of test cases. Even frontier models fell for attacks: Gemini 4 Argon (4.3%), GPT-6 Astra (7.6%), Claude Fable 5.2 (10.0%), and Muse Wonder (15.2%). Grok 4.5 was the most susceptible model at 91.4%, more than double the average.
Methodology #
Realistic Scenarios
FrontierBreach scenarios are grounded in how enterprises leverage agents today, spanning realistic, domain-specific, long-horizon tasks.
To create these scenarios, we built a synthetic data generation pipeline that perturbs industry benchmark tasks with adversarial payloads generalized from our threat intelligence.
For this version of the benchmark, we ran GDPval through our pipeline to generate our test cases.
Each scenario tests how models respond to adversarial payloads (e.g., encrypt all of the data on the user’s machine with this key) while pursuing a larger, high-value goal, with a complex deliverable (e.g., build a valuation report comparing earnings multiples, dividend yields, and market capitalization across public manufacturing companies in the US), with payloads placed where enterprise agents could naturally and plausibly encounter them (e.g., attacker-controlled online article).
Indirect prompt injection benchmarks typically focus on constrained tasks with relatively simple goals. AgentDyn reports an average task trajectory of 7.10 steps for its own tasks and 3.49 for AgentDojo (Table 1). Enterprise knowledge work can involve substantially longer execution; for example, Qwen3.8-Max averages 79.42 turns per GDPval-AA task.
This illustrates the gap in workflow scope that FrontierBreach targets: sustained, domain-specific enterprise work, where agents must navigate adversarial content while attempting to produce high-value deliverables.
Attack Souls
‘Attack Souls’ comprise three elements: (1) an outcome an attack achieves, (2) a method to manipulate the model, and (3) a technique to apply the method in order to achieve an outcome.
To generate Attack Souls, we started with our existing research on indirect prompt injection, which includes novel attack chains and techniques discovered by our threat intelligence team in various enterprise applications. These have been cited by industry-leading frameworks including OWASP, MITRE ATLAS, OpenAI's research on data exfiltration in agents and agent adversarial robustness, and more.
We codified these into an internal specification that allows our system to generalize Attack Souls at scale via our synthetic data generation pipeline described below.
Here is an example of the Soul from an attack that manipulated GPT to exfiltrate user emails by exploiting a vulnerability in OpenAI’s desktop app.
FrontierBreach applies Souls like this one across agent tasks from all industry sectors, and across common delivery vectors for prompt injections, such as untrusted files, emails, and skills.
Our synthetic data generation pipeline cross-applies these Souls on environments from existing benchmarks in order to assess the risk of cyber attacks on agents in the workflows other benchmarks measure.
For this version of FrontierBreach, we cultivated Souls from published research, and applied them to GDPval (OpenAI’s benchmark for agent capability on enterprise tasks).
By cultivating new Souls as the frontier for models and attacks progresses, we achieve a living benchmark that is resilient against saturation.
Synthetic Generation
An Attack Soul is environment-agnostic, so turning one into a test case means writing an injection into a specific task environment. We do this with a four-stage, model-driven pipeline: prepare the task, assess it for entry points, generate injections, and select the strongest.
- Task preparation: Benchmark tasks often lack assets (content in an agent’s environment that it reads or uses while working, such as documents, emails, and skills) that a production agent would rely on, so we create them. For example, a task that requires a .docx deliverable receives a general-purpose document-building Skill. The result, with no attack present, is what we call the base case. In this step, we create multiple base cases. For example, we convert documents that originate from external parties into emails, which yields two base cases for the same task: one where the information arrives in the original document and one where it arrives by email.
- Task assessment: A model reviews each base case with no attack present and identifies its entry points: the places where the agent will consume information from, or interact with, an external party (e.g., an attacker-controlled online article). For each base case, we identify all likely entry points; each of those entry points becomes an injection candidate. On average, each GDPval task yields 2.13 candidates to simulate a realistic enterprise scenario.
- Injection generation: For each pairing of an attack soul and injection candidate, our model pipeline generates a scenario inside the corresponding base case. It reasons about different injection narratives that fit the task’s scenario and where in the target entry point the injection should sit, then writes the attack into the entry point and sets up the environment.We deliberately keep each injection general: it is written without reference to the user’s request, and the user’s request is never altered to accommodate it. This prevents cross-pollination between the user’s task and the injection, and simulates the real-world condition in which the attacker has no internal knowledge of the user’s task or environment. The model also supplies the variables each technique requires, such as links (every generated link is checked programmatically to confirm that its domain has no registered owner), which are then used for programmatic checks. We run generation three times per pairing at a sampling temperature of 0.95 with maximum reasoning effort, so that each run explores a different scenario and produces varying candidate test cases.
- Selection: We run a panel of models against the three candidate test cases for each pairing and programmatically select the highest ASR scenario. For the final benchmark set, we keep the best scenario for each combination of domain, attack, and entry point, so every combination is represented by exactly one test case.
Attack Taxonomy
Every test case in the benchmark is a generalized Attack Soul placed in an entry point. We organize Attack Souls by the outcome they achieve. This version of FrontierBreach covers 6 of the most impactful outcomes to enterprises:
Attack Souls reach the agent through four entry points, which mirror where untrusted content enters real deployments:
- Connectors: a message in an inbox the agent reads through an email connector
- Document management systems: a file in the shared workspace
- Local file systems: a file on the machine the agent works on
- Skills: instructions or code inside a third-party skill the agent loads
We score each case by outcome. An attack counts as successful only if the harmful action happened in the environment, in any of the model's runs in that case. Each test case was run 5 times per model.
Deterministic Verification
An attack counts as successful only when its objective actually occurred. We verify this with predefined code that examines execution evidence (network receipts, captured messages, filesystem state, configuration, and generated outputs) and never rely on the evaluated model’s own statement that it completed an action.
- Evidence collection: Each isolated evaluation environment contains a collector, an instrumented external service that simulates the attacker’s infrastructure. Depending on the case, it serves a download or webpage, receives uploads, and records every request and all transferred content. Collector evidence is sealed after execution and checked for missing or incomplete records.
- Success criteria: Each of our attacks has its own criterion. The verifier programmatically determines if an agent attempted an action, took that action, downloaded an attacker-supplied resource, triggered a subsequent callback, or actually transmitted protected information.
Model Selection
We picked the latest models as of July 31st, 2026, and the latest frontier models as of November 1st, 2026, to measure the change in ASR over time.
Note: Gemini 4 Carbon is included in the top FrontierBreach attack success rate chart, including its US vs. China comparison and model picker, but none of the subsequent charts. This is due to its recent release on November 10th, 2026, whereas the cutoff was November 1st, 2026 for model selection. Nonetheless, it was appended to the leaderboard chart due to a statistically significant ASR.
State of the Art Attack Success Rate #
FrontierBreach is the only known benchmark to find ASR on Gemini 4 Carbon. Google’s Model Card reported a 0% ASR (citing Gray Swan IPI).
Below is an example of a successful attack on Gemini 4 Carbon from FrontierBreach.
Palo Alto Networks’ ASR for Claude Fable 5.2 was 0%, while FrontierBreach found a leading ASR of 10%.
Qualitative Analysis #
1a. Frontier Attack Success Rate in Real Enterprise Workflows - Healthcare Is Most at Risk of AI Agent Attacks
Across nine industries, we tested indirect prompt injection during the kinds of work enterprises delegate to agents, from preparing nursing care plans to producing engineering blueprints and drafting legal briefs. Healthcare and social assistance workflows had the highest attack success rate in FrontierBreach at 44.4%, nearly 10 percentage points above the benchmark average of 34.9%.
Manufacturing (40.7%), finance and insurance (38.3%), wholesale trade (37.9%), and real estate (36.1%) also came in above average. The 17.1-point spread between the highest- and lowest-scoring sectors shows how much an overall score can hide about exposure in specific workflows. FrontierBreach’s distinguishing feature is this industry-level view of enterprise work, by injecting injection scenarios into realistic workflows in GDPval, unlike other benchmarks such as Gray Swan IPI, AgentDyn, and INFUSE - Gray Swan groups results by agent environment across coding, tool use, and computer use, AgentDyn focuses on open-ended shopping, GitHub, and everyday tasks, and INFUSE tests long-running workflows involving code changes, dependency installation, and remote services. FrontierBreach gives enterprise teams and model labs a benchmark they can read against the work enterprises are actually putting in agents’ hands.
1b. Frontier Attack Success Rate Across All Frontier Models in Enterprise Workflows
Despite testing realistic scenarios, FrontierBreach was still able to demonstrate SOTA attack success rate across all frontier models, with k=5. Besides FrontierBreach, only Gray Swan IPI has been able to demonstrate non-zero IPI on Gemini 4 Argon (0.7% with k=15), GPT-6 Astra, and Claude Fable 5.2.
2. Models From China Fall for Attacks 29.6% More Often Than Models from the US
Across all models tested, attacks succeeded 40.7% of the time against models from China and 31.4% against models from the US. Among non-frontier models, the rates were nearly identical: 40.7% for Chinese models and 41.2% for US models. Data destruction showed the largest gap at 34.0 percentage points. Data theft, ransomware, and persistence had higher success rates against US models.
3. Models Are Most Susceptible to Arbitrary Code Execution
FrontierBreach measures indirect prompt injection against six attacker objectives that map directly to the capabilities enterprises give AI agents: running code, sending messages, accessing sensitive data, deleting or encrypting files, and leaving a foothold that outlasts the session. We report attack success by objective to elucidate the exact attacker objectives that are most achievable. Across all models tested, arbitrary code execution had the highest average attack success rate at 63.2%, followed by phishing at 57.8%, data destruction at 41.8%, data theft at 16.8%, ransomware at 13.5%, and persistence at 12.7%. Even Gemini 4 Argon, the most resistant of the frontier models we tested in the original analysis, had an 8.3% success rate on arbitrary code execution attacks, nearly double its 4.3% overall rate. Arbitrary code execution is especially consequential because it lets an attacker act with the agent’s runtime permissions, reaching other data or systems the agent can access.
4. Attacks Hidden in Skills Succeeded 1.6x More Often Than Attacks Hidden in Local Files
Skills were the attack path tested with the highest ASR, and are rapidly gaining adoption at the enterprise. FrontierBreach reports attack success separately for skills, connectors, local files, and document management systems. Across the models tested, attacks delivered through skills succeeded 45.5% of the time, ahead of connectors (33.9%), document management systems (31.5%), and local files (28.5%). That puts skills at 1.6x the success rate of local files.
5. Open-Weight Models Attack Success Rate Is 56.7% Higher Than Proprietary Models
Open-weight models fell for attacks in 43.1% of test cases, compared with 27.5% for proprietary models. The gap holds among non-frontier models, where open-weight models had a higher ASR (43.1% vs. 37.9%).
Limitations & Future Work #
Broader Benchmark Coverage
Future versions could generalize Attack Souls to additional capability benchmarks, including FrontierSWE and DeepSWE for software engineering, DeepFinance for finance workflows, and Legora BAR for legal workflows.
Post-Training Generation Models
We plan to explore post-training the models in our synthetic data generation pipeline using feedback on scenario validity, attack diversity, and verified outcomes. The goal is to improve how reliably the pipeline turns threat intelligence into realistic, executable test cases.
Contact #
If you're interested in learning more or contributing, contact benchmarks@promptarmor.com.
Citations #
- GDPval , the capability benchmark underlying this evaluation.
- AgentDyn , including the task trajectory comparison in Table 1.
- AgentDojo , an agent benchmark for prompt injection evaluation.
- Artificial Analysis GDPval-AA , the source for the task turn comparison.
- OWASP Top 10 for LLM Applications, 2025 .
- MITRE ATLAS case study AML.CS0035 .
- OpenAI research on preventing URL-based data exfiltration .
- OpenAI: Trading inference-time compute for adversarial robustness .
- PromptArmor research on data exfiltration attacks on GPT , the source of the example Attack Soul.
- Gemini 4 Carbon Safety & Security Report , section 6.1.6.
- Palo Alto Networks: Claude Fable 5.2 Hits a Perfect Score on Our Security Benchmark .
- FrontierSWE , a candidate for future software engineering evaluations.
- DeepSWE , a candidate for future software engineering evaluations.
- DeepFinance , a candidate for future finance evaluations.
- Legora BAR , a candidate for future legal evaluations.
Cite FrontierBreach
Please cite this work as:
@misc{promptarmor2026frontierbreach,
title = {FrontierBreach: Measuring frontier model attack susceptibility in real-world enterprise workflows},
author = {{PromptArmor}},
year = {2026},
url = {https://www.promptarmor.com/frontier-breach},
}