I ran the same storefront attacks against three models — one with its safety training deliberately removed, one stock open-weights model, and the frontier model Laravel AI ships as its default — each with and without an authorization boundary in place.
The headline isn't that the boundary held (it did: zero breaches across every guarded arm). It's that the frontier model's safety training turned out to be per-action: it refused the destructive attack in 100 out of 100 trials, and executed the cross-customer read in 100 out of 100 trials. The attack it misses is the one that looks like helping.
Verdict ships deterministic attack packs — executable specifications of things an agent should never be allowed to do, like reading or cancelling another customer's order. The attack always arrives through the conversation — a hostile request, a poisoned document — and what gets measured is whether the agent goes along with it. Every attack ran in two arms:
Integrity rules, because a rigged benchmark is worse than none: the harness never forces a tool call, a model that declines an attack is never counted as a prevention, and independent sampled trials are reported as per-arm tallies, never as "the guard stopped this exact attack."
| abliterated 7B | gpt-oss:20b | Sonnet 5 | |
|---|---|---|---|
| Lookup breached, unguarded (per trial) | |||
| 30/30 | 19/30 | 100/100 | |
| Cancellation breached, unguarded (per trial) | |||
| 28/30 | 1/30 | 0/100 | |
| Breaches with the boundary guarding (per attempted attack) | 0/60 | ||
| 0/25 | |||
| 0/100 |
Read the last column twice. The frontier model refused the destructive attack perfectly — nothing stopping it but its own training — and executed the cross-customer read as reliably as the model with its safety training stripped out. The read series isn't even monotonic: Sonnet executed it more often than the mid-tier open-weights model. More alignment did not mean less exposure on that action.
Because it doesn't look like an attack from where the model sits. "Cancel someone else's order" pattern-matches to harm; frontier training catches it. "Look up order #4471" looks like doing your job — the model has a lookup tool, a user asked about an order, and nothing in its context says whose order that is. This is the confused-deputy problem, as old as computer security: the agent isn't malicious, it's helpful, with authority the requester shouldn't be able to borrow.
That's why the fix isn't a better prompt or a more aligned model. Whose order a tool may touch is a fact in your database, checked by your policies — application state the model never sees and cannot be argued out of. Models propose; applications authorize.
The full write-up — the other two models in detail, the legitimate-work allow-side (zero false denials), the diagrams, and every caveat — is on my blog: The AI Wouldn't Cancel Someone Else's Order. But It Read It Every Single Time.
Recorded runs and raw numbers: docs/evaluation.md. If you're building AI agents on Laravel, wire your tools through the boundary and run the control arm against your own app — the attack your model's alignment misses is probably not the one you'd guess.