everysuccessful attack; you need the ones that actually move the needle on model resilience.
This is where a defender-centric approach changes the game. Instead of asking "Did this break the model?", the question becomes "Does including this attack in my training set actually improve downstream safety?"
The A-MESS framework tackles this by using AttackSHAP—a Shapley-based scoring system. For those unfamiliar, Shapley values are typically used in game theory to determine how much each player contributes to a total outcome. In this context, it treats individual jailbreak attacks as "players" and calculates their marginal utility toward improving the model's safety.
The findings here are pretty revealing:
ASR vs. Utility: High attack success rates correlate weakly with actual safety utility. A "noisy" jailbreak might have a high ASR but provide zero value for alignment training.Subset Optimization: Selecting a compact subset of high-utility attacks is significantly more effective than just dumping every successful attack into the training pipeline.Efficiency: AttackSHAP can be estimated accurately without needing an infinite number of utility queries, making it practical for real-world LLM agent development.
Essentially, we need to stop treating jailbreaks as just "failures" to be patched and start treating them as high-value data resources. The goal isn't just to stop one specific prompt, but to identify the underlying vulnerability that the prompt exposed.
If you're building a safety pipeline from scratch, focusing on marginal utility rather than raw success rates will likely lead to a much leaner and more effective red-teaming dataset.
[Next Preemptive Hardening for Agentic LLM Security →](/en/threads/2326/)