The Safeguard Worked. Is the LLM System Safer?

A new evaluation framework argues that refusal rates and attack success rates do not measure how much harmful assistance a deployed LLM service still provides, and proposes a complementary metric based on the expected utility of the service's responses to harmful requests.