How Narrative Wrapping Affects LLM Refusal: A Cross-Language Benchmark and Defense

A new arXiv paper (2610.11005v1) reports that narrative or role-play wrappers let harmful requests bypass safety refusals in Qwen3-1.7B at 89.4% attack success in English, 93.0% in modern Chinese, and 95.7% in Classical Chinese. The authors built GUISE, a cross-language benchmark with parallel English, modern Chinese, and Classical Chinese requests, matched harmful and benign pairs, held-out wrapper types, and a stricter criterion counting warn-then-answer responses as successes, and found that language and register shift harmful-request representations only slightly from the refusal direction while narrative wrappers move them much farther. They propose AXIS, which combines preference optimisation with a rotation objective aligning harmful-request representations to the refusal direction and a commitment objective training full refusal, and report AXIS achieves the highest combined safety and usability score across Qwen3-1.7B, Qwen3-4B, and GLM-4-9B.

arXiv:2610.11005v1 Announce Type: new Abstract: Safety-aligned language models often refuse a harmful request stated directly but answer the same request inside a role-play or narrative wrapper. We measure this vulnerability across languages and registers: attack success on Qwen3-1.7B is already 89.4% in English and 93.0% in modern Chinese, and reaches 95.7% in Classical Chinese. We build GUISE, a benchmark for systematically studying this vulnerability. It includes parallel requests in English, modern Chinese, and Classical Chinese, matched harmful and benign pairs, wrapper types held out for evaluation, and a stricter criterion that counts warn-then-answer responses as attack successes. Representation analysis shows that language and register move harmful-request representations only slightly away from the model's refusal direction, whereas narrative wrappers move them much farther away. We propose AXIS, which combines preference optimisation with a rotation objective that aligns harmful-request representations with the refusal direction and a commitment objective that trains the model to refuse completely rather than produce a warn-then-answer response. Across Qwen3-1.7B, Qwen3-4B and GLM-4-9B, AXIS achieves the highest combined safety and usability score among the compared methods.