Between Suppression and Collapse: Evaluating Narrative Unlearning with LENS

A new study introduces LENS, a protocol for evaluating whether machine-unlearning algorithms can suppress disinformation-aligned narrative frames in large language models. Testing on four near-12B multilingual models, the researchers found that selected checkpoints reduced narrative reproduction and that suppression could transfer beyond direct forget prompts, though entity recovery emerged as a side effect. The findings demonstrate LENS as a diagnostic tool for guiding narrative unlearning research.

arXiv:2607.22657v1 Announce Type: new Abstract: Large language models LLMs can reproduce disinformation-aligned narrative frames as plausible explanations, raising the question of whether existing machine-unlearning algorithms can suppress this behavior. We introduce Level-based Evaluation of Narrative Suppression LENS , a contextualization based evaluation protocol for testing target narrative reproduction across direct, attributed, contrastive, and abstract resistance levels. We evaluate two source-grounded narratives: one framing Russia's war against Ukraine as forced by NATO expansion, and one framing the United States as exploiting or abandoning Taiwan. The experiments cover four near-12B multilingual instruction models: Lapa LLM, Gemma-12B, Qwen-14B, and TAIDE-Gemma. We introduce the Suppression-Collapse Efficiency SCE score as a checkpoint selection summary that rewards target-narrative suppression while penalizing degraded outputs. Our results shows that selected checkpoints can reduce narrative reproduction and suppression may transfer beyond direct forget prompts. We also report entity recovery as a separate side effect: abstract A/B/C prompts can cause models to recover the real-world actors associated with the target frame after unlearning. These findings demonstrate that LENS is a successful diagnostic protocol for both reporting and guiding the further study of the deeper structure of narrative unlearning.