Can LLMs Truly Forget? Revealing Unlearning Gaps Through Adversarial Evaluation

A new arXiv study (2608.21606v1) finds that large language models (LLMs) can still recover supposedly 'forgotten' information through adversarial prompting, with attack success rates (ASR) between 72.8% and 84.3% for fine-tuning-based unlearning methods on the TOFU benchmark using Llama-3.2-3B-Instruct, close to the 87.5% ASR of the unprotected base model. The researchers introduce ASR, an LLM-as-judge metric, and show that while clean-query forgetting scores exceed 0.91, adversarial robustness remains poor, highlighting the need for adversarial stress-testing in unlearning evaluation.

arXiv:2608.21606v1 Announce Type: new Abstract: Machine unlearning aims to remove the influence of targeted training data from a model while preserving its remaining capabilities, but evaluating whether such information has truly become inaccessible remains challenging. Existing benchmarks primarily assess unlearning under clean, non-adversarial queries, leaving open whether information that appears forgotten can still be recovered through strategic prompting. We address this gap through a unified evaluation of prompt-based and fine-tuning-based unlearning methods on TOFU using Llama-3.2-3B-Instruct, followed by an adversarial robustness evaluation of methods that perform strongly under standard metrics. We introduce Attack Success Rate ASR , an LLM-as-judge metric that measures the fraction of adversarial responses whose leakage score exceeds $0.2$, and evaluate recovery across eight attack suites. Our results reveal a substantial gap between clean-query forgetting and adversarial robustness. Although several fine-tuning-based methods achieve Forget Quality above $0.91$, targeted information remains recoverable with ASRs between $72.8\%$ and $84.3\%$, close to the $87.5\%$ ASR of the unprotected base model. In contrast, clean multilingual reformulations yield only $2.95\%$ measured leakage. A manual audit further finds agreement between binary ASR decisions and human factual assessments in seven of ten cases, indicating that ASR provides a useful, though imperfect, signal of behavioral recoverability. These findings show that strong standard-metric performance alone is insufficient to establish robustness after unlearning and motivate adversarial stress-testing as a complementary component of unlearning evaluation.