Measuring the Depth of LLM Unlearning via Activation Patching

Researchers from Sungkyunkwan University, Samsung Research, and KAIST presented the Unlearning Depth Score (UDS), a mechanistic metric that quantifies how deeply a model has unlearned target knowledge by measuring how much of it remains recoverable through two-stage activation patching, at EMNLP 2026 as an Oral presentation. The team benchmarked UDS against 20 unlearning metrics on the TOFU forget10 dataset using Llama-3.2-1B-Instruct and the Open-Unlearning framework, evaluating faithfulness via AUC-ROC over 30 knowledge-present and 30 knowledge-absent models and robustness under 4-bit NF4 quantization and 1-epoch relearning. UDS was also evaluated across 152 models spanning 8 unlearning methods, where it is combined with membership inference attacks as Privacy = HM(MIA, UDS).

<sup 1</sup Sungkyunkwan University <sup 2</sup Samsung Research <sup 3</sup KAIST EMNLP 2026 Oral We present the Unlearning Depth Score UDS , a mechanistic metric that quantifies the depth of unlearning by measuring how much target knowledge is recoverable through two-stage activation patching. This page provides: i an interactive walkthrough of the UDS pipeline , ii a meta-evaluation comparing 20 metrics on faithfulness and robustness , and iii per-method benchmark results across 150 unlearned models . The results below benchmark UDS on the TOFU \ 12\ ref12 dataset forget10 using Llama-3.2-1B-Instruct and the Open-Unlearning \ 18\ ref18 framework. Construct a prompt from question and answer prefix, and mark the entity span to evaluate. Patch retain model's hidden states into the full model at each layer. Log-probs are compared token-by-token under teacher forcing. Large drops reveal layers encoding knowledge the retain model lacks. Repeat with the unlearned model's hidden states. Drops matching Stage 1 = erased; near-zero = knowledge intact. ① Filter layers that significantly encode the target knowledge. ② Compute per-layer erasure ratio from the two stages. ③ Aggregate into a final score: 0 intact → 1 erased . We evaluate 20 metrics to measure how reliable each unlearning metric is , using the Open-Unlearning benchmark framework. Faithfulness : How well each metric separates knowledge-present P, 30 models vs knowledge-absent N, 30 models , measured by AUC-ROC. Robustness : Stability of each metric under post-hoc perturbations 4-bit quantization, 1-epoch relearning . Faithfulness = AUC-ROC over P/N pools 30 knowledge-present vs 30 knowledge-absent models Robustness = HM Q, R — Quantization Q $= 1 - \dfrac{|m {\text{after}} - m {\text{before}}|}{|m {\text{before}}| + |m {\text{after}}|}$ — penalizes both recovery and destruction after 4-bit NF4 quantization Relearning R $= 1 - \dfrac{|\Delta {\text{unl}} - \Delta {\text{ret}}|}{|\Delta {\text{unl}}| + |\Delta {\text{ret}}|}$, $\Delta = m {\text{after}} - m {\text{before}}$ — penalizes both over- and under-recovery relative to retain Overall = HM Faithfulness, Robustness ES EM Prob — geometric mean of per-token probabilities. $\exp\ \bigl - 1/T \sum \text{CE} t\bigr $ ParaProb — geometric mean of Prob across paraphrased answer variants. Truth Ratio — normalized correct vs. incorrect probability. $p c / p c + p w $, $p = \exp -\text{avg loss} $ ROUGE — standard prompt. Para. ROUGE — paraphrased prompt. Jailbreak ROUGE — adversarial prompt prefixed with “Sure, here is the answer:” . MIA-LOSS MIA-ZLib MIA-Min-K MIA-Min-K++ s <sub LOSS</sub · s <sub ZLib</sub · s <sub Min-K</sub · s <sub Min-K++</sub CKA Logit Lens Fisher Masked UDS Ours — Measures whether knowledge remains recoverable via activation patching. | Metric | Overall ↑ | Faithfulness ↑ | Robustness | | | |---|---|---|---|---|---| | | | | Aggregate ↑ | Quantization ↑ | Relearning ↑ | | Loading... | | | | | | We evaluate 152 models 8 methods × varying hyperparameters × 2 epochs + full + retain across three axes. All unlearned model checkpoints are from the Open-Unlearning https://github.com/locuslab/open-unlearning framework. Memorization : How much target knowledge was forgotten. ↑ higher = more forgotten Privacy : How well sensitive information from the forget set is protected from being extracted. ↑ higher = better protected Utility : How well the model retains general capabilities on non-target knowledge. ↑ higher = better retention Overall : Harmonic mean of all three axes. ↑ higher = better Privacy = HM MIA, UDS , capturing both statistical MIA and mechanistic UDS aspects. Mem. $= \text{HM} 1{-}\text{ES},\; 1{-}\text{EM},\; 1{-}\text{ParaProb},\; 1{-}\text{TruthRatio} $ MIA $= \text{HM} s {\text{LOSS}},\; s {\text{ZLib}},\; s {\text{Min-K}},\; s {\text{Min-K++}} $ Privacy = HM MIA, UDS ModelUtility MU = HM retain Prob, retain ROUGE, retain TruthRatio, ra Prob, ra ROUGE, ra TruthRatio, wf Prob, wf ROUGE, wf TruthRatio Fluency = generation fluency score Utility = HM MU, Fluency , then normalized: $\text{Utility} {\text{rel}} = \text{Utility}\, /\, \text{Utility} {\text{full epoch }}$ | Method | Learning Rate | Swept Hyperparameters | Fixed | Epochs | Models | |---|---|---|---|---|---| | GradDiff \ 12\ ref12 , IdkNLL \ 12\ ref12 , IdkDPO \ 12\ ref12 , NPO \ 13\ ref13 , AltPO \ 14\ ref14 | {1e-5, 2e-5, 5e-5} | $\alpha$ ∈ {1, 2, 5} | β = 0.1 | {5, 10} | 5 × 3 × 3 × 2 = 90 | | SimNPO \ 15\ ref15 | {1e-5, 2e-5, 5e-5} | $\beta$ ∈ {3.5, 4.5}, $\gamma$ ∈ {0.125, 0.25} | δ = 1, α = 1 | {5, 10} | 1 × 3 × 4 × 2 = 24 | | RMU \ 16\ ref16 | {1e-5, 2e-5, 5e-5} | layer ∈ {5, 10, 15} | steering coeff = 10 | {5, 10} | 1 × 3 × 3 × 2 = 18 | | UNDIAL \ 17\ ref17 | {1e-5, 1e-4, 3e-4} | $\alpha$ ∈ {1, 2, 5} | β = 10 | {5, 10} | 1 × 3 × 3 × 2 = 18 | | Total: 150 unlearned + full + retain | | | | | 152 | | Model | Overall <sub w/o UDS</sub ↑ | Overall <sub w/ UDS</sub ↑ | Mem. ↑ | Privacy ↑ | Utility ↑ | LL ↑ | UDS ↑ | | |---|---|---|---|---|---|---|---|---| | Loading... | | | | | | | | |