# Does routine compression undo LLM unlearning? A short project > Source: > Published: 2026-07-20 22:11:21+00:00 *I completed this project over 2 weeks as part of a **BlueDot Project** cohort. It was my first solo project and I learned a lot! Feedback is super welcome :)* **Code and full results: ****GitHub** **TLDR:** I tested if standard post-training compression processes (quantization, magnitude pruning, SVD truncation) reverses unlearning on TOFU `forget10` / `Llama-3.2-1B-Instruct` , across a few unlearning methods (NPO, SimNPO, IdkDPO). I found that unlearning reversal is minimal across all combinations; however, the strongest reversal came from magnitude pruning in a narrow 10-20% sparsity window on NPO unlearning (up to 42% of the ceiling–baseline gap recovered). Once a model is released as open-weight, it becomes easy to finetune away any refusal training that has gone into it. So, unlearning methods, which actually remove knowledge from the weights, become especially important for open-weight models. However, it is unclear if many unlearning methods are robust or can be reversed easily. There is already prior [evidence](https://arxiv.org/abs/2410.16454) that quantization specifically can reverse some unlearning. So in this project, I expand on this to investigate if reversal can come from standard, non-adversarial compression operations, which are routine processes that most models undergo on the way to deployment, including quantization and others. `forget10` , `Llama-3.2-1B-Instruct` .`forget_Q_A_Prob` (teacher-forced prob of the correct forget-set answer).`model_utility` — any cell where this collapsed (≤ 0.01) is discarded as uninterpretable.`tofu_..._full` (model trained on all data, no unlearning) = 0.881`tofu_..._retain90` (model never trained on the forget set, no unlearning) = 0.116. `recovery = (compressed − baseline)/(ceiling − baseline)` . Essentially measures how much closer the model moves from compressed to ceiling from the baseline, in terms of the probability mass it places on the correct answer tokens.`forget_Q_A_Prob` ): NPO 0.209, SimNPO 0.075, IdkDPO 0.017 — all at/below the floor, confirming effective unlearning.**Compression processes** **Qualitative inspection shows that even though there is recovery on teacher-forced answer probabilities (what **`forget_Q_A_Prob` ** measures), it is not enough to resurface the unlearned answers in free generation. **When observing model answers of the NPO baseline vs. the NPO 4-bit quantization and vs. the NPO 20%-pruned on 20 forget-set questions, I found that still neither of these reproduces the exact memorized TOFU facts that were unlearned (names, book titles, awards). **SimNPO and IdkDPO are more robust than NPO.** Unlike NPO, these unlearning methods had ≤3% recovery in all non-destructive cells. However, NPO did also have a higher baseline, which gave it more distance to recover. Especially given the limited time I spent on this, and my own lack of experience with research projects, there are some limitations that I'm aware of. Future works could be devoted to filling in these gaps: `forget10` forget-set questions.`Llama-3.2-1B-Instruct` ), 1 TOFU split (`forget10` ), and 3 unlearning methods.There are no doubt many limitations that I'm not aware of too. Super open to any and all feedback! The main technical takeaway is that standard compression isn't a practical attack here. No type restores knowledge to the ceiling; the strongest recovery case was only still 42%. However, the pruning peak result shows the effect can be larger than the quantization result that originally motivated this project. From another perspective, I didn't do this project just to answer the research question. I wanted to learn, by myself, more about what it is like to be a researcher. Some key takeaways on this front over the past few weeks: I'm really hoping to get deeper into the field of AI safety, and my next goal is to get involved in university research. Now, I feel a lot more prepared to try to chase that!