{"slug": "quantization-robust-unlearning-through-the-lens-of-retain-forget-loss-landscapes", "title": "Quantization-Robust Unlearning through the Lens of Retain-Forget Loss Landscapes Interaction", "summary": "A new arXiv paper (2609.27355v1) proposes a quantization-robust unlearning framework that keeps forgetting intact after LLM compression by targeting sensitive weights identified through a curvature-based criterion. The method, sensitivity-guided noisy regularization, steers convergence toward smoother minima of uniformly low forget and retain losses, while forget-critical optimization updates only forget-critical layers to preserve retained knowledge. Experiments on the MUSE and TOFU benchmarks across multiple LLM unlearning algorithms show substantially more quantization-resilient forgetting while maintaining utility.", "body_md": "arXiv:2609.27355v1 Announce Type: new \nAbstract: Unlearning ensures LLM compliance by removing the influence of private or copyrighted training data. However, since LLM models typically undergo post-training compression, like quantization, in practical deployment, it has been observed that the unlearning effect can be substantially weakened, with the forgetting behavior degrading more severely than that of model utility. This paper proposes a quantization-robust unlearning framework that makes forgetting robust to quantization while maintaining overall model utility. We analyze this gap through the lens of loss landscape. Specifically, our analysis reveals a curvature-based criteria that pinpoints sensitive weights in the unlearned model that leads to both non-robust forgetting and reduced utility. We therefore propose sensitivity-guided noisy regularization, which is applied on the sensitive parameters to steer the model convergence towards a smoother minima of uniformly low forget and retain losses. Balancing unlearning and utility, we further propose forget-critical optimization, which updates only forget-critical layers, preserving most of the network to retain useful knowledge. Extensive experiments on the MUSE and TOFU benchmarks across multiple LLM unlearning algorithms show that our approach achieves substantially more quantization-resilient forgetting while maintaining utility.", "url": "https://wpnews.pro/news/quantization-robust-unlearning-through-the-lens-of-retain-forget-loss-landscapes", "canonical_source": "https://www.machinebrief.com/news/quantization-robust-unlearning-through-the-lens-of-retain-fo-8bvu", "published_at": "2026-09-24 04:00:00+00:00", "updated_at": "2026-09-24 06:00:04.878768+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "ai-research", "ai-safety"], "entities": ["arXiv", "MUSE", "TOFU"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/quantization-robust-unlearning-through-the-lens-of-retain-forget-loss-landscapes", "markdown": "https://wpnews.pro/news/quantization-robust-unlearning-through-the-lens-of-retain-forget-loss-landscapes.md", "text": "https://wpnews.pro/news/quantization-robust-unlearning-through-the-lens-of-retain-forget-loss-landscapes.txt", "jsonld": "https://wpnews.pro/news/quantization-robust-unlearning-through-the-lens-of-retain-forget-loss-landscapes.jsonld"}}