{"slug": "ruler-representation-level-verification-of-machine-unlearning", "title": "RULER: Representation-Level Verification of Machine Unlearning", "summary": "Researchers introduced RULER, a set of representation-level verification metrics for machine unlearning, after finding that current output-level tests can pass even when models retain forgotten data internally. The oracle-comparative metric M2 detected significant residual information in 10 of 12 conditions across four approximate unlearning methods, while the oracle-free metric M4 identified identity-level memorization in face recognition models that no tested method could fully erase. The findings expose a critical gap in existing unlearning verification protocols, revealing that models can satisfy membership inference and accuracy benchmarks while still encoding forgotten records in their intermediate representations.", "body_md": "arXiv:2605.27569v1 Announce Type: new\nAbstract: Machine unlearning aims to remove the influence of specific training records from a deployed model without retraining from scratch. Current protocols verify this at the output level through membership inference, retain accuracy, and forget-set accuracy, but a model can satisfy all three whilst still encoding forgotten records in its intermediate representations. We introduce RULER, a set of representation-level verification metrics. The oracle-comparative metric M2 measures whether forget-set records occupy the same representational position as in a model retrained without them. The oracle-free metric M4 detects residuals from the unlearned model's internal similarity structure alone, without retraining. Four approximate unlearning methods all pass output-level evaluation, yet under a linear mixed-effects model M2 detects significant residuals in 10 of 12 conditions (p<0.05), with effect sizes growing as the forget fraction increases. A fifth method, Bad Teacher, shows the same residuals despite a different forgetting mechanism. M4 acts as a pre-unlearning diagnostic across tabular, image, clinical text, and face-identity settings: it detects identity-level memorisation in face recognition models where no tested method fully erases the signal.", "url": "https://wpnews.pro/news/ruler-representation-level-verification-of-machine-unlearning", "canonical_source": "https://arxiv.org/abs/2605.27569", "published_at": "2026-05-28 04:00:00+00:00", "updated_at": "2026-05-28 04:31:29.547163+00:00", "lang": "en", "topics": ["machine-learning", "ai-safety", "ai-research", "neural-networks", "ai-ethics"], "entities": ["RULER"], "alternates": {"html": "https://wpnews.pro/news/ruler-representation-level-verification-of-machine-unlearning", "markdown": "https://wpnews.pro/news/ruler-representation-level-verification-of-machine-unlearning.md", "text": "https://wpnews.pro/news/ruler-representation-level-verification-of-machine-unlearning.txt", "jsonld": "https://wpnews.pro/news/ruler-representation-level-verification-of-machine-unlearning.jsonld"}}