{"slug": "saver-2-reasoning-based-safety-alignment-for-large-reasoning-models-via-rewards", "title": "SAVeR$^2$: Reasoning-based Safety Alignment for Large Reasoning Models via Verifiable Rewards", "summary": "Researchers posted arXiv:2611.07482v1, introducing SAVeR$^2$, a safety alignment method for large reasoning models that decomposes the safety objective into four fine-grained verifiable reward signals and adds a reasoning-preserving gradient projection mechanism to resolve conflicts between safety and reasoning gradients. Experiments on models ranging from 1.5B to 671B parameters showed SAVeR$^2$ achieved superior safety performance and significantly lower over-refusal rates across multiple benchmarks without compromising general reasoning capabilities, according to the paper's abstract. The work targets what the authors call a \"safety tax\" on reasoning and poor adversarial generalization in existing alignment methods.", "body_md": "arXiv:2611.07482v1 Announce Type: new \nAbstract: Large Reasoning Models (LRMs) have achieved impressive performance via explicit Chain-of-Thought (CoT) reasoning, yet this process introduces critical safety risks. We formalize LRM safety objectives as requiring non-malicious reasoning and answers, safety consistency, and explicit intent identification. We argue that existing alignment methods suffer from poor adversarial generalization, a significant ``safety tax'' on reasoning, and unfaithful coarse-grained rewards, primarily due to their neglect of the LRM's intrinsic reasoning capabilities for safety. To address these, we propose \\textbf{SAVeR$^2$}, a novel \\textbf{S}afety \\textbf{A}lignment method via \\textbf{R}ewards with \\textbf{R}easoning capability. SAVeR$^2$ decomposes the safety objective into four fine-grained verifiable reward signals and utilizes a difficulty-aware data selection strategy to stabilize training. Crucially, we introduce a reasoning-preserving gradient projection mechanism that analytically resolves conflicts between safety and reasoning gradients, effectively eliminating the safety tax. Extensive experiments on models ranging from 1.5B to 671B demonstrate that SAVeR$^2$ achieves superior safety performance and significantly lower over-refusal rates across multiple benchmarks without compromising general reasoning capabilities.", "url": "https://wpnews.pro/news/saver-2-reasoning-based-safety-alignment-for-large-reasoning-models-via-rewards", "canonical_source": "https://arxiv.org/abs/2611.07482", "published_at": "2026-11-06 05:00:00+00:00", "updated_at": "2026-11-06 05:23:06.752304+00:00", "lang": "en", "topics": ["ai-safety", "large-language-models", "ai-research", "machine-learning"], "entities": ["SAVeR$^2$", "arXiv", "Large Reasoning Models"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/saver-2-reasoning-based-safety-alignment-for-large-reasoning-models-via-rewards", "markdown": "https://wpnews.pro/news/saver-2-reasoning-based-safety-alignment-for-large-reasoning-models-via-rewards.md", "text": "https://wpnews.pro/news/saver-2-reasoning-based-safety-alignment-for-large-reasoning-models-via-rewards.txt", "jsonld": "https://wpnews.pro/news/saver-2-reasoning-based-safety-alignment-for-large-reasoning-models-via-rewards.jsonld"}}