Mitigating LLM Over-Refusal via Dynamic Semantic Routing Calibratione

A new arXiv paper (arXiv:2609.25049v1) proposes Semantic Routing Calibration (SRC), a lightweight, training-free inference framework that reduces over-refusal in safety-aligned large language models by localizing and dynamically suppressing a sparse subset of "Hypersensitive Safety Heads" in transformer attention. The authors trace over-refusal to high-entropy routing conflicts in which these heads bind harmless target entities to refusal semantics, and pair SRC with a dual-branch logits fusion that acts as a safety regularizer during decoding. Experiments show SRC alleviates over-refusal while preserving intrinsic safety performance as much as feasible.

arXiv:2609.25049v1 Announce Type: new Abstract: Large language models LLMs aligned for safety often suffer from over-refusal, incorrectly rejecting benign yet safety-related instructions. Prior studies primarily attribute this to static representation overlap, largely overlooking the underlying dynamic mechanisms. In this paper, we present the mechanistic analysis of over-refusal through the lens of internal routing conflicts within transformer attention. We discover that a sparse subset of Hypersensitive Safety Heads misfires on Hard-Safe prompts, exhibiting abnormal attention entanglement that forcefully binds harmless target entities to refusal semantics. This triggers a severe, high-entropy routing conflict that deprives target entities of necessary attention. To counteract this, we propose Semantic Routing Calibration SRC , a lightweight, training-free inference framework. SRC precisely localizes and dynamically suppresses these hypersensitive safety heads at the inference stage. Coupled with a dual-branch logits fusion that acts as a safety regularizer during subsequent decoding, SRC seamlessly restores trustworthy reasoning. Extensive experiments demonstrate that SRC alleviates over-refusal, with intrinsic safety performance preserved as much as feasible.