{"slug": "depth-dependent-safety-geometry-in-vision-language-models-layer-wise-and-refusal", "title": "Depth-Dependent Safety Geometry in Vision-Language Models: Layer-Wise Localization and Bidirectional Refusal Control", "summary": "A lightweight inference-time framework can localize and manipulate safety-sensitive representations in frozen vision-language models without retraining, according to arXiv paper 2610.32182v1. On Qwen3-VL-4B, activation steering along a learned safety direction in early vision layers produced bidirectional refusal control with unsafe rejection spanning 6% to 96%, while deeper layers stayed separable along the safety direction but were largely insensitive to steering; InternVL2.5-4B showed qualitatively similar depth-dependent geometry. The authors say the results expose a white-box, representation-level refusal-bypass attack surface in VLM vision encoders and motivate inference-time stabilization of safety-sensitive representations.", "body_md": "arXiv:2610.32182v1 Announce Type: cross \nAbstract: Vision-Language Models (VLMs) are widely deployed, yet the internal mechanisms behind their safety behavior remain poorly understood. We present a lightweight inference-time framework for localizing and manipulating safety-sensitive representations in frozen VLMs, without retraining or modifying parameters. On Qwen3-VL-4B, layer-wise activation divergence analysis reveals non-uniform safety geometry across depth. Causal interventions show that early vision layers support strong bidirectional refusal control through activation steering along a learned safety direction, with unsafe rejection spanning 6%-96%, whereas later layers respond more to activation magnitude scaling and exhibit a substantial safety-selectivity trade-off. Deeper layers remain well separated along the safety direction yet are largely insensitive to steering, distinguishing representational separability from causal controllability. InternVL2.5-4B shows qualitatively similar depth-dependent activation geometry despite differences in layer location and magnitude. Our results expose a white-box, representation-level refusal-bypass attack surface in VLM vision encoders and motivate inference-time stabilization of safety-sensitive representations.", "url": "https://wpnews.pro/news/depth-dependent-safety-geometry-in-vision-language-models-layer-wise-and-refusal", "canonical_source": "https://www.machinebrief.com/news/depth-dependent-safety-geometry-in-vision-language-models-la-ev81", "published_at": "2026-10-26 04:00:00+00:00", "updated_at": "2026-10-26 06:41:38.438485+00:00", "lang": "en", "topics": ["ai-safety", "computer-vision", "large-language-models", "ai-research", "machine-learning"], "entities": ["Qwen3-VL-4B", "InternVL2.5-4B", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/depth-dependent-safety-geometry-in-vision-language-models-layer-wise-and-refusal", "markdown": "https://wpnews.pro/news/depth-dependent-safety-geometry-in-vision-language-models-layer-wise-and-refusal.md", "text": "https://wpnews.pro/news/depth-dependent-safety-geometry-in-vision-language-models-layer-wise-and-refusal.txt", "jsonld": "https://wpnews.pro/news/depth-dependent-safety-geometry-in-vision-language-models-layer-wise-and-refusal.jsonld"}}