Inference-Time Attention Steering for Vision-Language-Action Driving Models

A new study from arXiv (2608.17095v1) introduces inference-time attention steering for vision-language-action (VLA) driving models, applying a bounded additive pre-softmax attention bias to visual tokens of detector-localized traffic actors on Alpamayo-R1's Qwen3-VL backbone. In 50 lane-change scenarios from the Physical AI World Model Synthetic dataset, the trajectory decoder showed a monotonic dose response, reaching approximately 17 cm mean displacement with lateral shifts up to about 140 cm at the clamp. The method requires no retraining and works as a fail-open forward pre-hook, but the bias does not affect the reasoning pathway, so the invariance is verified exposure, not robustness.

arXiv:2608.17095v1 Announce Type: new Abstract: Vision-language-action VLA driving models couple a reasoning stage with a diffusion-based trajectory decoder, but do not give a direct way to redirect attention toward safety-critical actors at inference time without retraining. We studied a bounded additive pre-softmax attention bias on the visual tokens of detector localized traffic actors on Alpamayo-R1's Qwen3-VL backbone. It is applied as a fail open forward pre-hook with no weight changes. On 50 lane-change scenarios from the Physical AI World Model Synthetic dataset. The trajectory decoder shows a monotonic dose response in the bias magnitude, separate from a paired zero bias control at every tested magnitude. It reaches $\approx 17$\,cm mean displacement with lateral shifts up to $\sim 140$\ cm at the clamp. A layer ablation places the action-relevant signal in late layers, where the effect increases with the number of hooked layers 2.0cm for the first 8 layers; 67.6cm for all 36 . A per call injection audit explains why the Chain-of-Causation text never changes. The mask based bias never reaches the reasoning pathway in this serving stack, so the invariance is verified exposure, not robustness. Steered trajectories tend to shift toward the attended actor, suggesting the bias governs where the model looks rather than encoding a target behavior.