Robust Scene Transfer for PointGoal Navigation via Privileged Sensor Guided Contrastive Learning

Researchers developed a sensor-guided adaptive contrastive learning framework that uses privileged LiDAR data during training to teach visual encoders to focus on navigation-relevant geometry rather than scene-specific appearance for PointGoal navigation. The method decouples representation learning from policy optimization and introduces cross-stage domain mismatch to suppress environment-specific shortcuts, enabling agents to transfer navigation policies across diverse indoor and outdoor environments using only monocular RGB observations at deployment. The approach outperforms large pretrained vision models and standard contrastive baselines under severe appearance and semantic shifts, with the code and a multimodal dataset released to support further research.

arXiv:2606.05506v1 Announce Type: new Abstract: We propose a sensor-guided adaptive contrastive learning framework for visual representation learning in PointGoal navigation. During training, privileged LiDAR sensing guides the contrastive objective through a geometry-aware similarity metric and adaptive temperature scaling, encouraging visual embeddings to capture navigation-relevant structure rather than scene-specific appearance. The resulting encoder is pretrained independently, frozen, and used as the perceptual backbone for reinforcement learning, decoupling representation learning from policy optimization. We further introduce a cross-stage domain mismatch between representation pretraining and policy learning to suppress environment-specific shortcuts and promote reliance on task-relevant features. Extensive experiments in high-fidelity simulation demonstrate that our approach significantly improves policy-level scene transfer across diverse indoor and outdoor environments. At deployment, the agent relies only on monocular RGB observations together with standard task-related inputs such as goal position and proprioceptive signals, without access to LiDAR or other privileged sensors. Our method outperforms large pretrained vision models and standard contrastive baselines under severe appearance and semantic shifts. We also release a multimodal dataset to support future research on privileged-guided visual representation learning for navigation. The code is available at: