{"slug": "terratransfer-learning-end-to-end-driving-policies-without-expert-demonstrations", "title": "TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrations", "summary": "Researchers at TerraTransfer propose a method for end-to-end autonomous driving that eliminates the need for expert demonstrations by decoupling learning to drive from learning to see. The approach uses self-play in vectorized simulators to pretrain a driving policy, then aligns its latent space with a pretrained vision backbone using action KL divergence and a structural loss. The resulting policy matches or exceeds prior end-to-end methods on photorealistic closed-loop scenarios.", "body_md": "arXiv:2606.17386v1 Announce Type: new\nAbstract: End-to-end autonomous driving has achieved state-of-the-art performance on benchmarks and real-world deployments. Its standard training recipe, however, is expensive across all stages: collecting and labeling millions of driving frames is costly, and closed-loop RL on images is bottlenecked by the per-step cost of photorealistic rendering plus a forward pass through a large vision backbone. Self-play in vectorized simulators changes the economics: millions of rollout steps per second, and a state distribution naturally rich in collisions, near-misses, and recoveries that no driving log contains. Our approach exploits this asymmetry by decoupling learning to drive from learning to see. We pretrain a single policy by self-play, then align its latent space with a pretrained vision backbone, through the action KL divergence and a batch-relational low-rank structural loss. The action target comes from the self-play policy, so alignment never supervises against a logged trajectory: a paired dataset of (image, scene-state) frames suffices, with no need for the curated expert demonstrations that imitation pretraining is built on. On photorealistic 3D Gaussian splatting closed-loop scenarios, the resulting end-to-end policy matches or exceeds prior end-to-end methods.", "url": "https://wpnews.pro/news/terratransfer-learning-end-to-end-driving-policies-without-expert-demonstrations", "canonical_source": "https://arxiv.org/abs/2606.17386", "published_at": "2026-06-17 04:00:00+00:00", "updated_at": "2026-06-17 04:26:17.831107+00:00", "lang": "en", "topics": ["autonomous-vehicles", "machine-learning", "computer-vision", "ai-research"], "entities": ["TerraTransfer"], "alternates": {"html": "https://wpnews.pro/news/terratransfer-learning-end-to-end-driving-policies-without-expert-demonstrations", "markdown": "https://wpnews.pro/news/terratransfer-learning-end-to-end-driving-policies-without-expert-demonstrations.md", "text": "https://wpnews.pro/news/terratransfer-learning-end-to-end-driving-policies-without-expert-demonstrations.txt", "jsonld": "https://wpnews.pro/news/terratransfer-learning-end-to-end-driving-policies-without-expert-demonstrations.jsonld"}}