Future Representation Distillation for Imagination-free World Action Models

A research paper proposes Future Representation Distillation for Imagination-free World Action Models, a method that removes iterative video denoising from robot control pipelines that rely on pretrained video diffusion models. The authors argue that iterative video denoising is costly and that inaccurate visual predictions can compromise action prediction, and their approach distills the predictive knowledge of pretrained video diffusion models into future representations rather than generating future video frames. The work targets the cost and accuracy trade-offs facing World Action Models used for robot control.

World Action Models WAMs use the predictive knowledge of pretrained video diffusion models for robot control through future video prediction. However, iterative video denoising is costly, and inaccurate visual predictions can compromise action prediction. Our key idea is to make this predictive kn