Mirror Learning

Researchers propose mirror learning, a framework that trains AI policies from third-person observation by using a fine-tuned video diffusion model to transform perspectives and an inverse dynamics model to infer actions, enabling synthesis of pseudo first-person data. The method, detailed in arXiv:2607.28737v1, shows that mirror data alone can train effective policies and that augmenting behavior cloning with mirror data improves performance, offering a scalable alternative to teleoperation-heavy data collection.

arXiv:2607.28737v1 Announce Type: new Abstract: We investigate imitation learning through the lens of third-person observation and propose a framework for mirror learning: acquiring actionable policies from passive observation. While behavior cloning BC excels under dense, well-aligned first-person data, it fundamentally fails to leverage the rich observational signals arising from third-person demonstrations that humans and animals routinely exploit. We introduce a method that composes i a learned perspective transformation that places learners in demonstrators' shoes using a fine-tuned video diffusion model and ii an inverse dynamics model that infers action trajectories in the learners' control space. This enables the synthesis of mirror data, pseudo first-person expert data generated from third-person observations of demonstrator behavior. Empirically, we show that mirror data alone can train effective policies, and that augmenting first-person BC training with mirror data further improves downstream policy performance. Our results suggest that modern generative world models implicitly encode sufficient structure to enable a scalable and safe alternative to teleoperation-heavy data collection.