From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention

A new reinforcement learning approach targets long-horizon robot manipulation by training on individual failed subtasks rather than collecting full-task demonstrations, according to the paper "From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention." The method addresses the inefficiency of supervised fine-tuning, which requires operators to repeat behaviors a pretrained robot foundation policy already performs well. The work aims to let a pretrained policy overcome its few critical subtask failures with minimal human intervention.

A pretrained robot foundation policy may execute most of a long-horizon task yet repeatedly fail at a few critical subtasks. Collecting additional full-task demonstrations for supervised fine-tuning SFT requires operators to repeat behaviors the policy already performs well. Reinforcement learning