ExRL: Sample-Efficient Online Adaptation of Long-Chunk Policies via Execution Length Control

A paper titled "ExRL: Sample-Efficient Online Adaptation of Long-Chunk Policies via Execution Length Control" reports that adapting long-chunk robot policies — which predict action chunks spanning dozens of timesteps — with existing reinforcement learning methods can require many online samples because of the high dimensionality involved. The paper proposes execution length control as a method for sample-efficient online adaptation of these policies. The work bears on how robot policies that predict long action horizons can be fine-tuned online with fewer samples.

Predicting long action chunks, often spanning dozens of timesteps, has become a key ingredient in increasingly capable robot policies. In this paper, we find that adapting these long-chunk policies with existing reinforcement learning RL methods can require many online samples due to the high dime