FlexLAM: Resolving the Bottleneck Trade-off in Latent Action Learning

Researchers introduced FlexLAM, a latent action model that replaces fixed-capacity bottlenecks with variable-length codes trained via nested dropout, resolving the trade-off between discarding transition cues and preserving unnecessary variation. FlexLAM matches or surpasses fixed-capacity models across token budgets under scarce-label supervision and improves Ego4D transition reconstruction, offering an architecture-free upgrade for latent action models and video-pretrained interfaces.

arXiv:2606.19408v1 Announce Type: new Abstract: Latent actions provide a compact interface between action-free video and downstream decision-making, yet existing Latent Action Models LAMs force every transition through a fixed-capacity bottleneck. We identify a bottleneck trade-off: overly tight codes can discard transition cues needed for action alignment, while overly loose codes preserve additional transition variation that must be resolved when alignment labels are scarce or narrowly distributed. FlexLAM replaces this fixed capacity with variable-length latent actions trained by nested dropout, yielding prefix-valid codes that capture compact transition structure first and add detail only when needed, without new architectures or losses. A single FlexLAM matches or surpasses separately trained fixed-capacity LAMs at every evaluated token budget under standard scarce-label supervision and under a low-return single-task alignment stress test, indicating that FlexLAM is not merely adjustable at inference time but learns a better latent-action interface at the same token budgets. The same model supports inference-time token-budget adjustment without retraining, and FlexLAM improves Ego4D transition reconstruction. These results suggest that variable-length latent actions are an architecture-free, drop-in upgrade to the fixed-capacity bottleneck in latent action models, latent-action world models, and video-pretrained action interfaces.