Welcome! Short version, based on recent surveys and papers:
1. Mainstream
- Chunked policies trained by imitation: ACT-style transformers and Diffusion Policy.
- VLAs: a pretrained VLM backbone plus an action head. Two families: autoregressive action tokens (RT-2, OpenVLA) and diffusion/flow-matching heads (π0, π0.5, GR00T, SmolVLA). Dual-system designs (a slow VLM planner plus a fast low-level policy) are common too. See this survey .
2. Limitations
-
Data: VLAs need a lot of demos, and performance drops under distribution shift.
-
Physical properties: vision doesn’t show mass, friction or stiffness, so a vision-only VLA isblind to contact forces .
-
Latency: big models are slow, so actions can be computed from stale observations.
-
Evaluation: benchmarks like LIBERO are near saturated (>98%), so they say little about real generalization (survey ). 3. Emerging directions (matching your interests)
-
Tactile/force VLAs for weight, friction and stiffness:Tactile-VLA ,TacVLA , andHapticVLA , which distills force info so no force sensor is needed at inference.
-
Real-time execution: Real-Time Chunking (asynchronous chunk generation, works with flow-based VLAs without retraining), and RL fine-tuning for real-time control (Real-Time EXPO-FT ).
-
RL post-training on top of pretrained VLAs for reliability and data efficiency.
Tip: you also posted this in Research, so keeping one thread will help you get answers in one place.