{"slug": "show-hn-vlda-learn-compact-deltas-as-actionable-transition-interfaces-for-robot", "title": "Show HN: VLDA-learn compact Deltas as actionable transition interfaces for robot", "summary": "Researchers from Harbin Institute of Technology Shenzhen, Baidu and the Shenzhen Loop Area Institute released LoongForge-VLDA, a vision-language-delta-action model that distills only the state changes between frames into a compact Delta fed to the action head, reporting a 98.5% average on LIBERO versus 96.9% for π0.5 and 84.5% zero-shot on LIBERO-Plus. VLDA also scored 62.9% on 10 hard RoboTwin 2.0 tasks and 73.0% on the R1 Lite real robot and 72.0% on AgileX, with real-world inference running on a single GPU. The method uses a V-JEPA 2 EMA teacher, a local transition filter selecting the top-64 regions via Sinkhorn optimal transport, and a global transition anchor, trained on 8 GPUs with the LoongForge stack and requiring no future frames at inference.", "body_md": "# LoongForge-VLDA\n\n**Vision-Language-Delta-Action:** learning compact Deltas as\n        actionable transition interfaces for robotic manipulation.\n\n<sup>1</sup> Harbin Institute of Technology, Shenzhen  · \n        <sup>2</sup> Baidu, Inc.  · \n        <sup>3</sup> Shenzhen Loop Area Institute\n\n* Equal contribution † Corresponding author\n\n**98.5%** LIBERO average\n\n**84.5%** LIBERO-Plus, zero-shot\n\n**62.9%** RoboTwin 2.0, 10 hard tasks\n\n**73.0%** Real robot · R1 Lite\n\n**72.0%** Real robot · AgileX\n\n**1 GPU** Real-world inference on a single GPU\n\n**01** The idea\n\n## Don't predict the whole future. *Capture the change.*\n\n**What does an action actually change?** Manipulation is a chain of state\n            transitions, and most of every frame stays the same. VLA models must infer that change implicitly; world-action\n            models predict whole future frames and spend capacity on pixels that never move.\n\nVLDA distills only **what changes** into a compact Delta and hands it to the\n            action head as an explicit interface: understand the scene, express the transition, then execute.\n\n**Understand** Vision and language are encoded into a task-aware representation.\n\n**Transition · Δ** A compact Delta captures the local and global change the task requires.\n\n**Execute** The action head is refined by the Delta, with no future frames needed at inference.\n\n**02** The method\n\n## From a Delta to an action, *in three moves.*\n\nA V-JEPA 2 EMA teacher provides the target transition. Two filters distill it into a Delta; one refinement module feeds it back into the action head.\n\n### Local Transition Filter\n\nSelects the top-64 regions with the largest transition magnitude and matches them to the predicted Delta by Sinkhorn optimal transport, so the Delta is not diluted by static background.\n\n### Global Transition Anchor\n\nAnchors the local patches to a global summary of the transition, keeping the Delta consistent with the task as a whole.\n\n### Delta-Driven Action Refinement\n\n**DAMG** grounds the Delta on the\n            action manifold with an InfoNCE loss. **DSAF** seeds the action flow with a Delta-based noise bias and a\n            prefix token.\n\n**Learn the transition**\n\nPre-train the Delta from the teacher, without actions.\n\n`L1 = L_LTF + L_GTA`\n**Learn to act**\n\nTrain the action head with the Delta as an interface, keeping the stage-1 loss as a regularizer.\n\n`L2 = L_act + λ·L1 + λ_DDAR·L_DDAR`\n**Built on LoongForge.** VLDA is trained with the LoongForge embodied\n          stack: DDP, ZeRO-1, FSDP and HSDP, component-wise freezing and checkpointing. Training used 8 GPUs.\n          Inference needs no future observations, and the EMA teacher can be dropped after training.\n\n**03** The results\n\n## Stronger where it counts, *consistently.*\n\nThree simulation benchmarks and a set of ablations. Our row is highlighted.\n\n**98.5%** LIBERO average\n\n### Top average, with the biggest lift on long-horizon tasks.\n\nVLDA beats π0.5 by 1.6 points on average and edges out every other baseline, reaching 95.2% on the Long suite.\n\nSuccess rate (%) on LIBERO\n\n| Method | Spatial | Object | Goal | Long | Avg | \n|---|---|---|---|---|---|\n| VLDA | 99.8 | 99.6 | 99.4 | 95.2 | 98.5 | \n| π0.5 | 98.8 | 98.2 | 98.0 | 92.4 | 96.9 | \n| VLA-Adapter | 97.8 | 99.2 | 97.2 | 95.0 | 97.3 | \n| OpenVLA-OFT | 97.6 | 98.4 | 97.9 | 94.5 | 97.1 | \n| CronusVLA | 97.3 | 99.6 | 96.9 | 94.0 | 97.0 | \n| InternVLA-M1 | 98.0 | 99.0 | 93.8 | 92.6 | 95.9 | \n| UniVLA | 95.4 | 98.8 | 93.6 | 94.0 | 95.4 | \n| π0 | 96.8 | 98.8 | 95.8 | 85.2 | 94.2 | \n| GR00T-N1 | 94.4 | 97.6 | 93.0 | 90.6 | 93.9 | \n\n**+10.9** points over π0.5, zero-shot\n\n### Robust when the world is perturbed.\n\nAhead of π0.5 on all seven perturbation dimensions, and ahead of the strongest baseline, GuidedVLA, on six of them (+9.1 on average). The gap widens on the Long suite: +13.4 over π0.5 and +12.1 over GuidedVLA.\n\nZero-shot success rate (%) on LIBERO-Plus\n\n| Method | Camera | Robot | Language | Light | Background | Noise | Layout | Avg | \n|---|---|---|---|---|---|---|---|---|\n| VLDA | 72.4 | 75.3 | 84.0 | 96.0 | 95.5 | 89.4 | 85.7 | 84.5 | \n| GuidedVLA | 73.7 | 51.4 | 62.6 | 94.6 | 89.0 | 85.2 | 79.9 | 75.4 | \n| π0.5 | 59.7 | 65.5 | 75.3 | 87.0 | 82.4 | 72.1 | 80.3 | 73.6 | \n| DreamVLA | 65.0 | 40.8 | 63.5 | 85.7 | 82.6 | 84.9 | 74.0 | 69.9 | \n| OpenVLA-OFT | 56.4 | 31.9 | 79.5 | 88.7 | 93.3 | 75.8 | 74.2 | 69.6 | \n| π0 | 13.8 | 6.0 | 58.8 | 85.0 | 81.4 | 79.0 | 68.9 | 53.6 | \n\n**+20.2** points over π0.5, 10 hard tasks\n\n### Biggest gains on complex state-transition tasks.\n\nBest or tied-best on every selected task. Put Bottles Dustbin jumps +46, Stacking Three +30, Place Dual Shoes +27 and Open Microwave +27. Across all 50 tasks VLDA scores 50.86% against 43.84% for π0.5.\n\nSuccess rate (%) on RoboTwin 2.0\n\n| Method | Ranking RGB | Ranking Size | Grab Roller | Open Laptop | Open Microwave | Place Dual Shoes | Put Bottles Dustbin | Shake Bottle | Stacking Three | Stacking Two | Avg | \n|---|---|---|---|---|---|---|---|---|---|---|---|\n| VLDA | 53 | 16 | 96 | 60 | 64 | 34 | 55 | 100 | 65 | 86 | 62.9 | \n| π0.5 | 35 | 14 | 89 | 35 | 37 | 7 | 9 | 100 | 35 | 66 | 42.7 | \n| π0 | 5 | 1 | 80 | 46 | 50 | 0 | 13 | 60 | 24 | 41 | 32.0 | \n| RDT | 0 | 0 | 43 | 32 | 20 | 4 | 4 | 45 | 17 | 30 | 19.5 | \n\n**+24** points on Basket Relocation\n\n### Every module earns its place.\n\nRemoving any single piece costs accuracy. LT-Filter and GT-Anchor each help alone, and together they help most, notably on the real robot.\n\nAblations (LIBERO average unless noted)\n\n~~98.5~~\n\n**97.2**\n\n~~98.5~~\n\n**97.3**\n\n~~98.5~~\n\n**97.8**\n\n~~84.5~~\n\n**82.0**\n\n### LT-Filter and GT-Anchor\n\n| LT-Filter | GT-Anchor | LIBERO Spatial | Object | Goal | Long | Ring | Sweep | Sort | Basket | \n|---|---|---|---|---|---|---|---|---|---|\n| ✗ | ✗ | 98.6 | 98.4 | 98.0 | 92.8 | 76 | 52 | 48 | 48 | \n| ✓ | ✗ | 99.4 | 99.4 | 99.0 | 94.4 | 84 | 60 | 56 | 68 | \n| ✗ | ✓ | 98.8 | 99.0 | 98.8 | 93.8 | 80 | 60 | 52 | 60 | \n| ✓ | ✓ | 99.8 | 99.6 | 99.4 | 95.2 | 92 | 68 | 60 | 72 | \n\nSimulation columns are LIBERO success rate (%); real-world columns are Ring Stacking, Table Sweeping, Conveyor Sorting and Basket Relocation on R1 Lite. Each module helps alone; together they help most.\n\n**04** In the real world\n\n## Still ahead on *real robots.*\n\nFour long-horizon tasks on a GALAXEA R1 Lite, 25 trials each, plus a second platform, AgileX Cobot Magic. Numbers are final-stage success rates (%).\n\n#### Ring Stacking\n\n**92%** vs. π0.5 80%\n\n#### Table Sweeping\n\n**68%** vs. π0.5 52%\n\n#### Conveyor Sorting\n\n**60%** vs. π0.5 44%\n\n#### Basket Relocation\n\n**72%** vs. π0.5 48%\n\n**+17.0** points over π0.5, R1 Lite\n\nAveraged over four tasks, VLDA reaches 73.0% against 56.0% for π0.5. On a second robot the lead holds. Basket Relocation uses an out-of-distribution mat position, and VLDA still reaches 72% at the final placement.\n\n| Platform (average) | OpenVLA-OFT | π0.5 | VLDA | \n|---|---|---|---|\n| GALAXEA R1 Lite | 16.0 | 56.0 | 73.0 | \n| AgileX Cobot Magic | 17.0 | 54.0 | 72.0 | \n| Δ vs. π0.5 (pp) |  |  | +17.0 / +18.0 | \n\n### What the Delta looks at\n\nThe Delta concentrates on the regions that move and interact, not on the static scene.\n\n**05** Build with VLDA\n\n## Code and *citation.*\n\n[GitHub ↗](https://github.com/baidu-baige/LoongForge/tree/master/cookbook/loongforge-vlda)\n\n### LoongForge-VLDA Cookbook\n\nTraining code, configs and recipes for VLDA live in LoongForge.\n\n```\ngit clone https://github.com/baidu-baige/LoongForge.git\n```\n\n### Citation\n\n```\n@misc{zhang2026visionlanguagedeltaactionlearningcompactdeltas,\n  title={Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation},\n  author={Lingsen Zhang and Wei Li and Jie He and Henghua Zhang and Kai Mo and Shiyong Li and Yanpeng Wang and Rui Shao},\n  year={2026},\n  eprint={2610.25874},\n  archivePrefix={arXiv},\n  primaryClass={cs.RO},\n  url={https://arxiv.org/abs/2610.25874},\n}\n```\n\nRead the preprint on [arXiv ↗](https://arxiv.org/abs/2610.25874).", "url": "https://wpnews.pro/news/show-hn-vlda-learn-compact-deltas-as-actionable-transition-interfaces-for-robot", "canonical_source": "https://baidu-baige.github.io/LoongForge/projects/vlda.html", "published_at": "2026-10-22 12:30:11+00:00", "updated_at": "2026-10-22 12:40:22.361314+00:00", "lang": "en", "topics": ["robotics", "computer-vision", "ai-research", "machine-learning", "artificial-intelligence"], "entities": ["LoongForge-VLDA", "Harbin Institute of Technology, Shenzhen", "Baidu", "Shenzhen Loop Area Institute", "LIBERO", "LIBERO-Plus", "RoboTwin 2.0", "V-JEPA 2"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-vlda-learn-compact-deltas-as-actionable-transition-interfaces-for-robot", "markdown": "https://wpnews.pro/news/show-hn-vlda-learn-compact-deltas-as-actionable-transition-interfaces-for-robot.md", "text": "https://wpnews.pro/news/show-hn-vlda-learn-compact-deltas-as-actionable-transition-interfaces-for-robot.txt", "jsonld": "https://wpnews.pro/news/show-hn-vlda-learn-compact-deltas-as-actionable-transition-interfaces-for-robot.jsonld"}}