{"slug": "ai2-maniflow-tb-an-open-322m-robot-action-model-for-block-stacking-with-54-to-59", "title": "Ai2 ManiFlow-TB: An Open 322M Robot Action Model for Block Stacking, With 54 to 59% Sim Success", "summary": "Ai2 released ManiFlow-TB, a 322.5-million-parameter open robot action model for the ToyBlocks block-stacking project, published on Hugging Face under an Apache 2.0 license with a model card reporting 54 to 59% simulation success across four runs of 128 episodes using an oracle high-level plan. The model builds on the ManiFlow policy introduced at CoRL 2025 by researchers from the University of Washington, UC San Diego, Nvidia and the Allen Institute for AI, and was trained for 200,000 steps on 8 GPUs on three ToyBlocks-generated LeRobot datasets of simulated Franka arm data. ManiFlow-TB uses flow matching with consistency training at 12 integration steps and outputs 25-action chunks from a 50-step prediction horizon, with no real-robot results reported on the model card.", "body_md": "Ai2 has put a small, fully open robot action model on Hugging Face. **ManiFlow-TB** is the low-level action model for ToyBlocks, a block-stacking robot project, with 322.5 million parameters, an Apache 2.0 license, and simulation results the model card reports with unusual candor: 54 to 59% success across four runs ([model card](https://huggingface.co/allenai/maniflow-tb?ref=explainx)).\n\nIt builds on the ManiFlow policy introduced at CoRL 2025 by researchers from the University of Washington, UC San Diego, Nvidia and the Allen Institute for AI ([ManiFlow project page](https://maniflow-policy.github.io/?ref=explainx)). This is a research release, not a product. Here is what it is, how to run it, and how to read the numbers.\n\n## TL;DR: the questions people ask first\n\n| Question | Answer (from the model card) | \n|---|---|\n| What does it do? | Low-level actions for tabletop block pick-and-place and in-place rotation of three blocks | \n| Size? | 322.5M parameters, bf16 safetensors | \n| Method? | Flow matching with consistency training, 12 integration steps at inference | \n| Trained on? | Simulated Franka arm data, three ToyBlocks-generated LeRobot datasets | \n| License? | Apache 2.0 for weights and training code | \n| Result? | 54 to 59% success over four runs of 128 episodes, with an oracle high-level plan | \n| Real robot results? | None reported on the model card | \n\n## Where it fits: a two-level system\n\nToyBlocks splits the job in two. A high-level policy decides what to do next, such as which block goes where, and passes a target heatmap token. The low-level action model, ManiFlow-TB, turns images, state and that target into motor commands. The model card says the evaluation uses an oracle high-level policy, taking the plan from the recorded task, so the numbers measure the low-level action model alone.\n\nThat split matters when you compare it with end-to-end robot foundation models. See our coverage of [RoboColosseum](https://explainx.ai/blog/robocolosseum-physical-intelligence-eval-124-sites-5699-comparisons-gr00t-n1-7-pi0-5-g1-humanoid-2026) and [Physical Intelligence's API beta](https://explainx.ai/blog/physical-intelligence-pi-api-beta-robot-fine-tuning-2026) for models trained to do both levels at once.\n\n## Architecture in plain language\n\n**Inputs.**\n\n- Front and gripper RGB images at 224 by 224, encoded with CLIP ViT-B/16.\n- A 9-dimensional robot state.\n- The task name, encoded with T5-small.\n- A target heatmap token from the high-level policy, encoded by the shared RGB encoder.\n\n**Model.** A ManiFlowMultiModal model with a DiTX action head: 12 layers, 8 heads, width 768. The ManiFlow paper describes DiT-X as a diffusion transformer with adaptive cross-attention and AdaLN-Zero conditioning, which lets action tokens interact with visual, language and state features.\n\n**Outputs.** A chunk of 25 executed actions from a prediction horizon of 50, as 9-dimensional absolute actions. The last dimension is a play mode: 1 for play, 0 for pause, -1 for skill done. Because the model was trained with pause-on-attempt, the play mode must be fed back into the model when it is 1 or 0.\n\n**Why flow matching with consistency?** Standard diffusion or flow policies need 10 to 100 steps per action. The ManiFlow paper says consistency training can generate high-quality actions in 1 to 2 steps. This checkpoint uses 12 integration steps, so it is a more conservative setting than the paper's headline claim.\n\n## Training details\n\n- Data: three ToyBlocks-generated LeRobot datasets of tabletop block manipulation.\n- Compute: 200,000 steps on 8 GPUs, per-device batch 128.\n- Optimizer: AdamW, learning rate 1e-4, betas 0.9 and 0.95, weight decay 1e-3, cosine schedule with 500 warmup steps, bf16 mixed precision.\n- Weights: the online, non-EMA weights from the final step.\n\nThe repository mirrors a training run directory, with `checkpoints/steps_200000_model.safetensors`, `config.yaml` and `stats.json` for normalization. ToyBlocks reads model and action settings from the config next to the checkpoints.\n\n## The results, and why the caveats matter\n\nAi2 ran closed-loop evaluation in ToyBlocks simulation on 128 episodes from a held-out validation set, each started from the initial state of a recorded episode.\n\n| Run | Success | Success at episode end | Collisions | \n|---|---|---|---|\n| 1 | 73 of 128 (57%) | 64 | 32 | \n| 2 | 75 of 128 (59%) | 67 | 32 | \n| 3 | 76 of 128 (59%) | 66 | 30 | \n| 4 | 69 of 128 (54%) | 63 | 38 | \n\nSuccess means all goal predicates hold with the blocks settled and the gripper open. Three details from the card are worth reading carefully:\n\n1. **The time limit changes everything.** The limit is 3 times the longest expert demonstration (194 seconds). With the default 30 seconds, only 26 to 28 of 128 episodes succeed, because most time out before the task finishes. Ai2 says that measures the limit, not the model.\n2. **Runs are not repeatable.** The four runs use the same episodes and settings but differ because simulation and batched inference are not bitwise deterministic. Two runs agree on only 82 to 97 of the 128 episodes, so the same episode can flip between success and failure.\n3. **Oracle planning.** The high-level plan comes from the recorded task. A learned planner would likely lower the numbers.\n\nA roughly 57% success rate on a three-block tabletop task is a baseline for research, not a reliable robot. The value is that the checkpoint, config, training code and evaluation recipe are all open, so others can try to beat it.\n\n## How to try it\n\nThe card gives the path. Download the repository, then evaluate with ToyBlocks, whose policy server finds `config.yaml` next to the `checkpoints` directory:\n\n```\nhf download allenai/maniflow-tb --local-dir maniflow-tb\npython scripts/launch_eval.py --policy maniflow \\\n  --model-path maniflow-tb/checkpoints/steps_200000_model.safetensors \\\n  --dataset-path <ToyBlocks eval dataset> --expert-timeout-scale 3.0\n```\n\nTraining code lives in the `allenai/maniflow-tb` repository (`scripts/train_maniflow.py`), and ToyBlocks also uses it to load the model. You will need the ToyBlocks code and data, which the model card links. Check the card for current links and flags before running, since the evaluation command in the card is abbreviated.\n\n## What the ManiFlow paper adds\n\nThe project page for the original ManiFlow work gives wider context. The authors report evaluation across 64 simulation tasks and 8 real-world scenarios, and a 98.3% relative improvement over existing approaches on challenging real-world bimanual dexterous manipulation. They tested on a Unitree H1 humanoid, a dual xArm7 bimanual system and a single Franka Panda. Their stated limitation is that the policy fails on tasks needing detailed contact information and force feedback, such as delicate assembly or compliant insertion.\n\nThose figures are for the original ManiFlow research, not this ToyBlocks checkpoint. The model card for ManiFlow-TB reports simulation only.\n\n## Flow matching and consistency training, explained simply\n\nA robot action policy has to turn a camera image and a goal into a smooth sequence of joint or gripper commands. Diffusion and flow policies do this by starting from noise and refining it toward a good action chunk over several steps. Each step costs a forward pass through the network, so more steps mean slower control.\n\nFlow matching learns a path from noise to the correct action. Consistency training adds a second rule: points along the same path should map back to the same endpoint. The ManiFlow project page describes this as sampling several intermediate points on the path and learning to map any of them back to its origin. In theory that lets the model jump to a good action in one or two steps.\n\nThe ToyBlocks checkpoint uses 12 integration steps, per the card. That is far fewer than the 10 to 100 steps the paper says traditional diffusion or flow models need, but more than the 1 to 2 steps in its headline claim. If control speed matters to you, the number of steps is the first setting to test, and the card does not report how success changes with fewer steps.\n\n## Questions to ask before you build on it\n\n- **Does your task need the play-mode feedback?** The model emits a pause and skill-done signal that must be fed back into the next call. A wrapper that ignores it will behave differently from the evaluated setup.\n- **Do your cameras match?** The model expects a front camera and a gripper camera at 224 by 224. A different layout needs retraining or fine-tuning.\n- **Do you have the same action space?** Outputs are 9-dimensional absolute actions. A different robot or controller needs a mapping.\n- **Can you tolerate collisions?** The card logs 30 to 38 collisions per 128-episode run. Safety limits belong in your controller, not in the policy.\n- **Can you reproduce the numbers?** Use the 3x timeout setting. Run several seeds, because the card shows episodes flip between runs.\n\nNone of these are criticisms of the release. They are the practical gaps between a research checkpoint and a system you can depend on.\n\n## Where it sits among open robotics releases\n\nOpen robot models are arriving fast. For comparison, see our posts on [LeRobot datasets such as FineArt with 40,543 episodes](https://explainx.ai/blog/fineart-scale-ai-lerobot-dataset-40543-episodes-1718-hours-151-tasks-fineart-vla-pi0-5-2026), [NVIDIA's open FLORA models for robot data](https://explainx.ai/blog/nvidia-flora-open-models-robot-data-engine-captioner-rewarder-segmenter-gr00t-2026), [GreenWAM, an open world model that also acts as a policy](https://explainx.ai/blog/greenwam-sber-robotics-world-foundation-model-20-robot-families-2026) and [RoboValue, a benchmark for robotic value models](https://explainx.ai/blog/robovalue-benchmark-robotic-value-models-15-sim-20-real-tasks-lerobot-2026).\n\nManiFlow-TB is smaller and narrower than those. Its strengths are a clear recipe, a permissive license and honest evaluation notes.\n\n## What this means for what you build\n\nIf you do robot learning research, this is a small, reproducible baseline with a known failure profile: the card logs 30 to 38 collisions per 128-episode run. If you build products, it is not ready: sim only, three blocks, oracle planning. If you teach or learn robotics, the model card is a good example of what an honest evaluation section looks like, including the nondeterminism note.\n\n## Limits to remember\n\n- Simulation only. The card reports no real-robot results.\n- Oracle high-level plan, so results do not include planning errors.\n- Narrow task: three blocks, pick-and-place and rotation.\n- Evaluation is nondeterministic, so expect different numbers on your machine.\n- Our reading comes from the model card and project page. We did not run the model.\n\n## Related reading\n\n*Details are accurate as of November 5, 2026. Model files, links and evaluation settings may change on Hugging Face.*", "url": "https://wpnews.pro/news/ai2-maniflow-tb-an-open-322m-robot-action-model-for-block-stacking-with-54-to-59", "canonical_source": "https://explainx.ai/blog/ai2-maniflow-tb-toyblocks-open-robot-action-model-322m-franka-flow-matching-apache-57-percent-success-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog", "published_at": "2026-11-05 00:00:00+00:00", "updated_at": "2026-11-05 19:25:52.903303+00:00", "lang": "en", "topics": ["robotics", "artificial-intelligence", "ai-research", "machine-learning"], "entities": ["Ai2", "ManiFlow-TB", "ToyBlocks", "Hugging Face", "University of Washington", "UC San Diego", "Nvidia", "Allen Institute for AI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/ai2-maniflow-tb-an-open-322m-robot-action-model-for-block-stacking-with-54-to-59", "markdown": "https://wpnews.pro/news/ai2-maniflow-tb-an-open-322m-robot-action-model-for-block-stacking-with-54-to-59.md", "text": "https://wpnews.pro/news/ai2-maniflow-tb-an-open-322m-robot-action-model-for-block-stacking-with-54-to-59.txt", "jsonld": "https://wpnews.pro/news/ai2-maniflow-tb-an-open-322m-robot-action-model-for-block-stacking-with-54-to-59.jsonld"}}