ORA: Open Robot-Use Agents ORA v0.1, an open robot-use agent harness that lets a language model operate a robot through cameras and tools without robot-specific training, solves 39.9% of RoboDojo zero-shot, up from 5.2% in October's preview, the ORA team reported. In code mode the model writes Python against the same supervised robot tools, carrying 3.62 robot actions per turn instead of 1.10 and raising solved cases from 6 to 20 across 59 paired RoboDojo cases, while depth-camera-planned motion cut collision-stopped moves from 14.1% to 7.0%. On RoboLab-120, Qwen3.8-27B solves 5.4% alone, 52.5% inside ORA, and 61.9% with PEAR's training added. ORA v0.1: Robot-use agents now write code, move smoothly, and reach 40% on RoboDojo TL;DR ORA is an open harness: a language model operates a robot with cameras and tools, and no robot training 1 ref-isola . v0.1 with the open-weight DeepSeek-V4.1-Flash 2 ref-deepseek-flash solves 39.9% of RoboDojo 3 ref-robodojo zero-shot, up from 5.2% in October's preview 4 ref-ora , 5 ref-ora-preview . 1. Write code. code-mode The model writes Python against the same robot tools; one turn carries 3.6 robot actions instead of 1.1. 2. Move smoothly. smooth-motion Every move is planned around what the depth cameras see; moves stopped by a collision fall from 14.1% to 7.0% on RoboDojo. 3. Work across open models. models In code mode on 39 RoboDojo tasks, six open models solve from 48.7% DeepSeek down to 5.1% Nemotron . 4. Cold-start post-training. harness-and-model On RoboLab-120, Qwen3.8-27B solves 5.4% alone, 52.5% inside ORA, and 61.9% with PEAR's training on top 6 ref-pear . Overview How ORA works, what v0.1 solves, and six solved episodes you can inspect. ORA is a robot-use agent 1 ref-isola : it brings the loop of computer-use agents 7 ref-cua , 8 ref-claude-cu , 9 ref-osworld to robots, as a line of recent systems does 10 ref-openrua , 11 ref-ura , 12 ref-rpent , 13 ref-pyrualean , 14 ref-rax , 15 ref-aspire , 16 ref-astranomy , 17 ref-roboskills , 18 ref-auriga . The model asks for an action; the harness, with its manual, tools and context, carries it out on the robot and returns what happened; the model decides again. Language models drove robots zero-shot before these agents 19 ref-saycan , 20 ref-voxposer ; ORA's loop is built on Pi 21 ref-pi-agent . 1. Code mode The same harness, as a Python library in a workspace that persists. Code mode adds no robot capability: the cameras, the motion stack and the supervision that admits one robot command at a time are the same. What changes is the interface. Instead of one tool call per turn, the model writes Python in a persistent workspace and can run up to 12 robot motions in one decision. Programs run in a sandbox 27 ref-bubblewrap with no network and reach the robot only through the same supervised tools. Programs as an agent's action have a history: language-model programs for robot control 28 ref-codeaspolicies , 29 ref-capx , executable code in place of one tool call per turn 30 ref-react , 31 ref-codeact , and code mode for tool use 32 ref-cloudflare-codemode , 33 ref-anthropic-code-mcp . The gain over tool calls On build tower , the tool-call run gives up after 94 model calls with the planks still on the table; the code-mode run stacks the tower. One of its programs finds a block, measures it, grasps, lifts, places, and checks the height before it hands back. Across 59 paired RoboDojo cases a turn carries 3.62 robot actions instead of 1.10, and solved cases go from 6 to 20. Given a workspace, models fill it with tools of their own 34 ref-voyager , 35 ref-latm . Qwen 36 ref-qwen-flash-next and DeepSeek build perception and geometry helpers, a median of 122 and 37 functions per episode, and import them in later turns; Qwen3.8-27B 37 ref-qwen38 writes 46, Inkling 38 ref-inkling 12, Muse 39 ref-muse 7 and Nemotron 40 ref-nemotron 4. More code is not more success: DeepSeek 's failed episodes hold more functions than its solved ones. 2. Smooth motion Every move is planned around what the depth cameras see and timed inside joint limits. Between asking for a move and seeing the next picture, the model sees nothing. In October a move was a straight run at the target, as fast as the controller would go. Arms clipped boxes, hit each other and knocked finished towers over. In v0.1 every move goes through one tool, move pose . The harness builds a 1 cm voxel map from the robot's depth cameras, plans around it with cuRobo 43 ref-curobo , and times the path inside joint limits 44 ref-toppra , 45 ref-ruckig . A request that would collide, press into a surface or wind the wrist is refused before anything moves, with a named reason the model can act on. A guard stops an arm that falls behind its command. On matched cases, moves stopped by a collision fall from 14.1% to 7.0% on RoboDojo and from 18.4% to 6.8% on RoboTwin 24 ref-robotwin . Light contact is not solved: about one move in ten still shifts an object by 5 mm or more. 3. Analysis across models Six open models, the same prompt, tools and budgets, 39 RoboDojo tasks. In code mode DeepSeek-V4.1-Flash solves 44.9% of these cases 48.7% over four layouts and Qwen3.8-Flash-Next 35.9%; few of their programs fail 10% and 13% . Inkling-2-Small 38 ref-inkling solves 21.8% with programs of nine lines. Muse Spark Open 16.7% , Qwen3.8-27B 15.4% and Nemotron-3.5-Super-VL 5.1% lose about a third of their programs to a crash; for Muse and Nemotron it is mostly a refused call they did not catch. More reasoning is not what separates them: Qwen3.8-27B reasons the most, a median of 3,390 tokens per call against DeepSeek 's 2,083. The open models were served with vLLM 46 ref-vllm and SGLang 47 ref-sglang . The traces are open: 3,557 episodes from seven models on five benchmarks, with everything each model was given and everything it wrote, are on Hugging Face https://huggingface.co/datasets/FastRUA/ora-traces . Across the six, the RoboDojo result lines up with public scores for agentic coding far more than with those for visual reasoning. 4. Harness and model What a harness adds, and what training adds on top of it. Training a reasoning model to use robots 6 ref-pear , 49 ref-agenticroborl , 50 ref-captain or a vision-language-action model 51 ref-pi05 changes the model; a harness changes what the model is given. PEAR trains Qwen3.8-27B 37 ref-qwen38 with reinforcement learning in simulation to drive a robot with no tools. We ran the base model and PEAR's released checkpoint 52 ref-pear-ckpt inside ORA and scored them the way PEAR does. On RoboLab-120 26 ref-robolab the base model goes from 5.4% alone to 52.5% inside ORA, ahead of the trained model alone 37.3% . The trained model inside ORA reaches 61.9%. RoboDojo's long-horizon tasks stay at 0% either way. ORA is open source. To run it, go to the repository https://github.com/fast-rua/ORA . References 1. 1Isola 2026 . Robot-Use Agents https://web.mit.edu/phillipi/www/writing/robot-use-agents.html . 2. 2DeepSeek 2026 . DeepSeek-V4.1-Flash-1017 https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017 . 3. 3Chen et al. 2026 . RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies https://arxiv.org/abs/2607.04434 . 4. 4Yang et al. 2026 . ORA: Open Robot-Use Agents Zero-Shot Solve Robot Tasks for $0.35 https://openreview.net/forum?id=Z3d0gg7uU0 . 5. 5Yang et al. 2026 . ORA: Open Robot-Use Agents https://haoailab.com/blogs/ora/ . 6. 6Tang et al. 2026 . Training Reasoning Models in Simulation to Natively Use Real Robots https://arxiv.org/abs/2611.18562 . 7. 7OpenAI 2025 . Computer-Using Agent https://openai.com/index/computer-using-agent/ . 8. 8Anthropic 2024 . Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku https://www.anthropic.com/news/3-5-models-and-computer-use . 9. 9Xie et al. 2024 . OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments https://doi.org/10.52202/079017-1650 . 10. 10Chu et al. 2026 . OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies https://arxiv.org/abs/2610.02459 . 11. 11Yuan et al. 2026 . Democratizing Robot-Use Agents: A Unified Harness with Open Models https://arxiv.org/abs/2611.01560 . 12. 12Zhang et al. 2026 . Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents https://arxiv.org/abs/2607.08448 . 13. 13Si et al. 2026 . Fewer Tokens, Better Action: GPT-6 Astra Robot Agents with 14% Higher Success Rate but 65% Fewer Tokens https://arxiv.org/abs/2610.01939 . 14. 14Huang et al. 2026 . Scaling Online Exploration for Robot Agents https://arxiv.org/abs/2610.22496 . 15. 15Lu et al. 2026 . ASPIRE: Agentic /Skills Discovery for Robotics https://arxiv.org/abs/2607.00272 . 16. 16Yang et al. 2026 . Astranomy: An Empirical Study of GPT-6 Astra as a Native Robot Policy https://research.nvidia.com/labs/srl/projects/astranomy/ . 17. 17OpenRoboKit 2026 . RoboSkills: A minimal set of agent workflows and tools for robotics https://github.com/openrobokit/roboskills . 18. 18StellarEdge AI Technical Team 2026 . Auriga Technical Report: A Task-Agnostic Agent Harness for Zero-Shot Embodied Manipulation https://auriga.stelledge.com/ . 19. 19Ahn et al. 2022 . Do As I Can, Not As I Say: Grounding Language in Robotic Affordances https://arxiv.org/abs/2204.01691 . 20. 20Huang et al. 2023 . VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models https://arxiv.org/abs/2307.05973 . 21. 21Earendil Works 2025 . Pi https://github.com/earendil-works/pi . 22. 22RoboDojo Team 2026 . RoboDojo Sim Leaderboard, Agent Track https://robodojo-benchmark.com/leaderboard . 23. 23Zhang et al. 2026 . An Unexpected Robot Policy: Early Evaluations of GPT-6 Astra on RoboDojo and Beyond https://arxiv.org/abs/2609.24170 . 24. 24Chen et al. 2025 . RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation https://arxiv.org/abs/2506.18088 . 25. 25Zhou et al. 2025 . LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization https://arxiv.org/abs/2510.03827 . 26. 26Yang et al. 2026 . RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies https://arxiv.org/abs/2604.09860 . 27. 27Containers 2016 . Bubblewrap: low-level unprivileged sandboxing tool https://github.com/containers/bubblewrap . 28. 28Liang et al. 2022 . Code as Policies: Language Model Programs for Embodied Control https://arxiv.org/abs/2209.07753 . 29. 29Fu et al. 2026 . CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation https://arxiv.org/abs/2603.22435 . 30. 30Yao et al. 2023 . ReAct: Synergizing Reasoning and Acting in Language Models https://arxiv.org/abs/2210.03629 . 31. 31Wang et al. 2024 . Executable Code Actions Elicit Better LLM Agents https://arxiv.org/abs/2402.01030 . 32. 32Varda and Pai 2025 . Code Mode: the better way to use MCP https://blog.cloudflare.com/code-mode/ . 33. 33Anthropic 2025 . Code execution with MCP: Building more efficient agents https://www.anthropic.com/engineering/code-execution-with-mcp . 34. 34Wang et al. 2023 . Voyager: An Open-Ended Embodied Agent with Large Language Models https://arxiv.org/abs/2305.16291 . 35. 35Cai et al. 2023 . Large Language Models as Tool Makers https://arxiv.org/abs/2305.17126 . 36. 36Qwen Team 2026 . Qwen3.8-Flash-Next https://huggingface.co/Qwen/Qwen3.8-Flash-Next . 37. 37Qwen Team 2026 . Qwen3.8-27B https://huggingface.co/Qwen/Qwen3.8-27B . 38. 38Thinking Machines Lab 2026 . Inkling-2-Small https://huggingface.co/thinkingmachines/Inkling-2-Small . 39. 39Meta 2026 . Muse-Spark-Open-816B-A42B https://huggingface.co/meta-models/Muse-Spark-Open-816B-A42B . 40. 40NVIDIA 2026 . NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 . 41. 41Reimers and Gurevych 2019 . Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks https://arxiv.org/abs/1908.10084 . 42. 42van der Maaten and Hinton 2008 . Visualizing Data using t-SNE https://www.jmlr.org/papers/v9/vandermaaten08a.html . 43. 43Sundaralingam et al. 2023 . cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generation https://arxiv.org/abs/2310.17274 . 44. 44Pham and Pham 2018 . A New Approach to Time-Optimal Path Parameterization Based on Reachability Analysis https://arxiv.org/abs/1707.07239 . 45. 45Berscheid and Kröger 2021 . Jerk-limited Real-time Trajectory Generation with Arbitrary Target States https://arxiv.org/abs/2105.04830 . 46. 46Kwon et al. 2023 . Efficient Memory Management for Large Language Model Serving with PagedAttention https://arxiv.org/abs/2309.06180 . 47. 47Zheng et al. 2023 . SGLang: Efficient Execution of Structured Language Model Programs https://arxiv.org/abs/2312.07104 . 48. 48Artificial Analysis 2026 . Artificial Analysis Intelligence Index https://artificialanalysis.ai/evaluations/artificial-analysis-intelligence-index . 49. 49Liu et al. 2026 . AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics https://arxiv.org/abs/2610.28589 . 50. 50Fan et al. 2026 . CaPtain: A Pipeline for Post-Training Code-as-Policy Robot Agents https://arxiv.org/abs/2610.40799 . 51. 51Black et al. 2025 . π0.5: a Vision-Language-Action Model with Open-World Generalization https://arxiv.org/abs/2504.16054 . 52. 52MIT HAN Lab 2026 . Qwen3.8-27B-PEAR https://huggingface.co/mit-han-lab/Qwen3.8-27B-PEAR . 53. 53OpenRUA Team 2026 . Behind the RoboDojo Boom: Revisiting Embodied Evaluation of Robot-Use Agents https://openrua.ai/revisiting-rua . 54. 54Zhao et al. 2023 . Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware https://arxiv.org/abs/2304.13705 . 55. 55Haddadin et al. 2022 . The Franka Emika Robot: A Reference Platform for Robotics Research and Education https://doi.org/10.1109/MRA.2021.3138382 . 56. 56Khazatsky et al. 2024 . DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset https://doi.org/10.15607/RSS.2024.XX.120 . 57. 57OpenAI 2026 . GPT-6 Astra: A new generation of intelligence https://openai.com/index/gpt-6-astra/ . 58. 58Mittal et al. 2025 . Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning https://arxiv.org/abs/2511.04831 . 59. 59Xiang et al. 2020 . SAPIEN: A SimulAted Part-based Interactive ENvironment https://arxiv.org/abs/2003.08515 . 60. 60Zhu et al. 2020 . robosuite: A Modular Simulation Framework and Benchmark for Robot Learning https://arxiv.org/abs/2009.12293 . 61. 61Liu et al. 2023 . LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning https://arxiv.org/abs/2306.03310 . 62. 62Carion et al. 2026 . SAM 3: Segment Anything with Concepts https://arxiv.org/abs/2511.16719 . 63. 63Sundermeyer et al. 2021 . Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes https://arxiv.org/abs/2103.14127 . Citation To cite this post: @misc{ora2026v01, title = {{ORA} v0.1: Robot-Use Agents Now Write Code, Move Smoothly, and Reach 40\% on {RoboDojo}}, author = {Yuanbo Yang and Junda Chen and Quanquan Peng and Yulun Wu and Shaoxiong Duan and Hao Zhang}, year = {2026}, month = nov, howpublished = {FastRUA Blog}, url = {https://fastrua.ai/blog/2026-11-code-mode/} } Appendix How successes are counted RoboLab 26 ref-robolab and RoboDojo 3 ref-robodojo limit the simulator steps of an episode. ORA records the benchmark's verdict at the limit and lets the episode go on; this post counts a success reached later too, except in section 4, which follows PEAR 6 ref-pear and counts only successes within the limit. RoboTwin 24 ref-robotwin and LIBERO-Pro 25 ref-liberopro end an episode at their limit. | Comparison | | All successes | Within the step limit | | RoboDojo, 198 cases | ORA v0.1 | 79 39.9% | 21 10.6% | | | Uni-Robot-Agent | 55 27.8% | 32 16.2% | | RoboLab, 300 cases | ORA v0.1 | 235 78.3% | 200 66.7% | | | Uni-Robot-Agent | 187 62.3% | 179 59.7% | | RoboLab, 500 cases | ORA v0.1 | 79.2% | 67.0% | | RoboTwin, 235 cases | RPent | 167 71.1% | 151 64.3% | | Paired experiment, 59 cases | tool calls | 6 | 2 | | | programs | 20 | 5 | | Open models: DeepSeek, 156 cases | tool calls | 58 | 36 | | | programs | 76 | 30 | | Qwen, 78 cases | tool calls | 29 | 17 | | | programs | 28 | 7 | | Inkling-2-Small, 78 cases | tool calls | 12 | 6 | | | programs | 17 | 6 | | Muse, 78 cases | tool calls | 9 | 6 | | | programs | 13 | 8 | | Qwen3.8-27B, 78 cases | tool calls | 10 | 6 | | | programs | 12 | 5 | | Nemotron, 78 cases | tool calls | 3 | 3 | | | programs | 4 | 2 | Within the limit Uni-Robot-Agent 11 ref-ura solves more RoboDojo cases than ORA, and for DeepSeek and both Qwen models tool calls solve more than programs. RoboDojo's leaderboard 22 ref-robodojo-lb counts within the limit only, so 10.6% is the number nearest to it. We count later successes because the limits were set for policies that act at a fixed control rate 53 ref-openrua-revisiting , and a late success passes the benchmark's own check. Uni-Robot-Agent ran with its own limit of 3 hours per episode, ORA with 6. The numbers in full ORA v0.1 with DeepSeek-V4.1-Flash , zero-shot, every success counted: | Benchmark | Robot | Cases | Programs | Tool calls | | RoboTwin 2.0, clean to randomized | two-arm ALOHA 54 ref-aloha | 235 | 79.6% 187 | 80.4% | | LIBERO-Pro | Franka Panda 55 ref-franka | 800 | 87.5% 700 | 85.4% | | RoboLab, 100 of its 120 tasks | DROID 56 ref-droid Franka | 500 | 79.2% 396 | | | RoboDojo | two-arm ARX X5 | 198 | 39.9% 79 | | Baselines, run by us on the same cases: Uni-Robot-Agent 11 ref-ura solves 55 of 198 on RoboDojo and 187 of 300 on RoboLab; RPent 12 ref-rpent solves 167 of 235 on RoboTwin, 738 of 800 on LIBERO-Pro with its LIBERO VLA 51 ref-pi05 and explored memory, and 11 of 200 on RoboDojo with its VLA. On RoboDojo's leaderboard 22 ref-robodojo-lb , 23 ref-astra-robodojo , read on 16 November 2026, GPT-6-Astra 57 ref-gpt6astra scores 22.48% and DeepSeek V4.1 Flash 1.92%. The motion stack Under every planned move, on every simulator 58 ref-isaaclab , 59 ref-sapien , 60 ref-robosuite : | Stage | What it does | | Collision world | Rebuilt for each request from the robot's own depth cameras only RoboDojo: the head and both wrists; RoboTwin: four cameras; LIBERO 61 ref-libero : two , as 1 cm voxels. The robot's own body is removed; the start pose and a box around the hand at the goal are cleared; a held object moves with the hand; space no camera sees counts as free. A goal that puts a fingertip more than 1 cm below the surface under it is refused before planning. | | Path | cuRobo 43 ref-curobo plans from the measured joints, with the other arm as an obstacle where it stands. The path is checked a second time against the scene without the clearing at the goal. | | Wrist | Along the path the hand may turn at most 45 degrees more than between its two ends; beyond that the move is refused paced path wrist unwind . With empty jaws the same grasp turned a half-turn about the approach is tried as well. | | Timing | TOPP-RA 44 ref-toppra , then Ruckig 45 ref-ruckig . fast : each joint at up to 4 rad/s and 40 rad/s². normal : 0.3 m/s and 60 degrees/s at the hand. careful : 0.1 m/s and 20 degrees/s. slow approaching : the last 2 cm at careful . | | Contact | A move into contact is a straight line when it is within 50 mm and 15 degrees; a longer one is planned to a point short of contact and is straight for its last 40 mm. With expect contact the whole move is straight. The guard that stops a blocked arm reads how far the joints fall behind their commands; there is no force sensing. | | Refusal | Nothing moves and nothing is spent. The model is told one of goal unreachable , goal in collision , goal below support , start in collision , path blocked , paced path wrist unwind , paced path unreachable , self collision , table collision , inter arm collision , contact path blocked , paced motion budget or invalid motion arguments . | Cost, and every model call by call A model call is not cheaper in ORA than in Uni-Robot-Agent 11 ref-ura : a median of 190k input tokens against 133k on RoboDojo. An episode is, because code mode makes about eight times fewer calls a median of 58 against 462 . | | DeepSeek V4.1-Flash | Qwen 3.8-Flash-Next | Inkling 2-Small | Muse Spark Open | Qwen 3.8-27B | Nemotron 3.5-Super-VL | |---|---|---|---|---|---|---| | Episodes | | Casesone episode each | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | | Solvedshare of cases | 45% | 36% | 22% | 17% | 15% | 5% | | Thinking, per model call | | Reasoning tokensmedian; quartiles and 90th percentile | 2.1k 800 to 4.6k, p90 8.4k | 1.1k 139 to 3.2k, p90 6.3k | 1.0k 272 to 2.5k, p90 4.9k | 341 163 to 642, p90 1.1k | 3.4k 542 to 8.4k, p90 16.4k | 180 40 to 767, p90 2.8k | | Calls with no reasoningshare of model calls | 2.8% | 0% | 0% | 0% | 0% | 0% | | What a decision carries | | Model calls per decisiontotal calls over total decisions | 2.8 | 3.9 | 3.2 | 2.1 | 3.1 | 3.1 | | Robot actions per acting turnmean | 3.9 | 4.2 | 2.1 | 3.9 | 3.0 | 1.6 | | Acting turns with several actionsshare | 73% | 68% | 41% | 66% | 61% | 28% | | The programs | | Lines per programmedian; quartiles | 3523 to 54 | 4020 to 72 | 96 to 15 | 2414 to 39 | 4123 to 66 | 1712 to 26 | | Programs that end in an errorshare of programs | 10% | 12% | 21% | 32% | 31% | 39% | | Robot tool calls | | Moves refusedshare of motion calls, before anything moves | 5% | 5% | 16% | 9% | 6% | 26% | | Calls not accepted, by reasonshare of all robot tool calls | 3% | 5% | 10% | 37% | 6% | 19% | | Context and length | | Input per model callthousand tokens; median over episodes | 187k | 95k | 98k | 91k | 106k | 85k | | Model calls per episodemedian | 58 | 146 | 136 | 88 | 104 | 139 | | Decisions per episodemedian | 18 | 31 | 40 | 37 | 33 | 40 | | How episodes end | | Endingshare of cases | 45% | 36% | 22% | 17% | 15% | 5% | | | DeepSeek V4.1-Flash | Qwen 3.8-Flash-Next | Inkling 2-Small | Muse Spark Open | Qwen 3.8-27B | Nemotron 3.5-Super-VL | |---|---|---|---|---|---|---| | Episodes | | Casesone episode each | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | | Solvedshare of cases | 40% | 37% | 15% | 12% | 13% | 4% | | Thinking, per model call | | Reasoning tokensmedian; quartiles and 90th percentile | 1.3k 347 to 3.7k, p90 7.8k | 1.3k 329 to 3.2k, p90 6.1k | 877 230 to 2.4k, p90 4.9k | 126 49 to 287, p90 587 | 2.0k 480 to 5.5k, p90 12.8k | 153 51 to 467, p90 1.6k | | Calls with no reasoningshare of model calls | 1.1% | 0% | 0% | 0% | 0% | 0% | | What a decision carries | | Model calls per decisiontotal calls over total decisions | 1.9 | 2.3 | 2.5 | 2.0 | 2.1 | 2.6 | | Robot actions per acting turnmean | 1.2 | 1.1 | 1.2 | 1.3 | 1.1 | 1.2 | | Acting turns with several actionsshare | 6% | 4% | 5% | 8% | 4% | 6% | | Robot tool calls | | Moves refusedshare of motion calls, before anything moves | 12% | 12% | 28% | 26% | 16% | 41% | | Calls not accepted, by reasonshare of all robot tool calls | 8% | 12% | 20% | 20% | 13% | 38% | | Context and length | | Input per model callthousand tokens; median over episodes | 232k | 109k | 111k | 100k | 113k | 102k | | Model calls per episodemedian | 66 | 140 | 158 | 111 | 130 | 160 | | Decisions per episodemedian | 34 | 64 | 65 | 59 | 73 | 60 | | How episodes end | | Endingshare of cases | 40% | 37% | 15% | 12% | 13% | 4% | Not accepted:move refusedbad argumentsper-program limitother Ending:solvedgave upout of budgetended by a failed check Every model, call by call. The episodes of section 3, each row on one scale for both settings. A decision counts against the episode's budget and an acting turn moved the robot; a refused move was turned down by the planner, and an episode runs out of budget on decisions, model calls, six hours or the context window.