uni-robot-agent: An Open Qwen Model Beats GPT-6 Astra's Published Robot Score, With a Catch

uni-robot-agent is a new open harness that lets a language model operate robots through tool calls, and its headline number is that open-weight Qwen3.8-Flash-Next scores 31.39 on RoboDojo, above the published 28.97 for GPT-6 Astra. That is true, but it compares two different harnesses. Run Astra through the new harness and it reaches 51.90. The real finding is that the harness matters as much as the model.

The work comes from the OpenRUA group, with Pieter Abbeel and Ning Ding among the authors. The paper is on Hugging Face (arXiv 2611.01560), the project page is at openrua.ai, and the code is in the Open-RUA/uni-robot-agent repo under the MIT License. It was the number three paper of the day on Hugging Face on November 3, 2026.

TL;DR #

What is a robot-use agent, and why use open models? #

A robot-use agent treats a robot like a computer-use agent treats a desktop. The model looks at camera images, writes a plan, calls tools such as "move to this pose" or "triangulate this point," and reads back what happened. We covered the idea when Hao AI Lab released ORA with DeepSeek-V4.1-Flash, and when GPT-6 Astra ran on a robot control benchmark.

The authors' argument is that frontier-only results block research. Without weights, "we cannot directly train them to become better robot-use agents," and API dependence makes large-scale interaction costly. Open models are weaker today. So before blaming the model, the paper asks what an open model can do with a well-built harness. That is the contribution: a standard way to connect models, tools and environments, which also records interactions in one format so they can be reused as training data later. Training itself is explicitly future work.

How does the harness work? #

A tool call passes through five layers: agent, orchestration, tools, services and backends. Simulators and robots plug in through adapters, so adding an environment needs an adapter and not a rewrite of every tool. The authors list six design choices that keep decisions tied to evidence:

• Same snapshot. Images, calibration and robot state are bound into one immutable snapshot, so a pixel is not misread after the arm moves.
• Check before repeating. Request identities reject duplicate actions when a reply goes missing.
• Move and grip together. One tool takes a target pose and a gripper opening.
• Stop on progress. Motion tools check measured poses and stop on arrival, or return control when the arm stalls.
• Inspect outcomes. Pose errors, stop reasons and outcome images show whether a grasp worked.
• Keep useful context. Old images become text placeholders while the prompt prefix stays stable for caching.

There is also pipelining. A background tool call returns a job handle, so the model can keep reasoning while the arm moves. Calls to the same arm queue in order, and flush_queue cancels calls that have not started when new feedback changes the plan. The paper reports time reductions of 4.8% to 73.7% in five selected cases, and says the full difference cannot be credited to scheduling alone.

The main simulation setting is strict. Agents get RGB images, camera calibration, robot proprioception, robot descriptions and the task text. They do not get depth, CAD assets, hidden scene annotations, verifier feedback or web access. Localization has to come from permitted observations, for example multi-view triangulation. For background on how reusable procedures fit into agents, see our agent skills guide.

What are the simulation results? #

RoboDojo has 42 tasks in five dimensions: generalization, precision, long-horizon execution, memory and open-vocabulary instruction following. The main runs use three seeds and two layouts each, 252 episodes, with a three-hour wall-clock limit per episode. Score credits partial progress; success rate counts fully successful episodes.

Source: the project page's Table 1. Authors' configurations use 252 episodes; published baselines keep their own protocols.

Three things stand out. First, the same Qwen model goes from 1.90 under RoboProbe to 31.39 under uni-robot-agent. That is a 29-point swing from the harness alone. Second, the claim "beats Astra" compares against a published number from another harness and protocol; the authors note that published baselines "retain their own protocols." Third, the pi-0.5 vision-language-action policy scores 11.44 on the same benchmark, and the best listed VLA or world-action entries, Xiaomi-Robotics-1 and OpenWAM-alpha, reach 20.07 and 17.18. Open language-model agents now sit above those entries on this benchmark, although they trail on long-horizon tasks. We covered one of those policies in Xiaomi Robotics-1.

The profiles differ. Qwen and DeepSeek do better on memory and open-vocabulary tasks than on precision and long-horizon work. The VLA and world-action models show the opposite pattern: Xiaomi-Robotics-1 scores 38.39 on long-horizon tasks but only 7.81 on memory.

How stable are the scores?

The authors ran 2,700 episodes for a depth-enabled Qwen pool and got a full-pool Score of 34.59. A sample of six rollouts per task variant averaged 34.59 with a 95% range of 30.65 to 38.51. Take-away: single small runs can move by several points, so be careful with leaderboard-style comparisons of one or two points.

What happened on real robots? #

uni-robot-agent dual-arm robot scene for inserting test tubes into a rack Image: OpenRUA

The real-world benchmark has 19 tabletop tasks on a physical dual-arm robot, scored out of 100 across five dimensions worth 20 points each.

Source: the project page. The arXiv HTML version of the paper lists lower totals for the same setup: Qwen 49.50, Astra 75.50 and Dream-1 35.50 (pi-0.5 is unchanged at 47.75). The two versions also differ in some per-dimension cells. We use the project page figures and flag the gap, since the authors have not explained it.

Qwen's 53.50 is above the fine-tuned pi-0.5 at 47.75 and the zero-shot Dream-1 at 43.50 (49.50 against 35.50 in the arXiv text; the ordering holds either way). That is a strong line for an open model with no robot training. But the comparison is not like for like. The authors state that their agents use depth from Intel RealSense D405 cameras while pi-0.5 and Dream-1 use RGB only. And the pi-0.5 baseline was fine-tuned with 10 hours of robot data per task. Dream-1 is the world-action model we covered in Dream Labs' Dream-1 API.

Per-task results are uneven. Qwen scores 8 of 8 on arranging cubes by color order and 4 of 4 on wiping a whiteboard, but zero on taking a bowl from a microwave, unfolding a rag and the Tower of Hanoi. Unseen tasks stayed hard: both Qwen and Astra scored 8 out of 20.

Open-weight robot-use agent solving a Tower of Hanoi task on a physical dual-arm robot Image: OpenRUA

Which capabilities limit open models? #

The paper evaluates thirteen models across generations, families and sizes. Some findings are surprising:

• Robot use predates the newest models. GPT-5.6 Sol already completes tasks (19.17% success). GPT-6.1 Sol reaches 37.50%.
• Newer Qwen versions matter. Qwen3.5 through 3.7 variants record zero success. Qwen3.8-27B reaches 8.06% and Flash-Next 25.00%. GLM-5.3 Flash, though recent, also scores zero.
• Size is not the story. The 27B Qwen3.8 completes tasks that earlier 122B and larger models do not. The authors suspect training distribution matters beyond parameter count.

They name four failure sources: visual understanding, spatial grounding, geometric coding and embodied reasoning. Geometry code is the clearest. Earlier Flash and Plus models reverse coordinate transforms or mix quaternion conventions in 47.3% to 81.8% of executed calculations. That falls to 8.2% for Qwen3.8-Flash, 1.5% for DeepSeek and zero observed errors for GPT-6.1 Sol. Across six models, robot score correlates with embodied reasoning on ERQA (Pearson r of 0.97) and multimodal understanding on MMMU-Pro (0.93). These are associations, and the authors say they do not isolate any one cause.

Do you need depth and hand-written skills? #

Less than you might think. Without depth but with camera calibration, Qwen keeps 87.5% of its score (35.89 down to 31.39) and Astra keeps 98.6% (55.83 to 55.04). Remove calibration too and the numbers fall hard: Qwen to 9.13 and Astra to 31.29. Removing the predefined skill library costs Qwen only about 3%: 30.31 versus 31.39. So the results are not just a product of hand-coded procedures, although clear task instructions still matter.

How do you try it? #

The quick start from the README targets a Python 3.11 virtual environment on a Linux GPU host:

pip install uni-robot-agent
uni-robot-agent setup --integration robodojo
export UNI_ROBOT_AGENT_API_KEY='YOUR_API_KEY'
export UNI_ROBOT_AGENT_PROVIDER_URL='https://provider.example/v1'
uni-robot-agent run --integration robodojo --deploy local \
  --task stack_bowls --model YOUR_MODEL_ID \
  --action-profile direct --no-sam3 --output-dir ./runs --monitor

You need a vision model behind a Responses-compatible endpoint. The --monitor flag opens a local dashboard with camera streams, tool calls and timing. Use --no-depth for the paper's RGB-only setting, since RoboDojo enables depth by default. Other integrations include RoboUse (tasks from LIBERO, RoboCasa, MetaWorld and others) and RoboLab on Isaac Sim. There is a Python SDK for writing your own strategy.

What are the limits? #

• Simulation first. Real-robot support for ARX One and Franka is "coming soon." Nobody outside the authors can reproduce the physical results today.
• Early code. The repo shows three commits, version 0.1.0 and 25 stars. Expect rough edges.
• Different baselines. Depth versus RGB-only, fine-tuned versus zero-shot, and published numbers under other protocols all blur the comparisons. The authors flag them; headlines often will not.
• Slow. Episodes have a three-hour wall-clock limit, and the authors found performance largely plateaued by the third hour in four-hour runs.
• Gap to the frontier. Astra beats Qwen by about 20 points on RoboDojo and 27 on real tasks.
• Self-assessed analysis. Part of the embodied-reasoning study uses GPT-6 Astra as the judge.

What this means for what you build #

If you work on robot learning, the takeaway is the harness. A 29-point swing from tools and feedback, with the model held fixed, says that agent design deserves the same attention as model choice. If you build agents in other domains, the same six ideas apply: bind observations to one snapshot, make actions idempotent, and return evidence of what ran. If you plan to buy API access for robot control, note that the open models still trail Astra clearly, and that nobody has shown the cost to scale to real deployments here. For comparisons on other tasks, see the robotics benchmark of frontier models on arms and humanoids.

Related reading #

Figures are from the project page and paper as read on November 3, 2026. The paper is a preprint and has not been peer reviewed.