Has GPT-6-Astra Achieved Embodied Intelligence?

A benchmark study of GPT-6-Astra found the model reaches above 95% success on the LIBERO family and SIMPLE and about 90% on VLN-CE at a 10× execution budget, but only 69.2% zero-shot on RoboLab, where no task-level in-context-learning package can be built. The evaluation probed GPT-6-Astra across 12 benchmarks and eight real-robot platforms, allowing in-context learning from training data and up to 10× the official step limit; the authors report that even nearly solved benchmarks cost about 1M tokens and nearly an hour per LIBERO-Long++ episode, and roughly 9M tokens and nearly two hours on LIBERO-Ultra-Long. The study concludes that task-specific in-context-learning packages limit transfer and that high token cost constrains real-time deployment.

We ask not only how capable GPT-6-Astra is, but what its capabilities imply for the field: we probe its limits on 12 benchmarks, dissect the harness behind it, validate it on eight real-robot platforms, and prototype what the next generation of embodied benchmarks should measure. Probing the capability upper bound of GPT-6-Astra across 12 benchmarks Sort by resultSort by gain over SOTAGroup by embodiment GPT-6-Astra under a protocol that allows in-context learning on the training set and extends the execution budget up to 10×. Published baselines are reported under the standard budget, collected as of October 12, 2026. Result with a VLA model. ‡Zero-shot, without access to training data. In-context learning protocol ICL In-context learning from the training set Successful demonstrations are conventionally accessible when training smaller task-specific policies, so GPT-6-Astra may use them too. It studies the training data and builds a frozen knowledge package of task guidance, motion and visual references, executable helpers, recovery procedures and metadata, then retrieves and adapts these resources to each test scene. 10× A substantially larger execution budget We allow up to 10× the official step limit, since standard step limits can significantly underestimate GPT-6-Astra. Its emerging capability for failure recovery lets it translate additional steps into higher success, nearly saturating SIMPLE and the LIBERO family. Extensions continue the same episode — no resets. In-context learning: what GPT-6-Astra keeps Contents of an ICL knowledge package Knowledge package Kwritten by GPT-6-Astra during ICL preparation · frozen before testing Success rate versus step budget. The budget is expressed as a multiple of each benchmark's official step limit; dots show observed success rates and pink curves show fitted trends. We evaluate budgets up to 10× the official limit 1× for BEHAVIOR . The gray line marks the upper bound, which assumes that every episode that exhausts the evaluated budget without succeeding has the potential to succeed with an unlimited budget. Limitations and failure modes ←1 / 5→ 1LimitationTask-specific ICL2LimitationTime & cost3Failure modeVerifying physical success4Failure modeConsequential actions5Failure modeProgress over long horizons Limitation ICL packages are largely task-specific, which limits transfer Packages are built from each benchmark's training data, so their benefit weakens once test tasks move away from it. Packages that generalize beyond their source tasks remain an open direction. RoboDojo · Open group Lowest SR No task-specific demonstrations; it relies on a package studied from the other four groups. GPT-6-Astra + ICL still benefits from more interaction, but attains the lowest success of all groups. RoboLab · no training set 69.2% No task-level package can be built zero-shot, 10× budget . GPT-6-Astra leads the best listed baseline 42.9% , yet its absolute success remains moderate. Close coverage 95% Where test tasks follow the training distribution — LIBERO family, SIMPLE, VLN-CE — demonstrated motions or routes are retrieved almost directly and success approaches saturation: above 95% on the LIBERO family and SIMPLE, around 90% on VLN-CE at 10×. SIMPLE · zero-shot → ICL 70.8 → 90.8% Standard-budget success on the four common tasks, with shorter successful episodes under ICL. Limitation Time and cost remain high Even nearly solved benchmarks are expensive: about 1M tokens and nearly an hour per LIBERO-Long++ episode, roughly 9M tokens and nearly two hours on LIBERO-Ultra-Long. Extending the budget adds more, concentrated in difficult episodes that often still fail: on RoboDojo, an extended-budget episode takes more than three hours on average. About 98% of tokens are cached input, so 1M tokens ≈ 1 USD. Inference latency is not counted in the step budget, so this cost also limits deployment where real-time control is required. Average per-task cost by benchmark Tokens M Time min Failure mode Verifying physical success GPT-6-Astra can tell a task is unfinished but not which physical condition is unmet, so its corrections fail to converge. Looking done, or reporting done, is not the same as meeting the task's contact, support and alignment conditions. Failure mode Recovering from consequential actions Each attempt changes the state that recovery starts from, and some changes cannot be undone. A failed prerequisite blocks every later subtask while retries drain the budget. Recovery works when GPT-6-Astra changes strategy rather than repeating itself, as in DexJoCo Fold Glasses, where a side approach and a test lift complete the task. ← Pour liquid into cupMulti-room navigation → Failure mode Preserving progress over long horizons Earlier progress must stay intact while later actions run, yet a later motion can undo it, and small errors in grasping, placement or navigation compound across stages. GPT-6-Astra has to keep rechecking what it has already done rather than treating completed subgoals as permanently resolved. Next 2 · Time & cost→ 02 · Harness Ablation Harness Matters: An Ablation Token cost versus success rate on RoboDojo AllA · System 1–System 2B · Action interface 42 RoboDojo tasks; performance is the unweighted mean of category success rate. Dashed lines cross at the Agentic GPT-6-Astra baseline; upper-left is better on both axes. Standalone VLA π0.5, 6.91% uses no GPT tokens and is not plotted. Hover a point to compare it with the baseline. A · System 1 – System 2 collaboration Supervision pays off, but every review costs tokens Letting the VLA run alone is insufficient; GPT-6-Astra reviewing its proposed actions works, but costs tokens. Reviewing four at a time saves 63.5% of tokens over per-action review, and adding ICL gives the best result — at about 6.9× the tokens of the best closed-loop setting without ICL. 5.83%open-loop25.83%closed-loop · L = 454.17%ICL + VLA B · Action representation & chunking Interfaces should match how the model reasons Directional commands help and joint angles hurt. Letting the model choose its chunk size beats a fixed chunk in success rate, at a higher token cost. What Still Separates Benchmark Success from Deployment 01 / ZERO-SHOT No demonstrations, no knowledge package GPT-6-Astra performs as a generalist, without task- or robot-specific training. 02 / HUMAN PRIORS Structured context written by a human Tool schemas, joint limits, camera placement, safety constraints, hand presets and, on some platforms, task-level guidance such as admissible grasp points — no high-level semantic skills such as pick, place or insert. 03 / WALL CLOCK Budgeted in time, judged by a person Trials are normally limited to 60 minutes. An operator holds an emergency stop, and success is judged from the physical outcome, not the model's own claim. Embodiment Main questionTasks Single arm, gripperFranka 3, Franka Panda, PiPER + Pika gripper 3Do simulation results transfer, from rigid to deformable and precise tasks? Stacking and container nesting Cloth and towel folding Plug insertion Dual arm, grippersAgileX Piper 1Can GPT-6-Astra coordinate two arms on harder zero-shot tasks? Ordered folding Flower insertion Bimanual holding Sorting Dual arm, dexterous handsTianji M6S 1Can GPT-6-Astra control high-DoF hands? Toy and bottle grasping Wheeled mobile manipulatorGalaxea R1 Pro, AgileX Tracer + xArm7 2Can GPT-6-Astra act over room-scale distances and long horizons? Find washer and press button Flower collection Tray arrangement Navigation and search HumanoidUnitree G1 1Can GPT-6-Astra act as System 2 over low-level locomotion and arm control? Tabletop pick-and-drop Turn, approach and stop Findings and what changes in the real world ←1 / 4→ 1FindingOne model, every embodiment2FindingActive vision3FindingSystem 2 over System 04What changesPhysical limits & missing demos Finding 1 One model drives every embodiment; success depends on the task Without platform-specific training, the same model controls all eight platforms. Rigid pick-and-place is reliable and matches simulation, whereas deformable objects, precise contact and long-horizon mobile tasks remain markedly harder. A second arm extends the range of zero-shot tasks, and GPT-6-Astra re-plans from execution feedback when the scene changes. What GPT-6-Astra can do depends on what it can see In flower insertion, the gripper holds the stem vertically, so its own wrist camera cannot see the vase mouth. On a single arm the insertion fails. On the dual-arm Piper, GPT-6-Astra moves the free arm until its wrist camera sees the vase mouth, then completes the insertion — the viewpoint becomes part of the action space. Teleoperated demonstrations rarely contain this active vision. Finding 3 GPT-6-Astra as System 2 over existing System 0 motor control On a real Unitree G1 with Wuji hands, GPT-6-Astra commands arm-joint targets and binary hand states for manipulation 2/3 , and turns and walks through the vendor velocity interface 3/3 — without a task-trained System 1 policy. This establishes feasibility under slow, feedback-driven supervision rather than reliable real-time visuomotor control. What changes What changes in the real world In simulation, extra steps and failed attempts cost nothing, and demonstrations of the test task exist. On real robots neither holds: physical constraints limit extended execution, and missing demonstrations reshape in-context learning. Physical constraints Extended execution is not free on real hardware Extra steps and retries cost nothing in simulation. On real robots, protective stops and hardware faults end runs, and elapsed time adds tokens, risk, hardware stress and supervision. 2 / 3PiPER plug-insertion trials ended by a joint collision fault62 °Chand temperature guard~90 %of G1 time deliberating Missing demonstrations Context quality decides the outcome Human-written context can stand in for demonstrations, but it is expert prompt engineering. Building context from other sources is an open problem. Stated as a person would, e.g. “Could you make me breakfast?” Judged on the request Intent and on unstated obligations Duty . 02 / NO DEMOS Experience acquired at test time No in-domain demonstrations. The agent learns the scene through its own exploration. 03 / WORLD CLOCK A world that keeps running Thinking takes world time, as on a real robot. The judge watches the whole trajectory, even after the agent says it is done. 1 Open-source assetspublicly available object assets 2 Layouta floor plan composed into an interactive home 3 Agent-assisted placementcoding agents place items at real-world scale on named supports, under human review 4 Physics bindingarticulation, liquids, cloth, contact, heating and cooking 5 Task designa task states only a need; judged on Intent and on unstated Duty 6 Evaluationonboard cameras only; deliberation advances world time Interactive homeNot loaded Load interactive scene The scene couldn’t load. Please reload the page. Room tour: a camera fly-through of the apartment, from day to night.Laundry: carry the basket, load the washer, add detergent, start it.Coffee: brew a cup with the capsule machine.Breakfast: prepare a meal with the air fryer and toaster, while the world clock keeps running. ←→ 30 interaction clips in the home: doors, drawers, appliances, cooking and tidying. Drag or use the arrows to browse. ← 3D ViewerRoom tourLaundry taskCoffee taskBreakfast taskGallery → Findings in the household and open challenges ←1 / 4→ 1FindingLoses track of the home2FindingUnfamiliar fixtures, own body3FindingOutdated observations4Open challengesJudge, fidelity, time, cost Finding 1 It attends to the object in hand and loses track of the rest of the home Its own arm and the states it has changed seem to enter its decisions only when they block the next step. A final-state check would miss this, so the judge tracks the whole trajectory. Finding 2 Without demonstrations, it does not work out how unfamiliar fixtures move or where its own body has to be It plans for the gripper and handle, not its whole body, and repeats and adjusts the same motion. In Section 01, the demonstrations it distills in context show both, so this weakness does not appear there. Finding 3 It bases decisions on observations that become outdated while it thinks It rightly decides to turn off the burner once the egg sets, but thinking takes world time: the burner goes off about 90 s late and the egg chars. World clock Too late The simulators of Section 01 wait for the model. This world does not, so the action arrives too late. Open challenges What a next-generation benchmark still has to solve Measuring deployment rather than isolated tasks raises four open problems for the judge, the simulator and the evaluation budget. 01 / JUDGE Evidence for the judge Simulators model key physical facts heuristically; vision-language success detectors reach at most 60% balanced accuracy on contact-rich assembly. 02 / FIDELITY Simulation fidelity Modeling errors look like agent behavior; simulation has been shown to predict real-robot results only for short tabletop tasks. 03 / REAL TIME Real-time evaluation If the world keeps running, scores depend on latency, hardware and service load. 04 / COST Evaluation cost Hour-long deployments are expensive to repeat; even short robot trials are typically run too few times to separate policies reliably. Next 2 · Unfamiliar fixtures, own body→ A clear leap in embodied intelligence — but not its endpoint. Strong harness design with in-context learning and a sufficient execution budget can push many isolated benchmarks close to saturation. The remaining challenges shift toward transfer across tasks, long-horizon reasoning, failure recovery, and real-world constraints on time, cost and safety. The next frontier is not simply to solve more isolated tasks, but to build embodied systems that operate reliably and efficiently under realistic deployment conditions.