cd /news/robotics/robowild-benchmark-claude-fable-5-2-… · home › topics › robotics › article
[ARTICLE · art-173639] src=explainx.ai ↗ pub= topic=robotics verified=true sentiment=· neutral

RoboWild Benchmark: Claude Fable 5.2 Leads at 13.3% While RoboDojo Hits 64.9%

Claude Fable 5.2 leads XLANG Lab's public RoboWild leaderboard at 13.3% weighted success across 341 real-world-style robot tasks, ahead of GPT-6 Astra at 11.6% and Gemini 4 Carbon at 10.6%, while the same model family scores 64.9% on RoboDojo Sim in Anthropic's November 9 launch table. RoboWild spans 38 furnished scenes and 4,480 objects across 23 scene types at 3.60 objects per square meter, and grasp failure accounts for 42.6% of failed runs, with 87% of runs disturbing unrelated objects. The benchmark from XLANG Lab, the group behind OSWorld-style computer-use stacks, is positioned as the next frontier after digital workflows, with the dataset released on Hugging Face at roughly 18.5 GB under Apache-2.0.

recommendation

Evaluate robot-use agents on dense real-world-style scenes like RoboWild's 341 tasks rather than simulation alone; the same model family dropped from 64.9% on RoboDojo Sim to 13.3% on RoboWild.

more in playbook: Evals and testing for LLM apps
by read8 min views3 publishedNov 21, 2026
RoboWild Benchmark: Claude Fable 5.2 Leads at 13.3% While RoboDojo Hits 64.9%
Image: Explainx (auto-discovered)

Claude Fable 5.2 tops the public RoboWild leaderboard at 13.3% weighted success, yet that is still only about one task in eight across 341 real-world-style robot chores. The same model family scores 64.9% on RoboDojo Sim in Anthropic's November 9 launch table. RoboWild is the gap made visible: dense rooms, articulated objects, and safety side effects that simulation stacks smoothed away.

The benchmark comes from XLANG Lab, the group behind OSWorld-style computer-use stacks. Two years after OSWorld, frontier models handle many digital workflows; physical robot-use agents (RUAs) remain the next frontier, in the lab's wording on the site.

TL;DR: scores and what they mean #

Question Answer
Best model on RoboWild? Claude Fable 5.2 at 13.3% (ReAct, highest thinking)
Runner-up? GPT-6 Astra 11.6% ; Gemini 4 Carbon10.6%
Tasks / scenes? 341 tasks,38 furnished scenes,4,480 objects
Robots? Mobile ALOHA, Franka Mobile, Unitree G1, Dexmate Vega-1
Top failure mode? Grasp failure 42.6% of failed runs
Sim comparison? Opus 5.5 50% RoboDojo vs4.81% RoboWild (100 calls)
Assets? Hugging Face dataset (~18.5 GB, Apache-2.0)

What RoboWild adds beyond RoboDojo and BEHAVIOR #

RoboWild home entryway furnished scene for robot-use tasks Image: XLANG Lab

RoboWild's site argues existing robotics benchmarks were not demanding enough: small workspaces, simple object sets, and narrow task distributions. RoboWild targets wild robot use: homes, workshops, laboratories, and stores with 3.60 objects per square meter, which the authors report is nearly twice the next densest benchmark in their comparison table (Gym-Anything at 1.82).

Coverage numbers from the official page:

  • 23 scene types versus single-scene or low-diversity suites.
  • 61.9% of tasks needat least two of eight measured skill types.
  • 65.7% requiredependency chains of two or more subgoals.
  • 58.6% needvisual disambiguation among similar objects.

Task examples on the demo page include mundane but hard chores: stack mail in a drawer, hang umbrellas on specific hooks, fold glasses onto a plate, open a locked door. Those are the tasks humans dismiss as "easy" until a robot must do them with three wrist cameras and a JSON tool interface.

If you tracked Waddle-1 on RoboDojo-RC this week, read RoboWild as the opposite stress test: not six real bimanual tasks on a leaderboard row, but hundreds of diverse scenes where 87% of runs disturb unrelated objects, per the site's safety statistics.

Leaderboard: Fable 5.2 first, everyone still low #

The published ReAct leaderboard on robowild.xlang.ai (five models, highest thinking per model) ranks:

Rank Model Weighted success
1 Claude Fable 5.2 13.3%
2 GPT-6 Astra 11.6%
3 Gemini 4 Carbon 10.6%
4 Claude Opus 5.5 6.5%
5 GPT-6.1 Sol 4.6%

The abstract on the same page notes token economics: Fable 5.2, Opus 5.5, and Carbon consume several times more output tokens than GPT-6.1 Sol and GPT-6 Astra per task. Routing logic that picks cheap models for coding may need a different policy for multi-minute robot episodes.

Anthropic's launch post already showed Fable 5.2 at 64.9% on RoboDojo Sim versus 39.4% for Opus 5.5 and 27.6% for Fable 5.1. RoboWild preserves the ordering (Fable above Opus) but crushes absolute success: the site plots Opus 5.5 at 50.00% on RoboDojo and 4.81% on RoboWild with 100 tool calls.

That is the headline for top Fable 5.2 demos too: impressive puzzle cubes and bottle caps in curated clips are not the same distribution as 341 wild tasks with grasp failures dominating.

Task design: perceptual and physical skill mix #

RoboWild publishes a skill requirement breakdown on robowild.xlang.ai that explains why 13% success is not a formatting error.

Perceptual demands (share of tasks requiring the skill):

  • Visual disambiguation : 58.6% — picking the right mug, hook, or letter among lookalikes.
  • Visual reasoning : 14.6% — layouts and spatial relations beyond OCR.
  • Text reading (OCR) : 13.4% — addresses on mail, labels on controls.
  • Memory : 5.0% — carrying state across subgoals.

Physical demands:

  • Articulation or material interaction : 53.6% — doors, dials, deformable items.
  • Precise manipulation : 48.5% — stacking, folding, placement tolerances.
  • Navigation : 28.5% — moving bases through clutter.
  • Bimanual manipulation : 21.3% — two-arm coordination.
  • Physical tool use : 12.6% — drills, stand mixers, vise-like objects listed in the interactive object gallery.

61.9% of tasks require at least two skill types; 0.8% require five or more. That composition is why computer-use agents trained on flat screenshots do not transfer cleanly: the observation stream is multi-camera, temporally extended, and physically consequential.

Harness: how a VLM actually drives the robot #

RoboWild documents a ReAct-style loop on robowild.xlang.ai:

  1. The model receives a task instruction ,robot-use guidance (frames, units, quaternions),tool schema , andthree images (head, left wrist, right wrist).
  2. It calls move_robot (and** done** ) — one tool per turn.
  3. The simulator advances and returns observation images until termination.

Embodiments share that interface across mobile dual-arm, Panda on a base, humanoid, and wheeled humanoid platforms. The benchmark is explicitly about cross-embodiment generalization, not a single arm in an empty bench.

Ablations mentioned on the site (not fully tabulated on the landing page) include adding depth and enabling code execution as ways to raise success. Builders replicating OSWorld-style computer-use stacks should assume robot-use needs similar harness work plus physics-side safety.

Failure modes: grasp, side effects, and humanoid pain #

RoboWild's results section is unusually blunt for a leaderboard launch:

  • Grasp failure :42.6% of the188 failed runs analyzed.
  • Motion or execution failure :28.7% .
  • Functional object operation failure :16.0% .
  • Goal or target selection error :6.9% .

Failure recovery rates differ by error class: Interface errors recover locally 78.7% of the time; Contact errors only 25.7%. Agents can retry UI-like mistakes more often than physical slips.

The safety and side effects block is what product teams should quote:

  • Unrelated objects disturbed :87% of runs.
  • Collision stops :78% .
  • Task objects dropped or tipped :75% .

That aligns with regulatory attention on robotaxi perception after California's Waymo freeway order: low success is not only task failure but collateral motion in cluttered spaces.

The site also states VLMs struggle on the humanoid embodiment specifically, while depth and code execution help in separate harness tests. If you evaluate Perceptron Isaac checkpoints for VLA policies, RoboWild is the complementary question: what happens when a general VLM steers the same hardware through a tool API?

RoboWild embodied AI evaluation benchmark comparing sim scores to wild robot tasks

How to try RoboWild assets and code #

Primary sources:

  • Site and leaderboard :robowild.xlang.ai
  • Paper PDF : linked from the site citation to Hugging Face (RoboWild.pdf on the dataset repo)
  • GitHub :xlang-ai/RoboWild (installation guide referenced from HF)
  • Dataset :xlangai/RoboWild —18.5 GB OpenUSD scenes, objects, robots, materials forIsaac Lab

Quick start from the dataset card:

git lfs install
git clone https://huggingface.co/datasets/xlangai/RoboWild robowild_assets
export ASSETS_DIR="$PWD/robowild_assets"

Then follow the RoboWild installation guide on GitHub for simulator setup. The repo layout lists scene01–scene38, reusable objects/, robots/ for all four embodiments, and shared materials/.

There is no substantive Hacker News thread yet under the RoboWild name (search checked November 21, 2026). For developer reactions on robot evals, see uni-robot-agent harness effects.

Benchmark landscape: density and scene breadth #

RoboWild's comparison table on the official site positions it against RoboCasa365, BEHAVIOR-1K, Embodied First Exam, LIBERO, RoboARCade, and RoboDojo on scene types and objects per square meter. RoboWild reports 23 scene types at 3.60 objects/m²; RoboCasa365 shows 1 scene type at 1.41 objects/m² in the same table.

That is not an attack on prior benchmarks — each suite serves a purpose. RoboDojo remains the simulation slice Anthropic uses in launch materials and RoboCurve adapts for real arms. RoboWild is the stress test for generalist VLMs that must pick tools, disambiguate clutter, and avoid side effects in furnished spaces.

Teams comparing AA-Robotics humanoid scores or Gemini Robotics factory demos should add RoboWild when the question is "can a frontier chat model operate my robot API?" rather than "does a specialized VLA policy track boxes?"

What to do with these numbers in production #

Practical takeaways if you ship robot-use or VLA features:

  1. Treat sub-15% success as thecurrent ceiling for general VLMs onwild tasks, not an outlier.
  2. Budget tokens and wall clock per episode; Fable-class modelsspend more output tokens per task on the official abstract.
  3. Log side effects , not only task success —87% disturbance rate is aliability metric.
  4. Compare harnesses , not raw model names: RoboDojo, RoboDojo-RC, and RoboWild usedifferent scenes, caps, and rubrics .
  5. Pair sim training withfurnished-scene eval before marketing "general robot intelligence."

For Fable versus Opus routing, RoboWild reinforces Fable for long-horizon physical tool loops — but only if your product tolerates cost per attempt and low absolute success today.

Frontier comparisons that include Gemini 4 Neon and GPT-6 Astra live in our November frontier table; RoboWild adds robot-use as a row you should not infer from coding benchmarks alone.

Leaderboard figures and failure statistics are accurate as of November 21, 2026 per robowild.xlang.ai and Anthropic's Fable 5.2 announcement.

── more in #robotics 4 stories · sorted by recency
── more on @xlang lab 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/robowild-benchmark-c…] indexed:0 read:8min 2026-11-21 · —