Ropedia Xperience 2: 1,589 Scenes and Egocentric Video for Physical AI Scaling

Ropedia released Xperience 2 on November 12, 2026, a physical AI dataset of 1,589 metric 3D-scanned scenes and 4,230 egocentric recordings across 74 tasks totaling 131 hours, detailed in arXiv paper 2611.17060 and an Apache-2.0 xperience-register CLI on GitHub. The team argues robot foundation models still fail on long-horizon and contact-rich home tasks, and that capturing human behavior in real rooms once and reusing it across simulation, teleop and evaluation is the scaling path for physical AI. The full dataset drop was listed as coming soon on the project page at fetch time, with the Hugging Face organization ropedia-ai named as the portal.

Ropedia Xperience 2 attacks a bottleneck that still breaks "GPT-class" robot policies on real furniture and contact: you need both human behavior demos and reusable rooms in one coordinate frame. The team behind S-Agent 2.0 https://explainx.ai/blog/s-agent-2-0-spatial-reasoning-general-coding-agent-perception-cli-2026 published arXiv:2611.17060 https://arxiv.org/abs/2611.17060?ref=explainx on November 12, 2026 , opened a GitHub registration repo https://github.com/Ropedia/Ropedia-Xperience-2?ref=explainx , and documented benchmarks on its project page https://ropedia.com/xperience/2-research/?ref=explainx . The release argues physical AI needs something like language-model scaling laws , but for environments and behaviors first — which data to grow, and how to recycle each capture into sim, retargeting, and evaluation. TL;DR | Asset | Scale per arXiv abstract and project page | |---|---| | Metric 3D scans | 1,589 released scenes | | Egocentric recordings | 4,230 takes across 74 tasks 131 hours | | Capture rig | HOMIE Gen 2 head mount 4 fisheye cameras, IMU, 4-channel audio plus handheld LiDAR room scan | | Code | Apache-2.0 xperience-register CLI on GitHub | | Paper | arXiv 2611.17060 https://arxiv.org/abs/2611.17060?ref=explainx , submitted Nov 12, 2026 | | Dataset portal | Hugging Face organization ropedia-ai ; project page listed full drop as coming soon at fetch time | Why the team says robot data is stuck The abstract on arXiv https://arxiv.org/abs/2611.17060?ref=explainx opens with a familiar 2026 premise: large foundation models the paper names GPT-6 Astra and Claude Fable 5.1 can steer robots from pixels, yet still fail on long horizons and contact-rich home tasks. It contrasts that fragility with LLM scaling, where data and compute curves are well studied. For physical AI , Ropedia asks which environment–behavior pairs should scale, and how to capture once and reuse across simulation, teleop, and automated trajectory synthesis. The Hugging Face discussion post https://huggingface.co/papers/2611.17060?ref=explainx from submitter lifuguan gives a concrete cost anchor: DROID 's ~350 hours of demonstrations required 50 operators for a year , and real hardware cannot cheaply retry failures the way sim can. Xperience 2 therefore starts from people in real rooms , then derives robot-ready assets. That framing matches industry moves we track elsewhere: OpenAI disbanding its robotics team https://explainx.ai/blog/openai-disbands-robotics-team-gpt-6-astra-robots-fortune-2026 while other labs double down on world models Nvidia BirdDreams https://explainx.ai/blog/nvidia-birddreams-driving-world-model-bev-perspective-2026 and sim-to-real pick-and-place Nvidia SimReady with Isaac Sim 6.2 https://explainx.ai/blog/nvidia-simready-central-pick-and-place-isaac-sim-6-2-gpt-6-sol-agent-ur17g-robotiq-2026 . Capture pipeline: scan once, record many tasks The Ropedia Xperience 2 project page https://ropedia.com/xperience/2-research/?ref=explainx walks through a repeatable field workflow: 1. Room scan — A handheld LiDAR pass builds a metric-scale 3D scene once. 2. Task recordings — The operator wears HOMIE Gen 2 and performs multiple chores in that same room examples shown include wiping a dining table, microwaving food, and AC remote reach tasks . 3. Registration — Video, IMU, audio, and scan share one gravity-aligned frame , so downstream code can treat human motion as trajectories inside the measured room. Interactive figures on the page show 3D Gaussian splat renders beside raw head-camera frames, with sub-task and action labels synchronized. This is a different bet from one-demonstration VDA systems like Machenta https://explainx.ai/blog/machenta-instant-robot-learning-one-demonstration-vda-model-40m-seed-trained-in-simulation-claimed-vs-verified-2026 , which optimize sample efficiency on a single robot morphology. Xperience 2 optimizes throughput of diverse rooms and long daily routines , closer to how Work2Skill https://explainx.ai/blog/work2skill-human-gui-demonstrations-reusable-agent-skills-paper-2026 mines human demos — but here the domain is embodied household geometry , not desktop GUIs. What you can download and run today GitHub: scene registration The Ropedia/Ropedia-Xperience-2 https://github.com/Ropedia/Ropedia-Xperience-2?ref=explainx README centers on scene registration : pip install -e . xperience-register --kosmo /path/to/scene --homie /path/to/recording --out results/take The tool aligns a processed HOMIE recording to a KOSMO scene export and writes trajectories plus estimated hand and body motion in the scene frame. The repo lists Python 3.10–3.12 , includes docs , examples , tests , and cites the project URL in its BibTeX stub. License split: code is Apache-2.0 ; the README points to separate data access terms typical for large embodied releases . Paper and Hugging Face entry - Primary write-up: arXiv abs/2611.17060 https://arxiv.org/abs/2611.17060?ref=explainx - Community mirror: Hugging Face Papers page https://huggingface.co/papers/2611.17060?ref=explainx 26 upvotes at candidate time The project page still showed Dataset / Paper / Code badges as "soon" for the full Hugging Face drop even though the arXiv and GitHub pieces were live — worth checking both URLs before you plan a cluster job. Benchmarks bundled with Xperience 2 The project page is not just a dataset card; it ships three evaluation lanes with leaderboards: Depth and geometry 11 models Eleven reconstruction or depth models are compared on the scanned rooms the UI groups them under Depth and Geometry . This matters because misregistered scans poison every downstream policy. 3D object detection 7 methods Magnifying lens over a detected object icon for metric 3D box benchmarks on scanned home scenes On 20 annotated scenes , humans labeled category and 3D boxes for all objects. Methods receive only categories present and must return metric 3D boxes in the scene's gravity frame. Metrics include AP25 , AP50 , and mIoU@0.25 . The page credits Boxer as strongest, lifting open-vocabulary 2D detections to 3D boxes — a pattern reminiscent of spatial coding agents, but here evaluated on real homes instead of synthetic meshes only. Spatial intelligence QA 500 questions Embodied AI robot hand approaching a metric world model built from egocentric scan data Six question types probe relational layout object identity, distance, direction . Gemini-2.5-Pro leads at 74.2% accuracy on the page's table; HoliWorld-VL-4B at 67.8% beats listed open VLMs and GPT-5 High at 65.8% . That table is a useful sanity check for builders who use VLMs as planners over scanned apartments — the gap between closed and open models is still wide on metric reasoning . Long-horizon sim policies A highlight reel shows a mobile manipulator with dual hands completing seven subtasks in a simulated office derived from a scan: fetching cups, operating a water dispenser, pouring, and desk delivery. The page reports 7/7 subtasks passing 43 checks in 2.9 minutes continuous time, with 135 ml poured within ~10% tolerance per cup. Another clip tests eight subtasks across a home microwave to bedroom from 30 held-out starts , reporting 30/30 completions at 2.2 minute average trajectories with 10–16 minutes compute per trajectory during generation. Those numbers are project-reported on the demo page, not independent replication — but they show the intended use: policies authored as code in sim scenes cloned from human-centered captures . How this fits next to Nvidia, Meta, and ByteDance robot stacks | Lane | Example in our coverage | How Xperience 2 differs | |---|---|---| | Industrial sim assets | Nvidia SimReady + Isaac Sim 6.2 https://explainx.ai/blog/nvidia-simready-central-pick-and-place-isaac-sim-6-2-gpt-6-sol-agent-ur17g-robotiq-2026 | Xperience 2 emphasizes in-the-wild homes and human egocentric capture, not factory UR cells | | Video world models | PhysLoop LLM physics video https://explainx.ai/blog/physloop-llm-physics-video-generation-byte-dance-pku-2026 | Xperience 2 is interactive sim-ready geometry , not generative video alone | | Agent skills from demos | Work2Skill GUI traces https://explainx.ai/blog/work2skill-human-gui-demonstrations-reusable-agent-skills-paper-2026 | Targets 3D embodiment and contact , not browser automation | | Spatial coding agents | S-Agent 2.0 CLI https://explainx.ai/blog/s-agent-2-0-spatial-reasoning-general-coding-agent-perception-cli-2026 | Adds dataset + registration under the same Ropedia brand | If you are picking research repos this month, read S-Agent for agent harness design , Xperience 2 for data multiplication , and vendor sim posts for deployment integrators . Try it: minimal integration path 1. Clone github.com/Ropedia/Ropedia-Xperience-2 https://github.com/Ropedia/Ropedia-Xperience-2?ref=explainx and install in a 3.10+ venv. 2. Run the synthetic example in examples/ to validate IO without waiting for the full HF drop. 3. Read docs/ for KOSMO and HOMIE path conventions before pointing at real captures. 4. Skim arXiv PDF https://arxiv.org/pdf/2611.17060?ref=explainx sections on retargeting and virtual teleop to see how human clips become robot trajectories. 5. When the HF dataset opens, treat benchmarks as regression tests for your own VLA or code-as-policy stack — similar to how teams used S-Agent benchmarks https://explainx.ai/blog/s-agent-2-0-spatial-reasoning-general-coding-agent-perception-cli-2026 for spatial reasoning. Numbers next to DROID and Open X-Embodiment The Hugging Face discussion frames Xperience 2 against DROID scale: hundreds of hours collected with dozens of operators across many labs. Xperience 2's 131 hours across 4,230 clips is smaller in raw hours but denser per scene — multiple tasks in the same metric room, all registered. For teams already mixing open robot corpora with VLAs , the practical question is whether your trainer consumes: - Ego video + language only cheap, weak geometry - Registered point clouds + actions Xperience 2's pitch - Teleop on real arms accurate dynamics, expensive None replaces the others; Xperience 2 optimizes the middle path before you commit a UR arm to every kitchen. HOMIE Gen 2 and KOSMO scene exports The marketing site lists HOMIE Gen 2 with four fisheye cameras , IMU , and four-channel audio , plus a handheld LiDAR pass for the static scan. The GitHub CLI expects processed HOMIE takes and KOSMO scene exports — naming conventions and preprocessing live in docs/ inside the repo. If you only have phone video today, you cannot snap-in this pipeline without hardware; plan captures before you promise sim parity to robotics stakeholders. Citation block When the arXiv entry stabilizes, the repository suggests: @misc{ropedia2026xperience2, title={Ropedia Xperience 2: A Gold Standard for Scaling Physical AI Environments and Behaviors}, author={{Ropedia Research Team}}, year={2026}, url={https://ropedia.com/xperience/2-research/} } Link both the arXiv DOI 10.48550/arXiv.2611.17060 https://doi.org/10.48550/arXiv.2611.17060?ref=explainx and the GitHub commit you reproduced against. Open questions and limitations - Dataset availability: Project page badges said coming soon for the full Hugging Face corpus during our fetch; verify before planning multi-terabyte downloads. - Hardware lock-in: HOMIE Gen 2 is specific; registration may not generalize to arbitrary headsets without adaptation. - Sim-to-real gap: Passing sim checks on scanned offices does not guarantee robustness on your warehouse robot form factor — retargeting quality will vary by morphology. - Leaderboard freshness: VLM scores on the marketing page will drift as new models ship; re-run eval locally when you adopt the dataset.