Editor’s note: This article was written by Lucia, a TechNode reporter.
Physical AI is moving beyond pre-programmed robots toward systems that can understand, decide, and adapt. But unlike digital AI, which can learn from vast amounts of online data, physical AI faces a shortage of real-world data needed to train models and improve robotic capabilities.
What happened: At its Kosmo AI Day in Shanghai on Oct. 14, Hesai launched Kosmo, a 3D world data engine designed to transform physical environments into high-fidelity 3D digital spaces for robot training, world model development and immersive content creation.
Why it matters: Kosmo is positioned as data infrastructure for Physical AI, addressing the challenge of obtaining real-world data that is accurate, affordable and available at scale.
How it works: Kosmo captures real spaces with lidar, imaging and motion data, then turns them into 3D scenes with AI-generated completion and object labeling.
- Kosmo adapts scenes for robot simulation and can generate over 10,000 training trajectories from a single capture.
- “Physical AI needs infrastructure that lets it understand the real world and act within it,” said Dr. Yifan “David” Li, Hesai’s co-founder and CEO.
In practice: Hesai showcased applications in robotics, film and media, cultural tourism and heritage, and spatial commerce.
- Robotics: In a Dairy Queen store demo with robot maker Sharpa, a 15-minute Kosmo scan replaced about 300 hours of manual modelling. Sharpa’s robot then achieved a 99% task-success rate in the reconstructed setting.
- Visual effects: Pixar’s RenderMan tutorial used a Kosmo scan of its campus plaza. The scene contained 1.25 million splats and worked in Houdini and RenderMan.
- Spatial commerce: Through its Amap partnership, Hesai has brought 3D reconstructions of the Gravity Art Museum and PIFO Gallery to Amap’s “Flying Street View,” with the goal of extending navigation from stores to products inside them.
Ecosystem: Kosmo supports open access to raw capture data and 3D assets, with integrations including NVIDIA Isaac Sim, Isaac Lab, Unreal Engine, RenderMan and Houdini. Hesai offers hardware-and-software solutions as well as spatial-data services and has partnered with Galbot and Dexmal to advance embodied AI.
Zoom out: Digital AI can draw on vast online datasets. Physical AI needs scenes that reflect the geometry and constraints of actual places. Kosmo is one attempt to turn those scenes into reusable assets for simulation and training.