Nvidia has published four open-weight checkpoints under its new Cosmos 3 platform, targeting researchers building interactive world simulators for robotics and autonomous driving. The models—Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer, and Cosmos3-Nano-Sim-Bimanual—are post-trained variants of the 16-billion-parameter Cosmos 3 Nano foundation model, released under the Linux Foundation's OpenMDW-1.1 license [7], which permits commercial and non-commercial use [1]. Nvidia describes Cosmos 3 as the first fully open omni-model for physical AI [5], a claim that rests on the model's unified handling of language, image, video, audio, and action within a single Mixture-of-Transformers (MoT) architecture [7].
Four Checkpoints, Four Simulation Tasks #
Each checkpoint addresses a different simulation workflow:
- Transfer-Auto generates synchronized multi-view camera video (up to 11 views) and LiDAR from structured driving scenarios and text prompts, intended for offline synthetic data generation[1] .
- Sim-Auto is a causally distilled, few-step streaming variant of Transfer-Auto that produces successive chunks of camera and LiDAR observations for closed-loop driving simulation[1] .
- Sim-Transfer converts simulator-produced spatial controls (depth, segmentation, blur) plus a text prompt into photorealistic RGB video, leaving physics and ground-truth state to an external simulator[1] .
- Sim-Bimanual predicts the next visual observation from a bimanual robot's actions, supporting YAM and AgiBot embodiments controlled by policy or human-operated joysticks[1] .
All four share a four-stage post-training recipe: bidirectional adaptation on full clips, teacher-forcing causal training, consistency distillation, and self-forcing distribution-matching distillation on the student's own rollouts [3]. The consistency distillation stage trains a student that generates each chunk in a few denoising steps; the optimized Bimanual setup predicts two latent time steps per chunk [1]. The technical report notes that policy-driven closed-loop validation remains separate from the reported benchmarks [3].
Architecture: One Model, Multiple Modes #
Cosmos 3 uses a MoT backbone with separate reasoner and generator towers that share a unified representation space [7]. Language tokens are processed autoregressively by the reasoner; vision, audio, and action tokens are diffused by the generator, with joint attention allowing the generator to condition on the reasoner's output [7]. The same weights can operate as a vision-language model, text-to-image/video generator, video-transfer model, forward/inverse dynamics model, or joint video-action policy by changing which tokens are clean or noisy at inference [7].
Three scales are defined: Edge (4B total, based on a 2B transformer), Nano (16B total, based on an 8B transformer), and Super (64B total, based on a 32B transformer) [7]. Only Nano and Super checkpoints are released so far; Edge is slated for a later release [7]. The four new checkpoints are all Nano-scale, 16B-parameter models [1].
Licensing and Developer Tooling #
Code, model checkpoints, curated synthetic datasets, and evaluation benchmarks are released under OpenMDW-1.1 [7]. Nvidia also provides Cosmos Framework—an open, modular codebase for training, post-training (including LoRA, SFT, GRPO, PPO, OPD, DiffusionNFT), optimization (distillation, quantization), and inference across Hugging Face Diffusers, PyTorch, SGLang, vLLM-Omni, vLLM, TRT-LLM, and TRT-Edge-LLM [2]. For production deployment, Cosmos NIM microservices offer optimized containers with speculative decoding (measured 2–3× speedup on H200), quantization, and KV-cache-aware routing [2]. The NIM path requires an NGC API key and Docker; the open Framework path runs on Linux with an Nvidia GPU, Docker, and the Nvidia Container Toolkit [1][2].
Benchmark Claims and Independent Verification #
The Cosmos 3 technical report states: "Our post-trained Cosmos 3 models were ranked as the best open-source Text-to-Image and Image-to-Video models by Artificial Analysis, and the best policy model by RoboArena at the time the technical report was written" [7]. These rankings reflect the state of those leaderboards at the time of writing. Table 1 in that report shows Cosmos3-Super leads on Text-to-Image (91.36 vs. 90.85 for Gemini 3 Pro Image), while both Super and Nano outperform baselines on Text-to-Video (80.0/79.4 vs. 79.1 for Veo-3.1) and robot forward dynamics (26.0/25.5 vs. 23.0 for Ctrl-World) [7]. These results come from Nvidia's own evaluations in the technical report.
What Remains Unknown #
- The "200+ releases" figure cited in Nvidia's broader open-data catalog spans Physical AI, robotics, autonomous vehicles, biology, drug discovery, retrieval and evaluation benchmarks, and sovereign AI—not exclusively Cosmos 3 checkpoints [5] .
- Hardware minimums for the open Framework are not specified; the published vLLM-Omni setup was tested on a single GB200, explicitly noted as "not a minimum-memory specification" [1] .
- Cosmos3-Edge weights are not yet available, and no timeline is given.
- Closed-loop policy evaluation results for the four new checkpoints are not included in the technical report [3] .
Nvidia is hosting a livestream on November 5 to demonstrate the Framework [2]. The checkpoints, technical report, and Framework code are accessible via Hugging Face and GitHub [1][3][6].
Sources #
1. [New NVIDIA Cosmos 3 Open Models for Interactive World ...](https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation) (huggingface.co)
2. [Build World Models for Interactive Simulation with NVIDIA ...](https://developer.nvidia.com/blog/build-world-models-for-interactive-simulation-with-nvidia-cosmos) (developer.nvidia.com)
3. [\[PDF\] Interactive World Simulation with Cosmos for Physical AI](https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation/technical-report.pdf) (research.nvidia.com)
4. [NVIDIA Open Models for Physical AI webinar](https://www.nvidia.com/en-us/events/webinars/aps-open-models-for-physical-ai) (nvidia.com)
5. [nvidia (NVIDIA)](https://huggingface.co/nvidia) (huggingface.co)
6. [GitHub - NVIDIA/cosmos-framework: Our inference and training framework to run on the Cosmos Models · GitHub](https://github.com/NVIDIA/cosmos-framework) (github.com)
7. [Cosmos World Foundation Model Platform for Physical AI](https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf) (research.nvidia.com)