Nvidia Releases Cosmos 3 Family, Four Open-Weight Nano Checkpoints for Interactive World Simulation

Nvidia released four open-weight checkpoints in its new Cosmos 3 family, all post-trained variants of the 16-billion-parameter Cosmos 3 Nano foundation model, under the Linux Foundation's OpenMDW-1.1 license that permits commercial and non-commercial use. The checkpoints — Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer and Cosmos3-Nano-Sim-Bimanual — target interactive world simulation for robotics and autonomous driving, generating multi-view camera video (up to 11 views) and LiDAR, streaming closed-loop driving observations, converting simulator spatial controls to photorealistic RGB, and predicting bimanual robot observations. Nvidia calls Cosmos 3 the first fully open omni-model for physical AI, built on a Mixture-of-Transformers architecture that handles language, image, video, audio and action in one model, with Edge (4B total, based on a 2B transformer) among three defined scales.

Nvidia Releases Cosmos 3 Family, Four Open-Weight Nano Checkpoints for Interactive World Simulation Nvidia has published four open-weight checkpoints under its new Cosmos 3 platform, targeting researchers building interactive world simulators for robotics and autonomous driving. The models—Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer, and Cosmos3-Nano-Sim-Bimanual—are post-trained variants of the 16-billion-parameter Cosmos 3 Nano foundation model, released under the Linux Foundation's OpenMDW-1.1 license \ 7\ https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf , which permits commercial and non-commercial use \ 1\ https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation . Nvidia describes Cosmos 3 as the first fully open omni-model for physical AI \ 5\ https://huggingface.co/nvidia , a claim that rests on the model's unified handling of language, image, video, audio, and action within a single Mixture-of-Transformers MoT architecture \ 7\ https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf . Four Checkpoints, Four Simulation Tasks Each checkpoint addresses a different simulation workflow: - Transfer-Auto generates synchronized multi-view camera video up to 11 views and LiDAR from structured driving scenarios and text prompts, intended for offline synthetic data generation \ 1\ https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation . - Sim-Auto is a causally distilled, few-step streaming variant of Transfer-Auto that produces successive chunks of camera and LiDAR observations for closed-loop driving simulation \ 1\ https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation . - Sim-Transfer converts simulator-produced spatial controls depth, segmentation, blur plus a text prompt into photorealistic RGB video, leaving physics and ground-truth state to an external simulator \ 1\ https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation . - Sim-Bimanual predicts the next visual observation from a bimanual robot's actions, supporting YAM and AgiBot embodiments controlled by policy or human-operated joysticks \ 1\ https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation . All four share a four-stage post-training recipe: bidirectional adaptation on full clips, teacher-forcing causal training, consistency distillation, and self-forcing distribution-matching distillation on the student's own rollouts \ 3\ https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation/technical-report.pdf . The consistency distillation stage trains a student that generates each chunk in a few denoising steps; the optimized Bimanual setup predicts two latent time steps per chunk \ 1\ https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation . The technical report notes that policy-driven closed-loop validation remains separate from the reported benchmarks \ 3\ https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation/technical-report.pdf . Architecture: One Model, Multiple Modes Cosmos 3 uses a MoT backbone with separate reasoner and generator towers that share a unified representation space \ 7\ https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf . Language tokens are processed autoregressively by the reasoner; vision, audio, and action tokens are diffused by the generator, with joint attention allowing the generator to condition on the reasoner's output \ 7\ https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf . The same weights can operate as a vision-language model, text-to-image/video generator, video-transfer model, forward/inverse dynamics model, or joint video-action policy by changing which tokens are clean or noisy at inference \ 7\ https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf . Three scales are defined: Edge 4B total, based on a 2B transformer , Nano 16B total, based on an 8B transformer , and Super 64B total, based on a 32B transformer \ 7\ https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf . Only Nano and Super checkpoints are released so far; Edge is slated for a later release \ 7\ https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf . The four new checkpoints are all Nano-scale, 16B-parameter models \ 1\ https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation . Licensing and Developer Tooling Code, model checkpoints, curated synthetic datasets, and evaluation benchmarks are released under OpenMDW-1.1 \ 7\ https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf . Nvidia also provides Cosmos Framework—an open, modular codebase for training, post-training including LoRA, SFT, GRPO, PPO, OPD, DiffusionNFT , optimization distillation, quantization , and inference across Hugging Face Diffusers, PyTorch, SGLang, vLLM-Omni, vLLM, TRT-LLM, and TRT-Edge-LLM \ 2\ https://developer.nvidia.com/blog/build-world-models-for-interactive-simulation-with-nvidia-cosmos . For production deployment, Cosmos NIM microservices offer optimized containers with speculative decoding measured 2–3× speedup on H200 , quantization, and KV-cache-aware routing \ 2\ https://developer.nvidia.com/blog/build-world-models-for-interactive-simulation-with-nvidia-cosmos . The NIM path requires an NGC API key and Docker; the open Framework path runs on Linux with an Nvidia GPU, Docker, and the Nvidia Container Toolkit \ 1\ https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation \ 2\ https://developer.nvidia.com/blog/build-world-models-for-interactive-simulation-with-nvidia-cosmos . Benchmark Claims and Independent Verification The Cosmos 3 technical report states: "Our post-trained Cosmos 3 models were ranked as the best open-source Text-to-Image and Image-to-Video models by Artificial Analysis, and the best policy model by RoboArena at the time the technical report was written" \ 7\ https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf . These rankings reflect the state of those leaderboards at the time of writing. Table 1 in that report shows Cosmos3-Super leads on Text-to-Image 91.36 vs. 90.85 for Gemini 3 Pro Image , while both Super and Nano outperform baselines on Text-to-Video 80.0/79.4 vs. 79.1 for Veo-3.1 and robot forward dynamics 26.0/25.5 vs. 23.0 for Ctrl-World \ 7\ https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf . These results come from Nvidia's own evaluations in the technical report. What Remains Unknown - The "200+ releases" figure cited in Nvidia's broader open-data catalog spans Physical AI, robotics, autonomous vehicles, biology, drug discovery, retrieval and evaluation benchmarks, and sovereign AI—not exclusively Cosmos 3 checkpoints \ 5\ https://huggingface.co/nvidia . - Hardware minimums for the open Framework are not specified; the published vLLM-Omni setup was tested on a single GB200, explicitly noted as "not a minimum-memory specification" \ 1\ https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation . - Cosmos3-Edge weights are not yet available, and no timeline is given. - Closed-loop policy evaluation results for the four new checkpoints are not included in the technical report \ 3\ https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation/technical-report.pdf . Nvidia is hosting a livestream on November 5 to demonstrate the Framework \ 2\ https://developer.nvidia.com/blog/build-world-models-for-interactive-simulation-with-nvidia-cosmos . The checkpoints, technical report, and Framework code are accessible via Hugging Face and GitHub \ 1\ https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation \ 3\ https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation/technical-report.pdf \ 6\ https://github.com/NVIDIA/cosmos-framework . Sources 1. New NVIDIA Cosmos 3 Open Models for Interactive World ... https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation huggingface.co 2. Build World Models for Interactive Simulation with NVIDIA ... https://developer.nvidia.com/blog/build-world-models-for-interactive-simulation-with-nvidia-cosmos developer.nvidia.com 3. \ PDF\ Interactive World Simulation with Cosmos for Physical AI https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation/technical-report.pdf research.nvidia.com 4. NVIDIA Open Models for Physical AI webinar https://www.nvidia.com/en-us/events/webinars/aps-open-models-for-physical-ai nvidia.com 5. nvidia NVIDIA https://huggingface.co/nvidia huggingface.co 6. GitHub - NVIDIA/cosmos-framework: Our inference and training framework to run on the Cosmos Models · GitHub https://github.com/NVIDIA/cosmos-framework github.com 7. Cosmos World Foundation Model Platform for Physical AI https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf research.nvidia.com