{"slug": "nvidia-cosmos-framework-post-train-physical-ai-in-a-day", "title": "NVIDIA Cosmos Framework: Post-Train Physical AI in a Day", "summary": "NVIDIA open-sourced the Cosmos Framework, the training and inference toolkit for its Cosmos 3 world model family, shipping it with five bundled AGENTS.md agent skills that let coding agents such as Claude Code, Codex CLI, Cursor and Gemini CLI run setup, inference and post-training workflows. NVIDIA reported that LoRA post-training on the Woven Traffic Safety dataset raised exact-match video Q&A accuracy from a 54.41% baseline to 87.14%, and LoRA plus TAO AutoML across 43 Bayesian trials reached 93.35%, with a single LoRA epoch taking roughly 30 minutes on a node of A100 80 GB GPUs and using about 7x fewer GPU hours than full-parameter SFT. Cosmos 3 ships in 4B, 16B and 64B sizes, and NVIDIA positions the 16B Nano model as the practical post-training target for most teams.", "body_md": "NVIDIA open-sourced Cosmos Framework this week — the training and inference toolkit that sits underneath its physical AI model family. Combined with TAO 7’s new agent skills, the pitch is specific: post-train a world model on your own robot or vehicle data in a single day, using nothing but natural-language prompts to your coding agent. The numbers hold up. Here’s what actually changed for developers.\n\n## Cosmos Framework Is Not Cosmos 3\n\nCoverage of the Cosmos announcement has blurred two distinct things. Cosmos 3 is the model — a 4B/16B/64B omnimodal world model that understands video, images, text, and physical actions. Cosmos Framework is the toolbox: the inference scripts, training pipelines, and agent skills that let you actually work with those models.\n\nThe framework lives at [github.com/NVIDIA/cosmos-framework](https://github.com/NVIDIA/cosmos-framework). The model weights live on Hugging Face. You need both, but they serve different roles. If you want to run Cosmos 3 and adapt it to your data, you are working with the framework. That is the story here.\n\n## Coding Agents Are Now First-Class Citizens\n\nThe most interesting thing about cosmos-framework is not the training scripts — it is the AGENTS.md integration. NVIDIA shipped the repo with five bundled agent skills that make it natively readable by Claude Code, Codex CLI, Cursor, Gemini CLI, and any other AGENTS.md-aware tool.\n\nThe skills cover the entire workflow:\n\n- **cosmos3-setup** — environment install, checkpoint downloads, Docker\n- **cosmos3-inference** — offline and online inference, parallelism, sampling\n- **cosmos3-post-training** — SFT end-to-end: data prep, DCP conversion, training launch, export\n- **cosmos3-codebase-nav** — finding functions and parameters across the codebase\n- **cosmos3-env-troubleshoot** — CUDA errors, Docker issues, checkpoint failures\n\nWhen you open cosmos-framework in Claude Code and ask “set up Nano and run inference on this video,” the agent reads the cosmos3-setup skill and executes the right steps. You do not read the docs. The agent reads the docs. This is NVIDIA betting that in 2026, that is how developers actually work — and they are right to bet that way.\n\nThis is also the first major ML framework repo to ship a full AGENTS.md skill suite. Other repos will follow. Watch for this pattern.\n\n## One-Day Post-Training: The Numbers\n\nTAO 7’s agent skills automate the full post-training loop: data patching, baseline evaluation, training container execution, and hyperparameter sweeping via TAO AutoML. The developer interaction is natural language; the agent handles the pipeline.\n\nNVIDIA published real numbers using the Woven Traffic Safety dataset:\n\n- Baseline (no fine-tuning): **54.41%** exact-match video Q&A accuracy\n- After LoRA post-training: **87.14%**\n- After LoRA + TAO AutoML (43 Bayesian trials): **93.35%**\n\nA single LoRA epoch takes roughly 30 minutes on a node of A100 (80 GB) GPUs. LoRA also uses about 7x fewer GPU hours than full-parameter SFT — which is what makes the one-day timeline realistic for teams without a cluster. The accuracy jump from 54% to 87% is straightforward LoRA. The jump to 93% is AutoML sweeping learning rate, LoRA rank, dropout, batch size, and scheduler settings across 43 parallel trials. That second step is where the meaningful gains are.\n\nThe official tutorial is at the [NVIDIA developer blog post-training guide](https://developer.nvidia.com/blog/post-train-nvidia-cosmos-3-in-one-day-using-agent-skills/). The TAO skill bank is at [github.com/NVIDIA-TAO/tao-skill-bank](https://github.com/NVIDIA-TAO/tao-skill-bank).\n\n## Picking Your Model Size\n\nCosmos 3 ships in three sizes, each targeting a different hardware tier:\n\n- **Edge (4B):** Jetson AGX Orin and Thor, RTX PRO 6000 — on-device robot deployment\n- **Nano (16B):** RTX PRO 6000, H100, workstation-class — the practical post-training target for most teams\n- **Super (64B):** H200, B200, GB200 — data center scale synthetic data generation\n\nFor post-training experiments, Nano is the right starting point. It fits on a single H100 (80 GB) with FP8 quantization, and the TAO agent skills default to Nano. Edge is the deployment target once you have a trained checkpoint — it runs on Jetson Thor at the robot’s edge.\n\n## The License Is Not Apache 2.0 (and That Matters)\n\nCosmos Framework and the model weights ship under [OpenMDW-1.1](https://huggingface.co/blog/nvidia/cosmos-3-for-physical-ai), a license developed by NVIDIA and the Linux Foundation specifically to cover AI model weights — an artifact that standard open-source licenses were not designed for.\n\nWhat OpenMDW-1.1 permits: commercial use, modification, redistribution, and building commercial products on top. What it does not do: give you the same derivative-work freedom as Apache 2.0. It is not yet OSI-approved (submitted August 2026) and there are restrictions on relicensing derivatives that differ from Apache. Before shipping anything to production, read the license. For enterprise use cases with more complex requirements, NVIDIA offers custom terms at cosmos-license@nvidia.com.\n\nThe practical implication: you need to accept the OpenMDW-1.1 terms on each model’s Hugging Face page before the download link unlocks. It is a click, not a legal barrier — but do not skip it.\n\n## Who Should Actually Care\n\nThe honest answer is: not every developer. If you are building a standard LLM-powered application, Cosmos Framework is not relevant to your stack. This matters to teams building robots, autonomous vehicles, vision AI pipelines, or surgical simulation systems — and to ML engineers who want to generate synthetic training data for physical environments at scale.\n\nThe AGENTS.md pattern, though, matters broadly. NVIDIA just demonstrated that a large production ML repo can ship with agent skills that make it accessible to developers who would not otherwise have the depth to use it. That is the template other frameworks will copy. The question is which repo ships the second full AGENTS.md skill suite.", "url": "https://wpnews.pro/news/nvidia-cosmos-framework-post-train-physical-ai-in-a-day", "canonical_source": "https://byteiota.com/nvidia-cosmos-framework-post-train-physical-ai-in-a-day/", "published_at": "2026-11-06 03:14:07+00:00", "updated_at": "2026-11-06 03:52:28.991321+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-tools", "robotics", "developer-tools"], "entities": ["NVIDIA", "Cosmos Framework", "Cosmos 3", "TAO 7", "Claude Code", "Codex CLI", "Cursor", "Gemini CLI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/nvidia-cosmos-framework-post-train-physical-ai-in-a-day", "markdown": "https://wpnews.pro/news/nvidia-cosmos-framework-post-train-physical-ai-in-a-day.md", "text": "https://wpnews.pro/news/nvidia-cosmos-framework-post-train-physical-ai-in-a-day.txt", "jsonld": "https://wpnews.pro/news/nvidia-cosmos-framework-post-train-physical-ai-in-a-day.jsonld"}}