{"slug": "show-hn-simulusion-a-pytorch-library-for-training-and-running-world-models", "title": "Show HN: Simulusion – A PyTorch library for training and running world models", "summary": "Developer Param Thakkar released Simulusion 1.0, a PyTorch library formerly named TorchWM, on PyPI under the package name `simulusion`, unifying Dreamer, PlaNet, JEPA, IRIS, DIAMOND, DiT and Genie world models behind a single `create_config` / `create_model` / `make_env` API. The library ships built-in adapters for Gym/Gymnasium, DeepMind Control, MuJoCo, Brax, Atari and Unity ML-Agents, plus ONNX, TorchScript and TensorRT export, and every model includes a demo that trains from scratch on a 4 GB RTX 3050 laptop GPU in one to two hours. Trained Dreamer, DreamerV2, PlaNet, IRIS, DiT and I-JEPA checkpoints are published in the Simulusion demo models collection on the Hugging Face Hub.", "body_md": "**Formerly TorchWM.** The project was renamed to Simulusion for 1.0 and is\npublished on PyPI as `simulusion`: `pip install simulusion`, then `import simulusion`.\n\n**Simulusion brings the major world-model families together under a single PyTorch API.** Train Dreamer, PlaNet, JEPA, IRIS, DIAMOND, DiT, and Genie agents through `create_config` / `create_model` / `make_env`, or drop down to their encoders, decoders, and latent-dynamics backbones to compose your own architecture. Environment adapters (Gym/Gymnasium, DeepMind Control, MuJoCo, Brax, Atari, Unity ML-Agents) and ONNX / TorchScript / TensorRT export come built in.\n\nEvery model ships with a demo that trains from scratch on a laptop GPU (RTX 3050, 4 GB) in one to two hours and records what it learned. These are short runs, not paper-scale results: the point is to show what each model does.\n\n| **Dreamer** : the real environment (left) beside the world model imagining it open loop (right). | **DIAMOND** : Breakout in the emulator beside a diffusion model generating every frame itself. | \n| **Genie** : a real Sonic clip beside Genie regenerating it from the first frame, with actions it learned from unlabelled video. | **IRIS** : a policy boxing inside a Transformer world model made of discrete tokens. | \n| **PlaNet** : a latent model predicting the future while a CEM planner searches inside it. | **DiT** : CIFAR-10 samples from a class-conditional Diffusion Transformer, one row per class. | \n\nMore clips, learning curves and the commands to reproduce each one are in the\n[gallery](https://paramthakkar123.github.io/simulusion/gallery.html); the scripts live in [`demos/`](https://github.com/ParamThakkar123/simulusion/blob/main/demos).\nThe trained Dreamer, DreamerV2, PlaNet, IRIS, DiT and I-JEPA checkpoints are on\nthe Hugging Face Hub in the\n[Simulusion demo models](https://huggingface.co/collections/simillusion/simulusion-demo-models-6ae0419992e0177bc1695362)\ncollection; each model card shows how to load it.\n\n```\n# Install the core package from PyPI.\n# This keeps environment integrations and experiment logging optional.\npip install simulusion\n\n# With extras\npip install simulusion[gym]       # Gym/Gymnasium environments (runnable quick start)\npip install simulusion[dmc]       # DeepMind Control Suite (walker-walk, cheetah-run, ...)\n                               # On CPython 3.13 also run: python -m simulusion.install_dmc\npip install simulusion[worldmodels] # Classic World Models (ConvVAE + CMA-ES controller)\npip install simulusion[ml-agents] # Unity ML-Agents\npip install simulusion[ml]        # TensorBoard, W&B logging\npip install simulusion[viz]       # Latent-space visualization (plotly, UMAP)\npip install simulusion[dev]       # Testing and linting\n\n# Or add it to a uv-managed project.\nuv add simulusion\n```\n\nSimulusion depends on PyTorch but does not force a single PyTorch wheel index. If you need a specific PyTorch build, install or add the PyTorch packages with the index recommended for your platform by the [PyTorch installation selector](https://pytorch.org/get-started/locally/):\n\n```\n# Example: CUDA 12.1 wheels. Choose a different index for CPU, ROCm, CUDA 11.x, CUDA 12.4+, or macOS.\nuv add torch torchvision torchaudio --index https://download.pytorch.org/whl/cu121\n```\n\nUse the friendly top-level API for the common path. The example below runs on a\nbase `pip install simulusion[gym]` — no simulator downloads required:\n\n``` python\nimport simulusion\n\n# Trains a Dreamer agent on a Gymnasium task. Bump `total_steps` for real runs.\n# `seed_steps` of random play come first and count towards `total_steps`; the\n# final checkpoint is written to `<logdir>/ckpts/` when training finishes.\nagent = simulusion.create_model(\n    \"dreamer\",\n    env=\"Pendulum-v1\",\n    env_backend=\"gym\",\n    seed_steps=1_000,\n    total_steps=10_000,\n)\nagent.train()\n```\n\nTo train on DeepMind Control tasks such as `walker-walk`, install the DMC extra\n(`pip install simulusion[dmc]`) and use the default backend:\n\n```\nagent = simulusion.create_model(\"dreamer\", env=\"walker-walk\", total_steps=1_000_000)\nagent.train()\n```\n\nEvery algorithm in the table below is reachable through the same factory, so comparing them is a loop rather than a rewrite:\n\n``` python\nimport simulusion\n\nfor algo in [\"dreamer-v1\", \"dreamer-v2\"]:\n    agent = simulusion.create_model(\n        algo, env=\"Pendulum-v1\", env_backend=\"gym\", total_steps=20_000\n    )\n    agent.train()\n```\n\n`examples/algorithm_comparison.py` runs exactly this and writes a comparison\nplot.\n\nEvery family also trains on one shared `Trainer` from a YAML run config.\nDevice, precision, resuming, evaluation and checkpoints work the same for all\nof them, and every run\nwrites its resolved `config.yaml` and `run.json` (git commit, package\nversions) so it can be repeated:\n\n```\nsimulusion train --config dreamer model.env=cheetah-run trainer.precision=bf16\nsimulusion train --config iris game=ALE/Breakout-v5\nsimulusion train genie data.path=data/clips trainer.max_steps=20000\nsimulusion train --config runs/<a previous run>/config.yaml     # repeat it\n```\n\nWorld models also train **without their agent**, from recorded episodes,\nincluding robot data converted from LeRobot, and any\nagent that speaks the `WorldModel` protocol can then plan with them:\n\n``` python\nsimulusion datasets import lerobot datasets/pusht data/pusht\nsimulusion train --config dreamer-wm data.path=data/pusht\n```\n\nSee the [trainer guide](https://paramthakkar123.github.io/simulusion/trainer_guide.html)\nand the [data guide](https://paramthakkar123.github.io/simulusion/episodes_guide.html).\n\n- Unified interfaces across world-model algorithms: one `WorldModel` protocol,\none`Trainer` , one YAML run config per experiment\n- One episode format for video, RL and robot data, with a LeRobot reader\n- Modular encoders, decoders, dynamics models, and backbones\n- Training and inference utilities for model-based reinforcement learning\n- Environment integrations for Gym/Gymnasium, Unity ML-Agents, MuJoCo, Brax, and robotics extras\n- Optional logging, visualization, development, and documentation extras\n\n```\nflowchart LR\n    subgraph API[\"simulusion API\"]\n        CFG[\"create_config()\"]\n        MDL[\"create_model()\"]\n        ENV[\"make_env()\"]\n    end\n\n    subgraph CONFIGS[\"Configs\"]\n        DC[\"DreamerConfig\"]\n        JC[\"JEPAConfig\"]\n        IC[\"IRISConfig\"]\n        GC[\"GenieConfig\"]\n        DIC[\"DiTConfig / DiamondConfig\"]\n    end\n\n    subgraph AGENTS[\"Agents / Models\"]\n        DR[\"Dreamer / DreamerV1 / DreamerV2\"]\n        JP[\"JEPAAgent\"]\n        IR[\"IRISAgent\"]\n        GN[\"Genie\"]\n        DT[\"DiT / DIAMOND\"]\n    end\n\n    subgraph BACKBONES[\"Backbones\"]\n        RSSM[\"RSSM / ModularRSSM\"]\n        VIT[\"VisionTransformer\"]\n        VQ[\"VQ-VAE / VideoTokenizer\"]\n        ST[\"STTransformer\"]\n        DIF[\"DDPM / DiT diffusion\"]\n    end\n\n    subgraph ENVS[\"Environments\"]\n        GYM[\"Gym / Atari\"]\n        DMC[\"DeepMind Control\"]\n        MJ[\"MuJoCo\"]\n        BR[\"Brax\"]\n        UN[\"Unity ML-Agents\"]\n        ROB[\"Robotics\"]\n        more[\"...\"]\n    end\n\n    subgraph EXPORT[\"Export\"]\n        ONNX[\"ONNX\"]\n        TS[\"TorchScript\"]\n        TRT[\"TensorRT\"]\n    end\n\n    CFG --> CONFIGS\n    MDL --> AGENTS\n    ENV --> ENVS\n    AGENTS --> BACKBONES\n    AGENTS -.-> ENVS\n    AGENTS --> EXPORT\n```\n\nEvery row is a registry entry — pass the name straight to `simulusion.create_model(...)`\nor `simulusion.create_config(...)`. Run `simulusion.list_models()` for the live list.\n\n| Name | Algorithm | Description | Key Features | \n|---|---|---|---|\n| `dreamer` | **Dreamer** | Model-based RL with latent dynamics (alias for `dreamer-v1` ) | Imagination, actor-critic | \n| `dreamer-v1` | **DreamerV1** | Latent imagination with Gaussian heads | Normal heads, standard KL | \n| `dreamer-v2` | **DreamerV2** | Discrete latents for pixel control | Symlog two-hot heads, balanced KL | \n| `planet` | **PlaNet** | Latent planning from pixels, no explicit policy | RSSM, CEM planner | \n| `modular-rssm` | **ModularRSSM** | Composable recurrent state-space model | Swappable priors/posteriors, custom heads | \n| `iris` | **IRIS** | Sample-efficient RL with Transformers | Discrete VAEs, world models | \n| `jepa` | **JEPA** | Self-supervised visual representations | Masked prediction, ViT | \n| `dit` | **DiT** | Diffusion Transformer workflows | Patch embeddings, diffusion backbones | \n| `diamond` | **DIAMOND** | Diffusion world model for pixel-control RL | EDM sampling, Atari imagination rollouts | \n| `genie` | **Genie** | Generative interactive environments from video | Latent actions, spatiotemporal transformer | \n| `genie-small` | **Genie (small)** | Development- and test-sized Genie | Same architecture, reduced width/depth | \n| `genie-large` | **Genie (large)** | Scaled-up Genie variant | Higher capacity dynamics + tokenizer | \n\nSimulusion follows [semantic versioning](https://semver.org/) as of 1.0.0. The\npublic API — everything listed in the [Public API reference](https://paramthakkar123.github.io/simulusion/public_api.html)\nand re-exported from the top-level `simulusion` namespace — will not break within\nthe 1.x line; anything removed gets a deprecation warning for at least one\nminor release first. Submodule internals not listed there may still change.", "url": "https://wpnews.pro/news/show-hn-simulusion-a-pytorch-library-for-training-and-running-world-models", "canonical_source": "https://github.com/ParamThakkar123/simulusion", "published_at": "2026-10-28 04:50:59+00:00", "updated_at": "2026-10-28 05:12:18.405567+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "ai-tools", "developer-tools", "ai-agents"], "entities": ["Simulusion", "TorchWM", "PyTorch", "Dreamer", "PlaNet", "JEPA", "IRIS", "Hugging Face"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-simulusion-a-pytorch-library-for-training-and-running-world-models", "markdown": "https://wpnews.pro/news/show-hn-simulusion-a-pytorch-library-for-training-and-running-world-models.md", "text": "https://wpnews.pro/news/show-hn-simulusion-a-pytorch-library-for-training-and-running-world-models.txt", "jsonld": "https://wpnews.pro/news/show-hn-simulusion-a-pytorch-library-for-training-and-running-world-models.jsonld"}}