{"slug": "mira-multiplayer-interactive-world-models-with-representation-autoencoders", "title": "MIRA: Multiplayer Interactive World Models with Representation Autoencoders", "summary": "Researchers from General Intuition, Kyutai, and Epic Games released MIRA, a 5-billion-parameter latent diffusion world model that generates real-time 2v2 Rocket League matches at 20 FPS on a single GPU. The model, trained on a dataset of synchronized player views and actions, enables interactive gameplay inside the simulation and is available as open-source code with a live demo at mira-wm.com.", "body_md": "## readme.mp4\n\nMIRA is a real-time world model of Rocket League: a 5B parameters\nlatent diffusion model generates the video frame by frame from all four players' actions, so a full 2v2\nmatch can be played inside the model at 20 FPS on a single GPU. Play it live at\n[mira-wm.com](https://mira-wm.com).\n\nThis is the official code release for the technical report:\n\n[MIRA: Multiplayer Interactive World Models with Representation Autoencoders](https://github.com/mira-wm/mira), a collaboration between\n\n[General Intuition](https://www.generalintuition.com), [Kyutai](https://kyutai.org), and [Epic Games](https://www.epicgames.com/site/home).\n\n```\n@article{mira2026,\n  title={{MIRA}: Multiplayer Interactive World Models with Representation Autoencoders},\n  author={Hu, Anthony and Volhejn, V{\\'a}clav and Ramanana Rahary, Adrien and Mulder, Chris and Makkar, Aditya and Royer, Am{\\'e}lie and Liao, Alyx and Orsini, Manu and Jelley, Adam and Alonso, Eloi and Laurent, Florian and Nor{\\'e}n, Fredrik and Swingos, James and H{\\\"u}nermann, Jan and Rollins, Kent and Hosseini, Lucas and Le Cauchois, Matthieu and Peter, Maxim and de Witte, Pim and Brown, Tim and Micheli, Vincent and B{\\\"o}hle, Moritz and de Marmiesse, Gabriel and Sharmanska, Viktoriia and Specia, Lucia and Black, Michael and P{\\'e}rez, Patrick},\n  year={2026}\n}\npixi run setup   # one-time: creates the environment and installs MIRA (requires an NVIDIA GPU)\npixi run test    # run the test suite\n```\n\nRequires [pixi](https://pixi.sh) and torch >= 2.8 (installed for you).\n\nEach sample is a 4-second window of a 2v2 match, captured from all four players' synchronised views.\nFor every frame of every view, the dataset provides the video frame, the player's keyboard action,\nand the game state (ball, cars, and score). The full dataset is on\nthe [HuggingFace Hub](https://huggingface.co/datasets/kyutai/rocket-science) (see the\n[dataset card](/mira-wm/mira/blob/main/docs/dataset_card.md)).\n\n``` python\nfrom mira.data import RocketScienceDataset\n\n# Download the test split. shards=1 pulls just one shard for a quick look; omit it for the full split.\nds = RocketScienceDataset.from_hub(\"kyutai/rocket-science\", split=\"test\", shards=1)\n\n# Load the clips of the first match (16 frames each, sampled at 20 FPS) and take the first one.\nclips = ds.load_match(ds.match_ids()[0], clip_len=16, target_fps=20)\nclip = clips[0]\n\n# A clip holds P=4 synchronised player views over T=16 frames:\nclip.frames    # tensor (P, T, C, H, W) uint8 — the rendered video, one per player\nclip.actions   # tensor (P, T, 9)       int32 — multi-hot keyboard state, aligned to each frame\nclip.events    # list of game events overlapping this window (goals, boost pickups, ...)\nclip.physics   # per player, per frame: game state (ball, cars, score)\n```\n\nExplore it interactively (4-player grid, keyboard overlay, top-down overview of the game using the game state):\n\n```\npixi run explore\n```\n\nEntry points under `scripts/`\n\nare Hydra apps — override any config key as `key=value`\n\n. Multi-GPU runs\ngo through `torchrun`\n\n; single-GPU works too.\n\n```\n# Train the codec\npython scripts/train_codec.py dataset.train_index=/path/to/train dataset.test_index=/path/to/test\n\n# Train the single-player world model\npython scripts/train_world_model.py \\\n    model.architecture.config.codec_checkpoint=/path/to/codec_ckpt \\\n    dataset.train_index=/path/to/train dataset.test_index=/path/to/test\n\n# Train the 4-player world model, warm-started from a single-player checkpoint\npython scripts/train_world_model.py model=multi_wrapper_world_model dataset.n_players=4 \\\n    model.architecture.config.wm_config.codec_checkpoint=/path/to/codec_ckpt \\\n    run.finetune_from=/path/to/single_player_ckpt \\\n    dataset.train_index=/path/to/train dataset.test_index=/path/to/test\n```\n\nCodec training uses a frozen **DINOv3-L/16** encoder whose weights are gated by Meta. Download\n`dinov3_vitl16_pretrain_lvd1689m-8aa4cbdd.pth`\n\nfrom the\n[DINOv3 page](https://ai.meta.com/resources/models-and-libraries/dinov3-downloads/) and set\n`RS_DINO_WEIGHTS_DIR=/path/to/weights`\n\n. World-model training and inference don't need it.\n\n```\npython scripts/eval_world_model_offline.py /path/to/checkpoint-1000/checkpoint.pth\n```\n\nApache License 2.0 — see [LICENSE](/mira-wm/mira/blob/main/LICENSE).", "url": "https://wpnews.pro/news/mira-multiplayer-interactive-world-models-with-representation-autoencoders", "canonical_source": "https://github.com/mira-wm/mira", "published_at": "2026-07-07 19:54:11+00:00", "updated_at": "2026-07-07 19:59:59.316467+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "generative-ai", "ai-research", "ai-products"], "entities": ["General Intuition", "Kyutai", "Epic Games", "Rocket League", "MIRA", "HuggingFace", "Meta", "DINOv3"], "alternates": {"html": "https://wpnews.pro/news/mira-multiplayer-interactive-world-models-with-representation-autoencoders", "markdown": "https://wpnews.pro/news/mira-multiplayer-interactive-world-models-with-representation-autoencoders.md", "text": "https://wpnews.pro/news/mira-multiplayer-interactive-world-models-with-representation-autoencoders.txt", "jsonld": "https://wpnews.pro/news/mira-multiplayer-interactive-world-models-with-representation-autoencoders.jsonld"}}