readme.mp4 #
MIRA is a real-time world model of Rocket League: a 5B parameters latent diffusion model generates the video frame by frame from all four players' actions, so a full 2v2 match can be played inside the model at 20 FPS on a single GPU. Play it live at mira-wm.com.
This is the official code release for the technical report:
MIRA: Multiplayer Interactive World Models with Representation Autoencoders, a collaboration between
General Intuition, Kyutai, and Epic Games.
@article{mira2026,
title={{MIRA}: Multiplayer Interactive World Models with Representation Autoencoders},
author={Hu, Anthony and Volhejn, V{\'a}clav and Ramanana Rahary, Adrien and Mulder, Chris and Makkar, Aditya and Royer, Am{\'e}lie and Liao, Alyx and Orsini, Manu and Jelley, Adam and Alonso, Eloi and Laurent, Florian and Nor{\'e}n, Fredrik and Swingos, James and H{\"u}nermann, Jan and Rollins, Kent and Hosseini, Lucas and Le Cauchois, Matthieu and Peter, Maxim and de Witte, Pim and Brown, Tim and Micheli, Vincent and B{\"o}hle, Moritz and de Marmiesse, Gabriel and Sharmanska, Viktoriia and Specia, Lucia and Black, Michael and P{\'e}rez, Patrick},
year={2026}
}
pixi run setup # one-time: creates the environment and installs MIRA (requires an NVIDIA GPU)
pixi run test # run the test suite
Requires pixi and torch >= 2.8 (installed for you).
Each sample is a 4-second window of a 2v2 match, captured from all four players' synchronised views. For every frame of every view, the dataset provides the video frame, the player's keyboard action, and the game state (ball, cars, and score). The full dataset is on the HuggingFace Hub (see the dataset card).
from mira.data import RocketScienceDataset
ds = RocketScienceDataset.from_hub("kyutai/rocket-science", split="test", shards=1)
clips = ds.load_match(ds.match_ids()[0], clip_len=16, target_fps=20)
clip = clips[0]
clip.frames # tensor (P, T, C, H, W) uint8 — the rendered video, one per player
clip.actions # tensor (P, T, 9) int32 — multi-hot keyboard state, aligned to each frame
clip.events # list of game events overlapping this window (goals, boost pickups, ...)
clip.physics # per player, per frame: game state (ball, cars, score)
Explore it interactively (4-player grid, keyboard overlay, top-down overview of the game using the game state):
pixi run explore
Entry points under scripts/
are Hydra apps — override any config key as key=value
. Multi-GPU runs
go through torchrun
; single-GPU works too.
python scripts/train_codec.py dataset.train_index=/path/to/train dataset.test_index=/path/to/test
python scripts/train_world_model.py \
model.architecture.config.codec_checkpoint=/path/to/codec_ckpt \
dataset.train_index=/path/to/train dataset.test_index=/path/to/test
python scripts/train_world_model.py model=multi_wrapper_world_model dataset.n_players=4 \
model.architecture.config.wm_config.codec_checkpoint=/path/to/codec_ckpt \
run.finetune_from=/path/to/single_player_ckpt \
dataset.train_index=/path/to/train dataset.test_index=/path/to/test
Codec training uses a frozen DINOv3-L/16 encoder whose weights are gated by Meta. Download
dinov3_vitl16_pretrain_lvd1689m-8aa4cbdd.pth
from the
DINOv3 page and set
RS_DINO_WEIGHTS_DIR=/path/to/weights
. World-model training and inference don't need it.
python scripts/eval_world_model_offline.py /path/to/checkpoint-1000/checkpoint.pth
Apache License 2.0 — see LICENSE.