A 211M-param transformer acts as a game engine. It draws every next frame from the last 2.4 seconds and my inputs.
Built a simulator of 18 Backrooms levels in PyTorch to generate the data.
Recorded 440 hours of scripted play in it.
Trained the model from scratch on that, using flow matching in a pretrained autoencoder's latent space.
The model was trained on its own imperfect frames as context so it doesn't drift right away. It recalls a few older frames by position, so a quick look back usually works.
Inspired by Oasis Minecraft.
To try it you need an NVIDIA GPU with ~4 GB.
Weights: [https://huggingface.co/yelarys/noclip](https://huggingface.co/yelarys/noclip)
Video with sound: [https://x.com/yelaryss](https://x.com/yelaryss)
Comments URL: [https://news.ycombinator.com/item?id=50594070](https://news.ycombinator.com/item?id=50594070)
Points: 1