AIArticle
A from-scratch Super Mario world model clears short-horizon tests, then stalls on distant goals.
World models that learn dynamics straight from pixels and actions, with no reward labels, keep looking like the clean path to agents that can plan. LeWorldModel showed one compact Joint-Embedding Predictive Architecture (JEPA) doing reward-free planning on Push-T. LeMario rebuilds that stack from scratch and trains it on Super Mario Bros instead.
The model clears every test that usually counts as success: held-out prediction, action sensitivity, multi-step rollout quality, and short visual goal seeking. Then it fails the moment the goal sits past the first real obstacle. That gap is the useful story. Short-horizon dynamics and long-horizon progress are not the same skill, and a clean JEPA can master one while barely touching the other.
The architecture that actually ran #
Each training sample holds four Mario frames plus the button history between them. A vision encoder maps every frame to a 192-dimensional latent:
z_t = E_θ(x_t), z_t ∈ R^192
Frames arrive as [batch, 4, 3, 224, 224]
. Actions arrive as [batch, 4, 5, 6]
(five emulator steps of Left/Right/Up/Down/A/B). An action encoder collapses each 5×6 sequence into another 192-vector.
Those latents feed a causal predictor of six transformer blocks. Actions do not simply concatenate. They enter through Adaptive LayerNorm Zero (AdaLN-Zero): each action produces shift, scale, and gate values for both the attention and MLP branches. The gates start at zero so the network cannot inject random action noise early in training; it has to learn which controls matter. After the blocks, a small head emits three predicted future latents. Training minimizes MSE against the real next latents, plus a SIGReg term (weight 0.1) that keeps the representation from collapsing to a constant vector.
That is the whole machine: encoder, action-conditioned transformer predictor, prediction loss, anti-collapse regularizer. No decoder, no pixel reconstruction, no reward head.
The numbers that look like a win #
LeMario trained on 737,134 frames from 280 episodes spanning 32 Mario levels. Loss alone is a weak signal here. Nearby frames look similar, so “predict no change” is already a strong baseline.
On held-out episodes the comparison is clearer:
| Method | One-step error | Five-step error |
|---|---|---|
| LeMario | 0.013773 | 0.077717 |
| Predict no change | 0.014472 | 0.142473 |
| Shuffled actions | 0.016555 | 0.114648 |
Shuffling the actions raised one-step error by 20.2%. Over five recursive steps LeMario beat pure persistence by 45.5%, while shuffled actions sat 47.5% worse. Buttons matter more the farther the rollout goes. That is real evidence the model learned short-horizon, action-conditioned Mario dynamics, not just image similarity.
Planning that works until it doesn’t #
Once futures are imaginable, you can search them. LeMario’s reward-free planner could drive Mario toward nearby image goals and finish within two to five pixels of the targets. For a moment it looked like the model had learned to play.
Move the goal farther into the level and the illusion ends. Mario cannot reliably clear the first major obstacle or navigate toward a single distant goal image. The world model predicts what the screen will look like if you press buttons. It does not, by itself, learn which sequences of screens constitute progress through a level.
That distinction is easy to miss when demos stay inside the comfort zone of short visual matching. Prediction quality and planning competence diverge as soon as the horizon includes a pipe, a gap, or a Goomba that actually has to be dealt with.
What this means if you build world models #
Classic Mario agents (Double DQN, older deep Q setups with replay and hand-shaped x-progress rewards) optimize an explicit return. They often need heavy engineering of the reward and action space, but they are forced to care about forward motion. JEPA-style world models flip the stack: learn a latent dynamics model first, plan later, keep rewards optional. LeMario is a concrete, end-to-end template for the second path on a game people already know how to instrument with gym-super-mario-bros and related wrappers.
If you want to try the same recipe, the practical checklist looks like this:
- Collect multi-level, multi-episode pixel + button trajectories (hundreds of thousands of frames, not a single stage).
- Encode frames and short action windows into a shared latent size (here 192).
- Condition the transformer predictor with AdaLN-Zero rather than naive concatenation so actions can modulate features without early noise.
- Train multi-step latent prediction plus an anti-collapse regularizer; do not rely on reconstruction.
- Evaluate with three checks that LeMario used: held-out one-step error, multi-step recursive error, and action-shuffle ablations. Then test planning on near goals andfar goals that require obstacle handling.
Trade-offs are real. You avoid reward hacking and label collection, which is attractive for simulation, robotics, or any environment where a dense reward is expensive or gameable. You pay with planning fragility: pure latent MPC or goal-image matching can look strong locally and still lack the inductive bias that “rightward progress past hazards” is what matters. Compared with pixel-reconstruction world models you also get a more abstract latent, which is good for planning efficiency and bad if you ever need to debug by looking at decoded frames.
Who should care now? Researchers and tool builders who want a small, readable JEPA reference implementation on a familiar domain. Product teams shipping game AI or simulation agents should treat it as a diagnostic, not a drop-in controller: use it to learn dynamics, then layer hierarchical goals, learned value estimates, or hybrid reward signals before you trust long-horizon behavior. If longer-horizon JEPAs or better planners close the gap LeMario exposed, the same training recipe becomes a practical foundation rather than a demo. Until then, the honest read is “excellent short-horizon world model, incomplete agent.”
The postmortem is the product. Building the architecture from scratch, watching every conventional metric pass, then watching distant goals fail, is more informative than another polished success video. Prediction of Super Mario is tractable with a compact JEPA. Progress through Super Mario still demands something the pure dynamics model did not supply.
Sources & further reading #
LeMario: Training a JEPA World Model on Super Mario Bros— benjamin-bai.com - hckr news - Hacker News sorted by time— hckrnews.com - Train a Mario-playing RL Agent — PyTorch Tutorials 2.13.0+cu130 documentation— docs.pytorch.org - GitHub - aleju/mario-ai: Playing Mario with Deep Reinforcement Learning · GitHub— github.com
Mariana Souza· Senior Editor
Mariana covers the fast-moving world of machine learning and generative AI, with a particular focus on how these technologies are reshaping development workflows. When she isn't stress-testing the latest foundation models, she's usually at a local hackathon.
Discussion 0 #
No comments yet
Be the first to weigh in.