LeMario: Training a JEPA World Model on Super Mario Bros

It almost taught Mario to think—then the comments roasted the dream

TLDR: A developer built an AI system that could predict short Mario moments from gameplay, but it still couldn’t reliably get past early obstacles. Commenters loved the honesty, but argued the real issue is simple: predicting what happens next is not the same as knowing what matters to beat the level.

A coder tried to build a baby brain for Super Mario Bros.—a system that watches the screen, notices button presses, and tries to guess what happens next. At first, it looked like a tiny miracle: the model could predict short-term game moments and even steer Mario toward nearby goals. But the second the target got farther away, the fantasy collapsed. Mario basically froze at the first big obstacle, turning a victory lap into a very public postmortem of what went wrong.

And honestly? The community was into it. One camp praised the write-up as a rare honest dev diary, with one commenter calling it a "gem" while immediately side-eyeing the whole idea of planning from compressed hidden signals instead of plain old position on screen. That sparked the biggest underlying hot take: just because an AI can predict the game doesn’t mean it understands what actually matters to win. Another commenter pushed that point harder, saying the model can tell what’s predictable, but not what’s important—which is kind of a disaster when Mario needs to choose between looking right and actually clearing a pit.

Then came the internet’s favorite side quest: name discourse. Faced with terms like JEPA and LeMario, one reader admitted, "man I’m so brainrotted, I just see these names and I laugh." And really, that set the vibe. The science was serious, but the comments turned it into a mini drama about whether this was a clever breakthrough, a neat dead end, or just the latest case of AI being amazing at vibes and terrible at jumping.

Key Points

  • The article documents a from-scratch reproduction of LeWorldModel, adapting a JEPA world model from Push-T to Super Mario Bros.
  • The model encodes four video frames and corresponding button sequences into 192-dimensional frame and action latents.
  • A six-block causal transformer predictor uses AdaLN-Zero to inject action-dependent shift, scale, and gate controls into attention and MLP branches.
  • The trained model generalized to held-out episodes, used action inputs, and outperformed strong baselines on five-step future prediction.
  • Reward-free planning worked for nearby visual goals but failed on farther goals, showing that predicting game dynamics did not translate into reliable progress through the level.

Hottest takes

"it’s findings :)" — lucrbvi
"man I’m so brainrotted, I just see these names and I laugh" — jdiaz97
"the model has no way of assigning importance" — enjeyw
Made with <3 by @siedrix and @shesho from CDMX. Powered by Forge&Hive.
LeMario: Training a JEPA World Model on Super Mario Bros - Weaving News | Weaving News