Training a local model to play browser games on the fly

All posts

IronBee Gamer turns a browser game's URL into a trained AI model that plays it live. An LLM writes the rules, a 322M model is trained on them and makes a move in 22 to 45 ms, and here is what broke on the way.

The clip above is live. The game doesn't wait for anybody. A 322M model called Laya decides every flap, about 35 ms a decision, and nobody ever told it when to flap. An LLM worked that out and wrote it down, as code that rates each move and as rules, and Laya learned the rules.

Here is one jump from a Chrome Dino game. These are lines from its log, with a few fields of the state:

   t  game time  move  air    rising  dropSafe  cactus dx
 110  3.52 s     NOOP  false  false   false            73
 111  3.55 s     JUMP  false  false   false            61
 112  3.58 s     JUMP  true   true    false            49
 116  3.71 s     JUMP  true   true    false             1
 119  3.81 s     NOOP  true   false   false           -35
 120  3.84 s     DUCK  true   false   true            -47

It jumps when the cactus is 61 px away, keeps the key down while the dino rises so the jump goes full height, lets go at the top, and presses DUCK to drop back down fast once the cactus is behind it. In this game Laya took about 24 ms a decision.

• The game is left alone. The app needs no API and changes none of the game's code. It plays the public page.
• An LLM writes the logic. The code that turns the page into a state, and the rules. It keeps improving them from the games it loses.
• A small model plays. Laya, 322M parameters, about 25 ms a move. Fast enough to play live.

This post is how we got there in two weeks. Most of it is about things that broke. Every number comes from our own runs between September 25 and October 13, 2026.

Day one: can a fast model play a game at all? #

It started as a side question while we were building IronBee Express. Jev, the model we use there, picks one option from a list in about 275 ms. Could it play a browser game with nothing but the game's own instructions?

It could, once we had three things:

1. Something that turns the page into a small JSON state.
2. A frozen game clock, so a slow decision costs no game time.
3. An LLM that writes the rules for each game and keeps fixing them.

The first loop was simple. Play a few games, show the LLM what happened, let it write a new version, keep the version only if it scores higher.

The LLM found things we didn't know, from logs and screenshots only:

• Flappy Bird's top pipes are drawn flipped , with a negative height. The first version never saw them.
• Pac-Man only takes a turn within 3 px of a tile's center , and a key that is held never fires again. With 100 ms between decisions, it kept missing that window.

Then we let Dino run for three minutes instead of 45 seconds. It died every time, between 50 and 58 seconds. At 700 points the game turns to night and draws a moon and stars. The code that reads the page had never seen a star and let one through as an obstacle. So the dino jumped at nothing, then came down on a cactus or flew into a bird.

How it works #

The browser belongs to IronBee DevTools. The game tools run inside it as a plugin. There are four ways to read a game, and the app picks one when you add it:

In the first prototype, reading a Dino frame took about 1 ms and extract() under 0.1 ms. A Phaser dump took 10 to 22 ms. On Dino, the recorder's position for an obstacle was 0 px off the game's own value, over hundreds of frames.

The engine gets one question per decision. The options are the game's actions, each with a short description. This is Dino's, trimmed:

{
  "state": {
    "game": {
      "player": { "air": false, "rising": false },
      "threat": { "kind": "cactus_small", "dx": 61, "top": 35, … }
    }
  },
  "questions": {
    "action": {
      "type": "choice",
      "criteria": {
        "NOOP": "Keep running …",
        "JUMP": "Jump (only from the ground with a fresh press …)",
        "DUCK": "On the ground: duck under birds at head height …"
      },
      "instructions": {
        "goal": "Chrome Dino (T-Rex runner). Press Space or ArrowUp …",
        "instructions": "Apply these rules in order. …"
      }
    }
  }
}

The answer is one of NOOP, JUMP or DUCK, with a probability for each. You can watch all of it in the web UI:

Features, never the answer #

The LLM writes the code that builds the state. So it could just put the answer in there. The very first tuning run did exactly that: it added advice: "JUMP" to the state and wrote the rule "choose game.advice". Jev became a rubber stamp, and the score measured the LLM's code, not the engine. We threw that run away.

Now a field that names or recommends an action is removed before the engine sees the state, and the LLM is told how many were removed:

// from src/play/guard.ts, split in two here to fit
const ADVICE_KEY: RegExp = new RegExp(
    "recommend|advice|advise|suggest|best.?action|next.?action|^should" +
    "|^(decision|choice|action|todo|answer)quot;, "i");

A string value equal to an action id is removed too, unless its key names a fact, like a heading or a direction.

It also turned out that the engine does worse with a hint, not better. With a recommended field in the state, Pac-Man ate 21 dots. Without it, 50 to 89. With a correct rule on named fields, Jev followed it 99% of the time: 16 deviations in 1,680 decisions.

The line is still blurry, though. That check only reads field names, and the state code may still predict what each action leads to. How much thinking that leaves for the engine differs by game. In Dino the engine applies a table of rules over several fields. In Flappy Bird the state code rates flapping now against waiting over the next 80 frames and says which leaves more room. The rule is one line, flap when that field says "moreRoom", and the engine is only asked on those frames. Pop the Lock works the same way. You can read Flappy Bird's rule in the screenshot above.

Stopping time #

The browser's clock is Playwright's fake clock. It is d before the game loads, and each step runs exactly one tick of game time, then stops again. Timers, requestAnimationFrame, performance.now() and Date all follow it. The game has no idea a decision took 300 ms. VideoGameBench does the same thing to its emulator in its Lite mode.

That's what lets Jev play at all. It also shows the price:

90 s of Dino at a 30 ms tick  =  3,000 decisions
3,000 decisions × 0.28 s     ≈  14 minutes of wall time

How much game time runs between two decisions is the tick. The LLM picks it per game, and a new version can change it. Three games ended up at 16 ms, one frame, and that was too short:

A Laya decision plus the browser step takes about 28 ms, so at 16 ms Dino played at 60% of its own speed. Now 32 ms is the shortest tick with the clock d. Flappy Bird lost a seed at 32 ms: the code that builds its state assumed the next flap could land one frame later, and at 32 ms the next frame it got to see was two frames later. One retrain fixed that code, 38 / 38 / 38 again.

Same seed, same game #

Every game is a fresh page load with Math.random seeded. The same seed should give the same game, frame for frame. That's how two versions are compared fairly, and how a lost game can be replayed tick by tick. Under load it didn't hold: Dino's seed 303 scored 1,485 in one run and 825 in another. Six things broke it, or could have:

One more: seeds 101, 202 and 303 started as nearly the same game, because the old Park-Miller generator drew almost the same first numbers for them. Now a seed goes through splitmix32, then mulberry32.

ibgamer check plays a seed twice and compares every frame:

$ ibgamer check dino --seed 101 --seconds 8
Chrome Dino v11, its rules deciding, seed 101, one after the other:
the same for all 251 frames
  ends: budget at 8.00 s, score 89 | budget at 8.00 s, score 89

When two runs differ, it names the first frame and the field, like raw.top: 155.4 ≠ 158.3. We ran about 150 games with up to 27 browsers at once and a load average near 70. Two came out different, once each, and never again.

The strangest bug was Tetris scoring 0 when nothing was cached. The page loads a 1.8 MB soundtrack, which took 11 to 25 s, and our boot pressed PLAY on the screen. The whole game was played on the menu. Waiting for open requests with a 15 s timeout still lost 2 starts of 5. Waiting while data is still coming in (Network.dataReceived) fixed it: 4 cold starts of 4.

The trainer learns from its losses #

Most games start badly. The first rules come from the game's own instructions and one look at the page, and Laya knows nothing until it's taught:

The trainer is an LLM, the Claude Code CLI or the Codex CLI, on your own login. It never makes a move in the game. It only writes: the code that turns the page into a state, the rules, and tests. What it kept, game by game:

When a game is lost, the frames before the end are kept, and the LLM writes a test that pins its fix down. A test replays those frames offline, in seconds, so a new fix can't quietly break an old one. This one is from Dino's night mode. With only a star and an overhead bird ahead, the dino must stay on the ground, so on ticks 49 and 68 of a lost game the engine must choose NOOP:

{
  "ticks": [49, 68],
  "needsChoice": true,
  "expect": "choice === 'NOOP'",
  "why": "With only a star and an overhead bird ahead, the dino must …"
}

Things that went wrong here:

• Seeds it never saw. An Infinite Mario candidate that played 1,266, where the best version played 495, was thrown away, because on the unseen seeds it scored 859.0 against 859.3. We saved it by hand, it is v4 in the chart above, and changed the rule: now a candidate may be up to 1% worse there.
• The wrong score. We first scored Tetris in rows cleared in 120 s. The rules found a trick: keep a base of 8 rows that never clears, with one column left empty, so every piece falls about 8 rows less and more pieces fit in the time. Almost every clear was a single row. With the game's own points, four rows at once pay 1,200 against 40 for one. We changed the score, and told the trainer in a note that it never dropped the long bar into that empty column. Version 4 used the column to clear four at once. The first version scored 7,742 points, the fourth 40,659.
• Running out of time. One call of the trainer may take 30 minutes. A call past that is lost whole, thinking and all. It happened in Flappy Bird's training and again in Breakout's. Now every prompt says how much time there is, and the attempt after a timeout is told to decide sooner.

Laya: the rules, in 322M parameters #

With the clock d, Jev plays ten of the eleven games well (it hasn't played Breakout). But 275 ms a move is far too slow to play live, and a 90 s game takes 14 minutes to watch.

Laya is a small open model from Convai Innovations that picks from a list of options, like Jev, and runs on our own machine. Out of the box it's near chance on a game: on Dino it agreed with Jev on 16% of the moves and never jumped right. So each game gets its own Laya, distilled from the rules:

The loss is plain cross-entropy against the teacher's probabilities:

def soft_ce(logits, target):
    return -(target * F.log_softmax(logits, -1)).sum(-1)

This is how Dino went:

What broke on the way:

• A model served in the wrong precision. Laya is fine-tuned in bfloat16. Served in float32, Tetris's model answered "drop" to almost every state, and Dino's played 448 / 1,333 / 870 instead of 1,093 / 1,485 / 870. Now a model is served the way it was trained.
• A frozen Mac. A batch of sixteen game states, about 950 tokens each, took 30 to 55 GB on a 48 GB Mac during fine-tuning. It filled the swap and the machine froze. A batch now runs in pieces of at most 2,048 tokens, and a run holds 6 to 9 GB at the same speed.
• The last three mistakes. After four rounds Laya agreed with its teacher on 99.9% of the states it saw, and still lost two games. Both times the dino was landing, a cactus was close, the rules waited one tick and Laya pressed JUMP. That press was lost in the air. Those were three states out of 10,058. Now the states a model still gets wrong make up a tenth of every batch, and a round stops only when it gets each one right. The next round played 1,485 ×3.

Every game we trained, with the clock d:

A state is counted each time the rules labeled it. Identical ones are merged before training, so Dino's 133,616 are 23,656 different states.

In three games the student beats its teacher. Our guess is that it smooths over some of the rules' rough edges.

Playing live #

With the clock d, the game waits for every decision. Live, it doesn't. A decision lands about 50 ms after the frame it was made on, and the game has moved on by then. Four things made live play work:

1. Simulate it first. Live games don't replay: on the Phaser copy of Flappy Bird we started with, Laya scored 34 one run and 15 the next. So training simulates live play on the d clock, each decision landing late on purpose, and every run gives the same result. You can play a game that way too:ibgamer play pop-the-lock --lag 45 .
2. Tell the state code about the lag, then hold every input to it. The state is computed for where the game will be when the input lands. And an input is held back so it always lands the same time after its frame, even if Laya answered faster. An earlier Dino version (v7), with Laya answering in 23 ms: 1,493 / 290 / 1,116. Held to 45 ms: 1,494 / 1,091 / 1,493.
3. Keep Laya warm. After 2 to 5 seconds idle, a decision took 70 to 130 ms. Back to back, 28 ms. Live, Laya now gets a tiny question whenever it has been idle for 100 ms.
4. Train for a busy machine. A browser tab using most of a core pushed Laya's inputs to 56 to 99 ms, where the versions were trained for 45 to 60. Now every seed is trained at 45, 68 and 90 ms.

Laya live, every input held to the lag its version was trained for (point 2):

Jev can't play these live. Even when we asked it for a plan of the next eight moments in one request, so one request could cover 400 ms, Pop the Lock scored 1, 0 and 2 and Dino 312, 154 and 208. No game offers Jev live.

Breakout, from a URL to a model #

Breakout is the newest game, and the only one added the way a user would: paste the URL in the UI and press a few buttons. Here is what happened, minute by minute:

Breakout draws on a 2D canvas, but its own state is easy to reach, so the LLM read the page's scripts and wrote one expression that returns it. This is the start of it:

(() => { try {
  const G = (typeof Game !== 'undefined' && Game) ? Game.current : null;
  if (!G || !G.court || !G.paddle || !G.ball || !G.score)
    return { playing: false, screen: '' };
  …

It also found the score in the game's code:

(() => { try {
  const G = (typeof Game !== 'undefined' && Game) ? Game.current : null;
  if (!G || !G.score) return { over: false, score: 0 };
  return {
    over: G.current === 'menu' && !!G.ball && G.ball.countdown !== undefined,
    score: +G.score.score || 0
  };
} catch (e) { return { over: false, score: 0 }; } })()

The first Laya fell short of its rules. That's what the Train button is for. It checks how the engine plays beside its own rules, fixes whatever loses, and checks again:

For Breakout, the check said Laya lost on its own. Two DAgger rounds, 30 minutes:

In those rounds the rules corrected 331, then 234, of Laya's 10,000 decisions.

Breakout plays with the clock d so far. We haven't trained it for live play yet.

What we threw away, and what still doesn't work #

Still open:

• Live play still varies, and costs some games points. Pac-Man's earlier model scored 12% less live than with the clock d in the three games above, and 16% less over 15 more. Its newest model played 5,440 in its one live game, on a seed where it scores 8,130 d. Doodle Climb's Laya climbs about 84% of the tower live on average, against 100% d. And one game can go badly: on October 5 Pop the Lock died at 30 s with 31 points.
• Long plans. In our games a plan covers a few seconds, or one level. A game where the plan spans many screens, like a puzzle or a map, hasn't been tried.
• Some pages are harder to read. The 2048 at play2048.co draws its canvas off the page's main thread, so the draw recorder sees nothing, and reading its pixels hasn't been tried yet. Pages behind bot protection are off limits.

Try it #

Every trained game is on Hugging Face at ironbee-ai/ironbee-gamer-library: its versions, the frames its tests replay, and the Laya model it plays with.

The code is on GitHub at ironbee-ai/ironbee-gamer, under the Elastic License 2.0. You need Node.js 22 and Python 3 for Laya:

git clone https://github.com/ironbee-ai/ironbee-gamer.git
cd ironbee-gamer
npm install && npm run build && npm link
ibgamer laya setup
npm run ui

Open http://127.0.0.1:1986, press Hugging Face to download the games, pick one and press Play. To add your own, press Add game and paste its URL. Training needs the Claude Code CLI or the Codex CLI, logged in.