# AI Agents That Live Inside a Dreamed-Up World

> Source: <https://dev.to/shridhar_shah2297/ai-agents-that-live-inside-a-dreamed-up-world-3n4j>
> Published: 2026-07-21 05:51:55+00:00

*An agent watches a game, learns to hallucinate the next frame, then plays inside its own dream — but only the model that knows players react to each other stays true.*

**TL;DR:** The hottest idea in agents right now: don't feed them the real world — let them **dream** it. An agent watches some footage, learns to hallucinate the next frame, and practices inside its own head. I built a tiny one, and found the catch: a dream only stays true if it knows the players **react to each other.** Runs on a laptop.

Two players in an 11-cell corridor: a predator steps toward the prey, the prey steps away. Every move is a *reaction*. An agent watches random games, then closes its eyes and **dreams 15 frames ahead**, feeding each prediction back in as the next input. I built two dreamers from the exact same footage:

Training an agent inside its own learned dream goes back to Ha & Schmidhuber's [ World Models](https://arxiv.org/abs/1803.10122) (2018); the open frontier is making that dream

% of the dream still matching reality, this many frames ahead:

| frames ahead | 1 | 3 | 5 | 10 | 15 |
|---|---|---|---|---|---|
| single-player dream | 11 | 0 | 0 | 9 | 0 |
multiplayer dream |
100 |
100 |
100 |
100 |
100 |

The single-player dream falls apart almost immediately. The multiplayer dream stays locked to reality the whole way.

```
real = dream = start
for _ in range(15):
    real  = real_next(*real)   # what actually happens
    dream = model(*dream)      # feed the dream its OWN last frame
    match += (dream == real)
```

Same footage, same loop. The only difference: whether the model predicts the two players **jointly** or independently.

The prey moves *because* the predator moved. A model that looks at the prey alone can't see that — so its tiny errors compound each frame until the dream is pure fiction. The multiplayer model conditions on both, so it captures the reaction and keeps re-predicting the real game.

A dream you can act in is a superpower — an agent can practice a thousand risky moves for free. But a dream that forgets everyone else reacts to you isn't practice. It's a delusion.

Dreamed worlds let agents rehearse infinitely, safely, at zero real-world cost. The lesson from this toy: for **multi-agent** dreams, model the reactions or the whole thing drifts. Get it right and agents can plan against each other entirely in imagination.

```
git clone https://github.com/Shridhar-2205/secret-lives-of-agents
cd secret-lives-of-agents/03-dreamed-world && python demo.py
```

**Shridhar Shah** — Senior Software Engineer on the AI team at Cisco. [GitHub](https://github.com/Shridhar-2205) · [LinkedIn](https://www.linkedin.com/in/shridhar-shah-220b1721b/)

Sources & further reading:Ha & Schmidhuber,[(2018) — the "train inside a dream" idea · Hafner et al.,]World Models[(DreamerV3, 2023) · Bruce et al.,]Mastering Diverse Domains through World Models[(2024).]Genie: Generative Interactive Environments