The term world model is used almost everywhere in AI now.
But the more often it appears, the less clear it sometimes become.
A reinforcement learning researcher may use the term for a latent dynamics model. A robotics team may use it for an action-conditioned simulator. A video generation company may describe a large generative model as a world model. An autonomous driving company may use the same expression for a system who creates traffic scenarios.
All of these systems have something in common.
But they are not solving exactly the same problem.
That was the starting point for our report:
State of World Models 2026: Taxonomy, Benchmarks and Open Challenges
We did not wanted to build another leaderboard with one global score. That would have been easy to understand, but probably misleading.
Instead, we tried to answer a more basic question:
How can we describe world models in a consistent way before trying to compare it?
For the report, we used this definition:
A world model is an AI model that learns a representation of an environment and uses it to predict, simulate, evaluate or support action inside that environment.
The definition is intentionally broad.
It can include:
But not every generative model should automatically be called a world model.
A model may generate a beautiful video and still fail to preserve object identity, physical consistency, spatial structure or the consequence of actions.
That is one of the main problems in the current discussion.
A model can look like it understands a world without being really useful for acting inside it.
Imagine three models:
1 One creates extremely realistic videos
2 One helps a robot to plan movements
3 One generates rare traffic situations for autonomous driving
Which one is the best world model?
There is no serious answer without more context.
The first may be better visually.
The second may be better for planning.
The third may be much more useful for safety testing.
Putting all three into one score would hide most of the important differences between them.
That is why we moved toward a taxonomy rather than a global ranking.
The framework uses a set of practical fields.
| Field | Description |
|---|---|
| Name | Public model or benchmark name |
| Organization | Company, lab or institution |
| Publication year | First public release, paper or announcement |
| Domain | Robotics, video, driving, RL, embodied AI, agents |
| Input modalities | Text, image, video, action, state or sensor data |
| Output modalities | Video, state, action, trajectory or simulation |
| Action-conditioned | Yes, no, partial or unknown |
| Representation | Latent, pixel, token, 3D, symbolic or hybrid |
| Temporal horizon | Short, medium, long or procedural |
| Evaluation type | Perceptual, physical, functional or planning |
| Availability | Paper, code, weights, dataset, demo or closed |
| Limitations | Known weaknesses or missing informations |
This is not meant to be a perfect academic ontology.
It is a practical tool for navigating a field that is changing very fast, maybe too fast sometimes.
The first thing to identify is what kind of environment the model is supposed to represent.
Current categories include:
This may sounds obvious, but it changes almost everything.
A robotics model and a video model may both predict future states. But one is expected to help a machine act, while the other may mainly be expected to generate plausible scenes.
The evaluation should reflect this difference.
Two models can work in the same domain and still have very different purpose.
A world model may be designed for:
Some systems combine several of these functions. Others are very specialised.
A visually rich model may be excellent for simulation but difficult to use for explicit planning.
A small latent model may be almost impossible to interpret visually, but very useful for an agent.
World models can represent environments in very different ways.
Some work directly with pixels or video frames.
Others use:
Each choice come with trade-offs.
Pixel-based systems preserve more visual detail, but they can be expensive and difficult to reason over.
Latent systems are more compact, but their internal states are usually harder to interpret.
Object-based and symbolic systems may support planning more naturally, but they can require stronger assumptions about the environment.
There is no obvious winner at this moment.
A model that predicts the next state is not necessarily able to simulate a useful future.
We therefore distinguish between:
This matters because many models look strong over a few frames and become unstable over longer period.
Small errors accumulate.
Objects drift. Geometry changes. The state slowly stop making sense.
For planning, this problem becomes even more serious because every mistake can affect all the next decisions.
One of the most important questions is whether the model understands actions.
A passive model estimates:
P(sₜ₊₁ | sₜ)
An action-conditioned model estimates:
P(sₜ₊₁ | sₜ, aₜ)
Where:
sₜ
is the current state;aₜ
is the selected action;sₜ₊₁
is the predicted next state.The difference looks small on paper.
In practice, it changes the whole use case.
A passive model answers:
What may happen next?
An action-conditioned model answers:
What may happen if I do this?
For robotics, autonomous driving and software agents, that second question is usually the one which is useful.
Current benchmarks evaluate different parts of world modelling.
Some focus on visual quality.
Others look at physical consistency, long-horizon reasoning, action conditioning or downstream performances.
We separated evaluation into four broad groups.
This asks whether the output looks convincing.
Examples include:
These measures are useful, but limited.
This asks whether the environment behaves in a plausible way.
Possible criteria include:
This is much harder to evaluate automatically and results are not always easy to compare.
This asks whether the model actually helps with a task.
For example:
This kind of evaluation is often more important than visual quality, but also more expensive to perform.
This asks whether the model helps to make multi-step decisions.
A model may need to compare possible trajectories, estimate risk, select actions or maintain state over a long sequence.
Current systems still struggle a lot on this.
One of the most useful ideas in current world-model research is the gap between perception and functionality.
A model can generate a sequence that looks excellent to a human viewer and still be unusable for a robot.
A small error in object position may be almost invisible in a video.
For a robot trying to grasp that object, the same error can make the full prediction useless.
This is why visual realism cannot be the only target.
A useful world model should help an agent making better decisions.
That sounds simple, but it changes the way models should be evaluated.
A world-model catalog can be represented with a fairly simple JSON structure.
{
"name": "Example World Model",
"organization": "Example Lab",
"year": 2026,
"domains": [
"robotics",
"embodied-ai"
],
"inputs": [
"video",
"actions"
],
"outputs": [
"future-video",
"trajectory"
],
"action_conditioned": "yes",
"representation": [
"latent",
"video"
],
"temporal_horizon": "medium",
"evaluation": [
"perceptual",
"functional"
],
"availability": {
"paper": true,
"code": false,
"weights": false,
"dataset": true
},
"limitations": [
"Limited public implementation details",
"Evaluation restricted to simulated environments"
]
}
This makes it easier to:
It also avoid rewriting every profile by hand whenever the taxonomy changes.
A lot of frontier models are not documented well enough.
In those cases, the correct value is often:
unknown
Not:
probably yes
A company may claim that a model understands physics without publishing the evaluation protocol.
A demo may suggest action conditioning without clearly documenting the model interface.
A paper may describe a dataset without releasing it.
The catalog should reflect what is known, not what seems probably true.
This is especially important for proprietary systems, where public documentation can be incomplete.
A good model profile should clearly distinguish between:
For example:
Documented:
The model accepts video and action inputs.
Externally evaluated:
The model was tested on benchmark X.
Editorial interpretation:
The available results suggests that long-horizon consistency remains limited.
This makes the information easier to trust and easier to correct later.
Benchmarks should not be mixed directly into the same table as models.
They need separate fields.
A benchmark record may look like this:
{
"name": "Example Benchmark",
"year": 2026,
"domains": [
"video",
"embodied-ai"
],
"evaluates": [
"physical-consistency",
"action-conditioning",
"functional-utility"
],
"evaluation_methods": [
"automated-metrics",
"human-evaluation",
"downstream-task"
],
"public_dataset": true,
"public_code": true,
"limitations": [
"Limited environment diversity"
]
}
Models can then be connected to the benchmarks they were tested on.
This helps preserve:
That last point is important because benchmark results can become outdated quite fast.
The world-model landscape changes too fast for a static database.
Models are updated.
Benchmarks evolves.
New papers correct earlier claims.
A basic versioning approach could be:
v1.0.0 — Initial public dataset
v1.1.0 — New models and benchmarks
v1.1.1 — Corrections and metadata fixes
v2.0.0 — Major taxonomy revision
Each release should include:
This prevents a report or dataset to change silently after publication.
A world-model directory can easily become a general AI catalog.
Some exclusion rules are needed.
For now, the framework excludes:
This does not mean those systems are not useful.
It simply helps to keep the scope understandable.
A project may provide only a paper.
Another may provide code but no weights.
Another may provide weights but no training pipeline.
So availability needs more detail than just “open” or “closed”.
For example:
{
"paper": true,
"code": true,
"training_code": false,
"weights": false,
"dataset": true,
"evaluation_scripts": true,
"demo": true
}
This gives a much more realistic picture about reproducibility.
The taxonomy is not finished.
The term world model is still unstable.
Many systems belongs to several categories.
Public information is incomplete.
Different communities use different evaluation standards.
And no structured dataset can capture all the nuance of a paper or architecture.
The framework should therefore be used as a navigation tool.
It does not replace reading the original research.
The main reason to make the taxonomy public is simple : it needs to be corrected.
A closed database would become outdated very quickly.
An open structure allows researchers, engineers and readers to:
That does not mean every suggestion must be accepted.
But the logic behind classification decisions should stay visible.
The next steps are:
The full report is available here :
Archived publication and DOI:
Project website:
Feedback, corrections and missing references are welcome.
The framework is still early, and it will probably need to evolve together with the field.
Regards,