# Fei-Fei Li Defines Functional World Model Taxonomy > Source: > Published: 2026-06-04 21:54:56.975088+00:00 # Fei-Fei Li Defines Functional World Model Taxonomy Fei-Fei Li and the World Labs team published "A Functional Taxonomy of World Models" on Jun 3, 2026 via Substack; a republished excerpt appears on a16z.news. The piece distinguishes three functional components commonly called "world models": **renderers**, **simulators**, and **planners**, and places these inside the classic agent loop derived from the partially observable Markov decision process (POMDP), a framing the post cites from Sutton and Barto. The authors contrast the statistical substrate of language models with the spatial and physical structure that world models must capture, citing examples from video generation, physics engines, robotics, and generative vision. The essay argues for clearer vocabulary so researchers and practitioners can target design, evaluation, and tooling for spatial intelligence systems. ### What happened Fei-Fei Li and the World Labs team published "A Functional Taxonomy of World Models" on Jun 3, 2026 on Substack; a republished excerpt appears on a16z.news. The post presents a structured breakdown of systems commonly labeled as "world models," separating them into **renderers**, **simulators**, and **planners**, and situating those components in the agent interaction loop that the authors link to the partially observable Markov decision process (POMDP) and the canonical Sutton and Barto textbook. The post contrasts the capabilities of current language models with models that learn the statistical structure of space and time and includes the epigraphs "The world is everything that is the case," and "The world is not made of words." ### Technical details The post describes **renderers** as the components that convert internal state into perceptual outputs, **simulators** as mechanisms that model state transitions under physics or dynamics, and **planners** as decision-making layers that select actions over trajectories. Per the article, the taxonomy is intended to separate perceptual generation (for example, video synthesis that may ignore physical plausibility) from dynamical prediction (for example, physics engines that enforce conservation laws) and from policy/trajectory search. The authors place these components inside the classic perception-action loop and emphasize the need to model both spatial structure and temporal dynamics rather than text-only statistics. ### Editorial analysis - technical context The taxonomy codifies a distinction practitioners already face when combining modalities and objectives. Industry and academic systems mix components that serve different functions: high-fidelity visual outputs, mechanically faithful dynamics, and goal-directed planning. Observed patterns in similar efforts show trade-offs between generative fidelity and physical realism; separating renderer, simulator, and planner clarifies where those trade-offs occur and what evaluation criteria apply to each component. ### Context and significance Editorial analysis: The post adds conceptual precision at a moment when multiple communities, computer vision, robotics, reinforcement learning, and generative modeling, use the single label "world model" for different artifacts. For researchers designing benchmarks or training curricula, the taxonomy helps map architecture choices to functional requirements: visual quality belongs in renderer metrics, forward-prediction fidelity belongs in simulator metrics, and decision performance belongs in planner metrics. For platform builders and toolmakers, the framing highlights the recurring integration problem of coupling a differentiable or learned renderer with a simulator that may be hybrid (learned plus analytic) and a planner that must handle uncertainty. ### What to watch - •Adoption: whether benchmark suites and evaluation papers adopt distinct renderer/simulator/planner metrics. - •Hybrid tooling: releases that explicitly expose interfaces between differentiable renderers and physics simulators. - •Agent evaluations: studies that ablate which component limits real-world transfer, e.g., when perceptual realism improves but dynamical prediction does not. Editorial analysis: Across projects that combine these components, observers should look for clearer API boundaries, modular datasets that isolate spatial vs temporal generalization, and evaluation protocols that penalize physically impossible but visually plausible outputs. ## Scoring Rationale This is a notable conceptual contribution from a high-profile researcher that clarifies terminology and engineering boundaries for world models. It is not a new model or dataset release but is likely to influence evaluation and system design in vision, robotics, and multimodal agents. Practice interview problems based on real data 1,500+ SQL & Python problems across 15 industry datasets — the exact type of data you work with. [Try 250 free problems](/problems)