What Is a World Model?

A world modelworld modelA model of how an environment evolves that an agent can query to make decisions: given the current observation or state and a candidate action, it predicts what happens next, so a policy can be trained, evaluated, or planned against the model instead of the real world. The 2026 robotics survey draws the functional line: producing plausible future images is not enough, because a system qualifies only if its predictions change under the agent's action in a way that is useful for decision-making. The single name covers at least six architecturally distinct paradigms, from compact latent dynamics models to action-conditioned video generators. predicts how an environment evolves in a form useful for a robot's decisions. Hou and colleagues' 2026 survey uses this functional definition for visual, latentlatent dynamicsA world-model paradigm that predicts in a compact learned latent space instead of in pixels: the model carries a recurrent latent state, forecasts how that state and the reward evolve under candidate actions, and the agent learns or plans entirely inside the imagined rollouts. Dreamer established the recipe of training an actor-critic purely on imagined trajectories, and TD-MPC2 showed the image decoder can be dropped altogether, scoring candidate action sequences under the latent model with a learned terminal value instead of reconstructing pixels., structured, and symbolic predictions; it does not require one architecture. Hou 2026↓World Model for Robot Learning: A Comprehensive SurveyBohan Hou, Gen Li, Jindou Jia et al., 2026

This article uses three questions as its organizing lens: what is predicted, in what representation, and for what purpose? The questions are an editorial comparison, not the survey's own three-axis taxonomy. The survey instead organizes its main sections around policy-coupling architectures, simulator applications, and video-world-model capabilities Hou 2026↓World Model for Robot Learning: A Comprehensive SurveyBohan Hou, Gen Li, Jindou Jia et al., 2026.

The six example groups below are this article's selection, not an exhaustive or universally agreed scientific taxonomy. Their panels compare different prediction targets and uses; selecting a panel is not a benchmark comparison between the named systems.

The functional cut

Hou and colleagues define a world model by whether it predicts how the future changes under robot-relevant actions in a way that supports embodied decision-making. They write: "a model does not qualify as a world model in our sense simply because it generates plausible future images or videos". Their action definition includes both low-level motor commands and high-level language or task instructions; this is the survey's functional scope, not a universal naming rule Hou 2026↓World Model for Robot Learning: A Comprehensive SurveyBohan Hou, Gen Li, Jindou Jia et al., 2026.

In Section 8.1, "Causal Conditioning Gaps," the survey identifies weak action conditioning as a technical bottleneck. The authors say many predictive objectives are trained mainly from observation history and task intent, allowing semantically plausible futures that are not causally tied to the pending robot action. This is their diagnosis, not a ranking of the field's challenges or a finding that every model below already satisfies the criterion Hou 2026↓World Model for Robot Learning: A Comprehensive SurveyBohan Hou, Gen Li, Jindou Jia et al., 2026.

Six example groups

Latent dynamics: predict a compact state, dream in it

The oldest learned lineage and the only one with a decade of evidence on sample-efficient control. DreamerV3 uses a recurrent state-space model with recurrent memory and stochastic latent representations, and reconstructs inputs during world-model learning. Its actor and critic learn from imagined trajectories; the critic also receives a loss on replay-buffer trajectories Hafner 2023↓Mastering Diverse Domains through World ModelsDanijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., Nature 2025, 2023.

DreamerV3 tested this approach across more than 150 tasks in eight domains with fixed hyperparameters. The paper describes reward prediction differently in its distribution section (two-hot reward/value targets) and earlier world-model paragraph (symlog squared loss) Hafner 2023↓Mastering Diverse Domains through World ModelsDanijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., Nature 2025, 2023. The latent-dynamics explanation covers the robustness techniques and this disagreement.

In their 2025 Nature article, Hafner, Pasukonis, Ba, and Lillicrap describe DreamerV3 as, to their knowledge, the first algorithm to collect diamonds in Minecraft from scratch without human data or adaptive curricula. The experiment used a MineRL-based environment with abstract crafting actions, accelerated block breaking, and sparse intermediate rewards Hafner 2023↓Mastering Diverse Domains through World ModelsDanijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., Nature 2025, 2023. DayDreamer used the DreamerV2 implementation, not DreamerV3. In one Unitree A1 run, it learned to roll off its back, stand, and walk forward with a pronking gait after about one hour of real-world training, without a simulator or reset policy. The experiment still used a PD controller and manual repositioning at the training-area boundary Wu 2022↓DayDreamer: World Models for Physical Robot LearningPhilipp Wu, Alejandro Escontrela, Danijar Hafner et al., CoRL 2022.

Decoder-free latent: no image, just value

TD-MPC2 learns joint-embedding, reward, and value predictions without reconstructing observations. At each decision step, MPPI evaluates sampled action sequences through latent rollouts and combines predicted rewards with a learned terminal value. The policy prior supplies some candidate sequences; the shifted previous planning solution provides the warm start, and the first planned action is executed. Its reported 317M-parameter multitask configuration was trained offline on 545M transitions from 240 single-task agents, covering 50 Meta-World and 30 DMControl tasks across different embodiments and action spaces. That 80-task experiment is distinct from the 104-task online benchmark Hansen 2023↓TD-MPC2: Scalable, Robust World Models for Continuous ControlNicklas Hansen, Hao Su, Xiaolong Wang, ICLR 2024, 2023.

Generative video: predict the pixels

These models predict future pixels from current frames plus an action or text condition, and they are where the money is. NVIDIA presents Cosmos 3 as a common Mixture-of-Transformers architecture for understanding, generation, forward dynamics, inverse dynamics, and policy. It uses autoregressive and diffusion subsequences with separate parameter sets: diffusion attends to both streams, while the autoregressive stream remains causal and does not read diffusion tokens. The report distinguishes the base models from task-specific post-trained variants NVIDIA 2026↓Cosmos 3: Omnimodal World Models for Physical AINVIDIA, Aditi, Niket Agarwal et al., 2026. DeepMind reports that Genie 3 generates text-prompted worlds navigable in real time at 24 fps and 720p, remaining largely consistent for a few minutes. Its blog lists constrained direct agent actions and a few minutes of continuous interaction among the model's limitations Parker-Holder 2025↓Genie 3: A new frontier for world modelsJack Parker-Holder, Shlomi Fruchter, Google DeepMind, 2025. For the broader research literature, Hou and colleagues distinguish learned simulation for reinforcement learning, rollout-based evaluation, and video generation for supervision. They also identify action faithfulness, rollout reliability, and functional evaluation as open problems. This is a map of research uses, not a deployment census for Cosmos 3 or Genie 3, nor evidence that either replaces a general-purpose physics engine Hou 2026↓World Model for Robot Learning: A Comprehensive SurveyBohan Hou, Gen Li, Jindou Jia et al., 2026.

JEPA: predict the representation, never the pixels

V-JEPA 2 uses action-free pretraining on over a million hours of internet video plus images. V-JEPA 2-AC then freezes the encoder and trains a separate action-conditioned predictor on less than 62 hours of filtered left-camera Droid video, using end-effector state signals and actions derived from state changes. “Unlabeled” means no reward, task-type, or success labels; Droid includes Franka Panda teleoperation Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025.

For evaluation, the same weights and inference code run zero-shot on Franka Emika Panda arms with RobotiQ grippers in two labs absent from Droid, without training on those evaluation robots. Camera placement was manually selected. This is a new-environment result, not universal new-embodiment transfer. Planning minimizes predicted-to-goal feature-map L1 distance with the Cross-Entropy Method, executes the first action, and replans. Pick-and-place uses two supplied subgoal images plus the final goal Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025. The JEPA module details these stages and their limitations.

Unified world-action: one backbone, two heads

Here the world model and the policy fuse into one backbone that emits future frames and action chunks together, so world modeling acts as an auxiliary objective that shapes the representations the policy reads. WorldVLA interleaves action and image tokens autoregressively and reports that the two objectives improve each other Cen 2025↓WorldVLA: Towards Autoregressive Action World ModelJun Cen, Chaohui Yu, Hangjie Yuan et al., 2025; Cosmos Policy is the same idea on top of the Cosmos backbone, post-training the Cosmos-Predict2 video model into a robot policy with no architectural modifications Kim 2026↓Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and PlanningMoo Jin Kim, Yihuai Gao, Tsung-Yi Lin et al., 2026. The prediction is real, but its job is to make the policy better, not to be inspected or rolled out interactively.

Symbolic and structured: predict predicates, not percepts

These models predict transitions over predicates, object relations, or 3D occupancy instead of anything continuous. OccWorld, from autonomous driving, predicts how a scene's occupancy grid evolves and produces competitive planning results without instance or map supervision Zheng 2023↓OccWorld: Learning a 3D Occupancy World Model for Autonomous DrivingWenzhao Zheng, Weiliang Chen, Yuanhui Huang et al., 2023. The appeal is long-horizon: discrete relational state does not accumulate pixel-space error the way a 50-step video rollout does. The cost is that someone, or some perception stack, has to produce the symbols.

Two boundary cases

Two systems are routinely called world models and deserve a precise verdict.

A physics engine is a world model

MuJoCo is an explicit, hand-specified state-transition model: given a state and an action, it predicts the next state, and its predictions are nothing if not action-dependent Todorov 2012↓MuJoCo: A physics engine for model-based controlEmanuel Todorov, Tom Erez, Yuval Tassa, IROS 2012. It is simply not learned. This is not a pedantic point. TD-MPC2 and classical MPCmodel predictive controlControl by constant re-planning: at each control step, optimize a short sequence of future actions against an explicit dynamics model, execute only the first action, and solve again from the freshly measured state. Because the plan is recomputed online, model error is rejected by feedback at every step instead of being frozen into a policy's weights, so a model-predictive controller shows less of the frozen-policy sim-to-real failure mode, though a wrong dynamics model still costs it closed-loop performance. Long assumed too slow for a robot's full dynamics, the method reached whole-body scale in 2026, when Zhang and colleagues solved iLQR against MuJoCo's physics fast enough to close the loop in real time on real hardware. are the same algorithm with one difference, whether the dynamics model is learned or derived, and much of the RL-versus-MPC debate in the locomotion domain reduces to that choice.

A 3D Gaussian Splatting twin is not

A 3DGS reconstruction of a real scene is a learned renderer: it predicts what the scene looks like from a new camera pose Kerbl 2023↓3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler et al., ACM Trans. Graph. 42(4), author manuscript (2023). When such a twin is used for robot training, the dynamics come from the physics engine the splats are bolted onto. Appearance is learned, physics is not, and a renderer alone fails the functional cut because it has no answer to what happens under an action.

What each paradigm is actually for

Return to the panel grid above to compare this article's six example groups. The pattern to internalize: the paradigm with the loudest name recognition, generative video, earns its keep in evaluation and data generation, while the two paradigms that plan (decoder-free latent, JEPA) never show you an image.

Reading world-model claims

When a paper or a vendor says "world model," use this article’s three comparison questions: what is predicted (latent, pixels, embedding, predicates), in what space, and for what purpose (policy learning, planning, evaluation, data generation). If the predictions do not change under the agent's action, the system fails the functional cut no matter how good the video looks.

Three tests before believing a world-model claim

First apply the intervention test: hold observation history fixed, vary the candidate action and check whether predicted consequences change in the correct direction. Second apply the ranking test: ask whether the model orders policies or action sequences the same way the real system does. Third apply the transfer test: repeat both checks across the objects, contacts, viewpoints and horizons named in the claim. Visual quality is useful only after those tests pass.

Then locate the model in an actual decision loop. A model used as an auxiliary training loss needs different evidence from one used for online planning or as a simulator of record. State what consumes the prediction, how often, at what cost and what happens when uncertainty grows. The evaluation guide turns these tests into a protocol; model-based robot learning shows the planning loop; and world models versus simulators separates learned, physical and hybrid transition models.

Self-check

A vendor pitches a photorealistic 3D Gaussian Splatting reconstruction of your workcell as a world model for training a manipulation policy. What is the honest engineering verdict?

It qualifies: it predicts the scene from any camera pose, which is a predictionIt is a learned renderer bolted onto a physics engine: appearance is learned, dynamics are notIt fails the cut because nothing hand-specified counts as a world model

Read the reasoning

• It qualifies: it predicts the scene from any camera pose, which is a predictionThis is the most common misreading of the real-to-sim literature. A 3DGS twin is a learned renderer: it predicts appearance from a new viewpoint and has no answer to what happens under an action. The survey’s functional cut is explicit on this: generating plausible future images alone does not qualify a model as a world model.
• It is a learned renderer bolted onto a physics engine: appearance is learned, dynamics are notThis is the correct division of labor. The splat supplies appearance while a conventional physics engine still supplies dynamics, and the reconstruction freezes the scene it captured: articulated objects, deformables, and lighting changes each need extra machinery on top.
• It fails the cut because nothing hand-specified counts as a world modelThe cut is about action-dependence, not about being learned. MuJoCo is an explicit, hand-specified state-transition model and it qualifies: given a state and an action it predicts the next state. The 3DGS twin fails for having no action answer, not for lacking weights.

Ask what happens under an action: a renderer that only answers where the camera goes fails the functional cut, whatever the video looks like.