Latent-Dynamics World Models

Of the six paradigms the taxonomy module separates, latent dynamicslatent dynamicsA world-model paradigm that predicts in a compact learned latent space instead of in pixels: the model carries a recurrent latent state, forecasts how that state and the reward evolve under candidate actions, and the agent learns or plans entirely inside the imagined rollouts. Dreamer established the recipe of training an actor-critic purely on imagined trajectories, and TD-MPC2 showed the image decoder can be dropped altogether, scoring candidate action sequences under the latent model with a learned terminal value instead of reconstructing pixels. is the oldest and the only one with a decade of evidence that it improves sample efficiency on control. The idea is one sentence: learn a compact recurrent latent state, predict how that state and the reward evolve under the agent's actions, and do the learning or the planning entirely inside that imagination, without touching the real environment. Everything else, the decoder versus no-decoder split, the actor versus planner split, is a design choice on top of that sentence.

The interactive is a deterministic toy of compounding error, not a measured rollout from any of the papers. Drag the horizon slider to explore its assumed recurrence; the 3 to 15-step shaded band is illustrative, not a published reliable-horizon range. Table W.1 of the 2023 DreamerV3 preprint lists imagination horizon H = 15, while the same document's actor-critic section uses prediction horizon T = 16. The 2024 revision and 2025 Nature article also use T = 16. The two horizon labels have different values even within the 2023 preprint Hafner 2023↓Mastering Diverse Domains through World ModelsDanijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., Nature 2025, 2023.

The RSSM: a state you can dream in

In DreamerV3, the Recurrent State-Space Model (RSSM) combines a recurrent state that carries memory with a stochastic categorical representation. An encoder infers the representation from the recurrent state and observation. From the combined state, the model reconstructs inputs during training and predicts reward and episode continuation Hafner 2023↓Mastering Diverse Domains through World ModelsDanijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., Nature 2025, 2023.

The payoff is where learning happens. The actor and the critic are trained purely on imagined latent rollouts: start from a real encoded state, unroll the learned dynamics for a short horizon, and backpropagate value gradients through the learned model to improve the policy Hafner 2019↓Dream to Control: Learning Behaviors by Latent ImaginationDanijar Hafner, Timothy Lillicrap, Jimmy Ba et al., ICLR 2020, 2019. This imagination phase reuses the learned world model without collecting new environment transitions Hafner 2019↓Dream to Control: Learning Behaviors by Latent ImaginationDanijar Hafner, Timothy Lillicrap, Jimmy Ba et al., ICLR 2020, 2019. The environment is only needed to collect the experience the model is fitted to.

DreamerV3: one recipe across 150 tasks

DreamerV3 was evaluated on more than 150 tasks across eight domains with fixed hyperparameters. Its robustness techniques include symlog transforms for vector observations, free bits with KL balancing, and percentile-based return normalization. The paper's reward-loss descriptions differ: the detailed distribution section describes two-hot targets for reward and value prediction, whereas an earlier world-model paragraph says the reward predictor uses symlog squared loss Hafner 2023↓Mastering Diverse Domains through World ModelsDanijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., Nature 2025, 2023.

The headline result is Minecraft. In their 2025 Nature article, Hafner, Pasukonis, Ba, and Lillicrap describe DreamerV3 as, to their knowledge, the first algorithm to collect diamonds in Minecraft from scratch without human data or adaptive curricula. The experiment used a MineRL-based environment with abstract crafting actions, accelerated block breaking, and sparse intermediate rewards. Its 12 rewarded milestones are log, plank, stick, crafting table, wooden pickaxe, cobblestone, stone pickaxe, iron ore, furnace, iron ingot, iron pickaxe, and diamond Hafner 2023↓Mastering Diverse Domains through World ModelsDanijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., Nature 2025, 2023.

TD-MPC and TD-MPC2: drop the decoder, plan at every step

The TD-MPC line takes the opposite position on the decoder. TD-MPC jointly trains reward prediction, TD value prediction, and latent-state consistency without reconstructing observations. At each control step it uses short-horizon MPPI trajectory optimization and a learned terminal value to estimate return beyond the rollout Hansen 2022↓Temporal Difference Learning for Model Predictive ControlNicklas Hansen, Xiaolong Wang, Hao Su, ICML 2022. TD-MPC2 learns joint-embedding, reward, and value predictions without reconstructing observations. At each decision step, MPPI evaluates sampled action sequences through latent rollouts, combining predicted rewards with a learned terminal value. Some candidate sequences come from the learned policy prior; planning is warm-started from the previous decision step's solution shifted by one step. The first planned action is executed. Hansen 2023↓TD-MPC2: Scalable, Robust World Models for Continuous ControlNicklas Hansen, Hao Su, Xiaolong Wang, ICLR 2024, 2023

The multitask experiment trained a single reported 317M-parameter configuration on 545M replay-buffer transitions from 240 single-task agents: 50 Meta-World tasks and 30 DMControl tasks, spanning different embodiments and action spaces. This offline 80-task experiment is separate from the 104-task online benchmark. Across five tested model sizes, the authors report increasing aggregate normalized scores, averaging Meta-World success rates and normalized DMControl episode returns; this is not a claim of perfect success on every task Hansen 2023↓TD-MPC2: Scalable, Robust World Models for Continuous ControlNicklas Hansen, Hao Su, Xiaolong Wang, ICLR 2024, 2023. Unlike a reconstruction-based world model, TD-MPC2 does not produce predicted observation images. It selects actions by planning, with its policy network supplying a fraction of candidate trajectories Hansen 2023↓TD-MPC2: Scalable, Robust World Models for Continuous ControlNicklas Hansen, Hao Su, Xiaolong Wang, ICLR 2024, 2023.

DayDreamer: imagination on physical hardware

DayDreamer builds on the DreamerV2 implementation and learns online on four physical robots without a simulator: Unitree A1, UR5, XArm, and Sphero Ollie. The experiments use the same learning hyperparameters, not identical sensor inputs, action spaces, or control rates Wu 2022↓DayDreamer: World Models for Physical Robot LearningPhilipp Wu, Alejandro Escontrela, Danijar Hafner et al., CoRL 2022. In one A1 training run, the robot learned to roll off its back, stand, and walk forward with a pronking gait after about one hour of real-world training. The setup used 20 Hz joint-angle commands realized by a PD controller, filtered motor commands, and manual repositioning at the training-area boundary without changing the robot's joint configuration or orientation. SAC learned to roll over but did not stand or walk within that experiment's small data budget; the paper does not measure how many days SAC would need. The one-hour result is training time, not inference latency or the duration of one rollout Wu 2022↓DayDreamer: World Models for Physical Robot LearningPhilipp Wu, Alejandro Escontrela, Danijar Hafner et al., CoRL 2022.

The horizon problem

Table W.1 of the 2023 DreamerV3 preprint lists imagination horizon H = 15, while the same document's actor-critic section uses prediction horizon T = 16. The 2024 revision and 2025 Nature article also use T = 16. The two horizon labels have different values even within the 2023 preprint Hafner 2023↓Mastering Diverse Domains through World ModelsDanijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., Nature 2025, 2023. In TD-MPC2 v2, Table 8 lists planning horizon H = 3; the planner bootstraps beyond the rollout with a learned terminal value Hansen 2023↓TD-MPC2: Scalable, Robust World Models for Continuous ControlNicklas Hansen, Hao Su, Xiaolong Wang, ICLR 2024, 2023. DayDreamer v1 likewise gives two labels: its actor-critic methods state a planning horizon H = 16, while Appendix D lists imagination horizon H = 15. Its methods and appendix thus report different horizon values Wu 2022↓DayDreamer: World Models for Physical Robot LearningPhilipp Wu, Alejandro Escontrela, Danijar Hafner et al., CoRL 2022. These source settings are step counts, not control frequencies or universal bounds on reliable prediction.

Both display modes in the figure are schematic. The Dreamer-labelled mode draws illustrative frames; the TD-MPC2-labelled mode omits reconstruction and shows a toy reward-error readout equal to 0.35 times the toy latent deviation. The curves, frames, and shaded band are not measured model predictions, published losses, or reliability estimates.

Successors and open limits

The lineage is still moving. Robotic World Model (RWM) uses a GRU-based model to predict observations over long horizons under partial observability. Its dual-autoregressive mechanism updates hidden states across the observation-action history and feeds predicted observations back into later predictions. The authors train velocity-tracking policies with MBPO-PPO in simulation and report zero-shot transfer to ANYmal D and Unitree G1, including tests under external disturbances. Safe online policy learning directly on hardware remains a limitation Li 2025↓Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in RoboticsChenhao Li, Andreas Krause, Marco Hutter, 2025. Dream-MPC samples candidate action sequences from a policy rolled out through a learned latent world model, then refines the actions by gradient ascent on predicted return with an uncertainty penalty. It mixes policy proposals with previously optimized actions and executes only the first action before replanning. Spieler and Behnke replace MPPI at test time in pretrained TD-MPC2 and BMPC models. Their main comparison covers 24 simulated continuous-control tasks: eight each from DeepMind Control Suite, HumanoidBench, and Meta-World, with three random seeds and ten test episodes. On the pooled normalized scores, the BMPC-based variant improves interquartile median by 26.7% and mean by 20.5% relative to BMPC; the TD-MPC2-based variant improves on its policy alone but does not consistently match MPPI. These are aggregate, model-specific results, not wins on every task or a hardware experiment. Six image-observation tasks and a four-environment study using the planner during training are separate experiments Spieler 2026↓Dream-MPC: Gradient-Based Model Predictive Control with Latent ImaginationJonathan Spieler, Sven Behnke, ICML 2026. Fast-WAM retains video co-training but removes explicit future-video generation at inference: its video backbone encodes the current frame once, then an action expert still denoises an action chunk. Its controlled comparisons report similar success rates to its imagine-then-execute variants on LIBERO and RoboTwin 2.0. On the Galaxea R1 Lite towel-folding task, the paper reports inference latency of 190 ms for Fast-WAM versus 810 ms for Fast-WAM-IDM, measured on a single NVIDIA RTX 5090D V2 32GB GPU. These are inference latencies, not robot-control frequencies Yuan 2026↓Fast-WAM: Do World Action Models Need Test-time Future Imagination?Tianyuan Yuan, Zibin Dong, Yicheng Liu et al., 2026.

The primary papers describe narrower limits than a verdict on the whole paradigm:

• Generalization remains open. TD-MPC2's authors describe transfer to new embodiments and integration with vision-language models as future opportunities, and say that this level of generalization will likely require several orders of magnitude more tasks. Extending their method to discrete action spaces is also an open problem Hansen 2023↓TD-MPC2: Scalable, Robust World Models for Continuous ControlNicklas Hansen, Hao Su, Xiaolong Wang, ICLR 2024, 2023.
• Hardware learning has costs. DayDreamer's authors note that many hours of learning can cause wear requiring human intervention or repair. They also call for longer training to establish the limits of both Dreamer and its baselines Wu 2022↓DayDreamer: World Models for Physical Robot LearningPhilipp Wu, Alejandro Escontrela, Danijar Hafner et al., CoRL 2022.

The one-hour A1 run and TD-MPC2's offline 80-task experiment address different questions. Neither establishes a universal horizon, a control-rate ceiling, or a field-wide advantage over every other world-model paradigm.

Pick the model's role before its architecture

A latent model can serve three different roles. It can generate imagined experience for actor and critic training, as in Dreamer. It can score candidate action sequences online, as in TD-MPC2. Or it can supply a representation while a separate policy acts. Each role has a different failure budget. Training-only imagination may tolerate slow generation but can bias the learned policy silently; online planning needs predictable latency and a fallback; representation learning needs evidence that the latent preserves task-relevant distinctions.

Write the loop before selecting the loss: observation encoder, latent update, action proposal, rollout horizon, score, replanning rate and stop condition. Evaluate multi-step state usefulness and policy ranking, not reconstruction alone. Log planner improvement over its policy prior, because a planner that never changes the proposed action is paying model cost without using the model. The model-based robot-learning guide covers these loops, and the evaluation guide defines evidence for each role.