Latent-Dynamics World Models

Latent dynamicslatent dynamicsLatent dynamics is a world-model paradigm predicting in a compact learned latent space instead of pixels. The agent learns or plans entirely inside imagined rollouts of a recurrent latent state and reward under candidate actions. Dreamer established training an actor-critic purely on imagined trajectories. TD-MPC2 dropped the image decoder altogether, scoring action sequences under the latent model with a learned terminal value. is the world-model paradigm that learns a compact recurrent state, predicts how it and the reward evolve under actions, and trains or plans inside that imagination^{Source 1}1 Hafner et al. 2019, Dream to Control: Learning Behaviors by Latent Imagination. Danijar Hafner, Timothy Lillicrap, Jimmy Ba et al., ICLR 2020, 2019. It is the oldest of the six paradigms in the taxonomy: Embed to Control learned locally linear latent dynamics from raw images in 2015^{Source 9}9 Watter et al. 2015, Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images. Manuel Watter, Jost Tobias Springenberg, Joschka Boedecker et al., NIPS 2015.

Paper figure: Nicklas Hansen et al. / arXiv. Licence: CC BY 4.0.

Drag "How far ahead to imagine" in the figure and the imagined throw peels away from the real one. In measured models, compounding error is shaped most by the system's own dynamics^{Source 10}10 Lambert et al. 2022, Investigating Compounding Prediction Errors in Learned Dynamics Models. Nathan Lambert, Kristofer Pister, Roberto Calandra, 2022.

Agents trained inside a recurrent latent model reached human-level Atari

Ha and Schmidhuber trained an agent "entirely inside of its own hallucinated dream" and transferred it back to the real environment^{Source 11}11 Ha et al. 2018, World Models. David Ha, Jürgen Schmidhuber, 2018. PlaNet planned from pixels with deterministic and stochastic latent transitions^{Source 12}12 Hafner et al. 2018, Learning Latent Dynamics for Planning from Pixels. Danijar Hafner, Timothy Lillicrap, Ian Fischer et al., 2018. DreamerV3's Recurrent State-Space Model (RSSM) pairs a recurrent state that carries memory with a stochastic categorical representation, and learns to reconstruct inputs and predict reward and continuation^{Source 2}2 Hafner et al. 2023, Mastering Diverse Domains through World Models. Danijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., 2023.

Dreamer's actor and critic are trained purely on imagined latent rollouts from real encoded states, backpropagating value gradients through the model^{Source 1}1 Hafner et al. 2019, Dream to Control: Learning Behaviors by Latent Imagination. Danijar Hafner, Timothy Lillicrap, Jimmy Ba et al., ICLR 2020, 2019. DreamerV2, which added discrete latents, was the first agent to reach human-level performance on the 55-task Atari benchmark by learning inside a separately trained world model^{Source 13}13 Hafner et al. 2020, Mastering Atari with Discrete World Models. Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi et al., ICLR 2021, 2020.

DreamerV3: one recipe across 150 tasks

DreamerV3 was evaluated on more than 150 tasks across eight domains with fixed hyperparameters. It uses symlog transforms for vector observations, free bits with KL balancing and percentile-based return normalization^{Source 2}2 Hafner et al. 2023, Mastering Diverse Domains through World Models. Danijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., 2023. In the 2025 Nature article its authors call it, to their knowledge, the first algorithm to collect diamonds in Minecraft from scratch without human data or adaptive curricula^{Source 14}14 Hafner et al. 2025, Mastering diverse control tasks through world models. Danijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., Nature 640, 647-653, 2025. Its MineRL-based task used abstract crafting actions, accelerated block breaking and sparse intermediate rewards for 12 milestones from log to diamond^{Source 2}2 Hafner et al. 2023, Mastering Diverse Domains through World Models. Danijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., 2023.

DreamerV3 learned online; in 2025 Dreamer 4 became "the first agent to obtain diamonds in Minecraft purely from offline data", with a world model that runs in real time on a single GPU^{Source 15}15 Hafner et al. 2025, Training Agents Inside of Scalable World Models. Danijar Hafner, Wilson Yan, Timothy Lillicrap, 2025.

TD-MPC2 drops the decoder and plans at every step

MuZero planned with a model predicting only "the reward, the action-selection policy, and the value function"^{Source 16}16 Schrittwieser et al. 2019, Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert et al., 2019, and value-equivalence theory holds that capacity is "better used to build models that are directly useful for value-based planning"^{Source 17}17 Grimm et al. 2020, The Value Equivalence Principle for Model-Based Reinforcement Learning. Christopher Grimm, André Barreto, Satinder Singh et al., NeurIPS 2020. TD-MPC trains reward, TD value and latent-state consistency predictions without reconstructing observations, and at each step runs short-horizon MPPI with a learned terminal value^{Source 3}3 Hansen et al. 2022, Temporal Difference Learning for Model Predictive Control. Nicklas Hansen, Xiaolong Wang, Hao Su, ICML 2022. TD-MPC2 keeps this planner, adds candidate sequences from a policy prior and warm-starts from the previous solution.^{Source 4}4 Hansen et al. 2023, TD-MPC2: Scalable, Robust World Models for Continuous Control. Nicklas Hansen, Hao Su, Xiaolong Wang, ICLR 2024, 2023

The multitask experiment trained one 317M-parameter agent on 545M transitions from 240 single-task agents. This offline 80-task experiment is separate from the 104-task online benchmark, and aggregate normalized scores rose across five model sizes^{Source 4}4 Hansen et al. 2023, TD-MPC2: Scalable, Robust World Models for Continuous Control. Nicklas Hansen, Hao Su, Xiaolong Wang, ICLR 2024, 2023. MuDreamer replaced Dreamer's reconstruction with value and action prediction^{Source 18}18 Burchi et al. 2024, MuDreamer: Learning Predictive World Models without Reconstruction. Maxime Burchi, Radu Timofte, 2024.

DayDreamer taught a real quadruped to walk in about an hour

DayDreamer builds on DreamerV2 and learns online without a simulator on four robots with shared hyperparameters. After about one hour of real training, a Unitree A1 learned to roll off its back, stand and walk, with manual repositioning at the area boundary that left its joints and orientation unchanged. SAC learned to roll over but neither stood nor walked within that budget^{Source 5}5 Wu et al. 2022, DayDreamer: World Models for Physical Robot Learning. Philipp Wu, Alejandro Escontrela, Danijar Hafner et al., CoRL 2022.

Photo: Philipp Wu et al. / arXiv. Licence: CC BY 4.0.

Latent models imagine only 3 to 16 steps ahead

The 2023 DreamerV3 preprint lists imagination horizon H = 15 in a table and prediction horizon T = 16 in its actor-critic section, the 2024 revision keeps both labels, and the 2025 Nature article uses T = 16^{Source 2}2 Hafner et al. 2023, Mastering Diverse Domains through World Models. Danijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., 2023 ^{Source 14}14 Hafner et al. 2025, Mastering diverse control tasks through world models. Danijar Hafner, Jurgis Pasukonis, Jimmy Ba et al., Nature 640, 647-653, 2025. TD-MPC2 plans with H = 3 and bootstraps beyond it with a terminal value^{Source 4}4 Hansen et al. 2023, TD-MPC2: Scalable, Robust World Models for Continuous Control. Nicklas Hansen, Hao Su, Xiaolong Wang, ICLR 2024, 2023. DayDreamer states H = 16 in its methods and H = 15 in its appendix^{Source 5}5 Wu et al. 2022, DayDreamer: World Models for Physical Robot Learning. Philipp Wu, Alejandro Escontrela, Danijar Hafner et al., CoRL 2022.

MBPO keeps model rollouts short and "branched from real data"^{Source 19}19 Janner et al. 2019, When to Trust Your Model: Model-Based Policy Optimization. Michael Janner, Justin Fu, Marvin Zhang et al., NeurIPS 2019, and STEVE varies the horizon per example to limit model error^{Source 20}20 Buckman et al. 2018, Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion. Jacob Buckman, Danijar Hafner, George Tucker et al., 2018. HWM, with world models at several time scales in one latent space, lifted real Franka pick-and-place from 0% to 70%^{Source 21}21 Zhang et al. 2026, Hierarchical Planning with Latent World Models. Wancong Zhang, Basile Terver, Artem Zholus et al., 2026. A 350M-parameter model trained on 427 hours across 210 tasks still showed perceptual, action-marginalized and scene-diverging hallucinations^{Source 22}22 Hansen et al. 2026, Hallucination in World Models is Predictable and Preventable. Nicklas Hansen, Xiaolong Wang, 2026.

Simulation-trained world models now transfer to real legged robots

Robotic World Model (RWM) predicts observations over long horizons under partial observability with a GRU that feeds its predictions back in. Policies trained with MBPO-PPO in simulation showed zero-shot transfer to ANYmal D and Unitree G1, including under external disturbances; safe online hardware learning remains open^{Source 6}6 Li et al. 2025, Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics. Chenhao Li, Andreas Krause, Marco Hutter, 2025. RWM-U adds uncertainty and trains policies "entirely from offline datasets" for a real quadruped and humanoid^{Source 23}23 Li et al. 2025, Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robots. Chenhao Li, Andreas Krause, Marco Hutter, 2025. SimDist pretrains in simulation and on hardware updates only the latent dynamics; on contact-rich manipulation and quadruped locomotion it kept improving where prior adaptation methods stalled or degraded^{Source 29}29 Levy et al. 2026, Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation. Jacob Levy, Tyler Westenbroek, Kevin Huang et al., RSS 2026.

Refinements to latent planning bring uneven gains

Dream-MPC refines policy-proposed action sequences by gradient ascent on predicted return, with an uncertainty penalty. Used in place of MPPI in pretrained TD-MPC2 and BMPC on 24 simulated tasks, over three seeds and ten episodes, the BMPC variant raises interquartile mean by 26.7% and mean by 20.5%. The TD-MPC2 variant beats its policy alone but does not consistently match MPPI^{Source 7}7 Spieler et al. 2026, Dream-MPC: Gradient-Based Model Predictive Control with Latent Imagination. Jonathan Spieler, Sven Behnke, ICML 2026. BMPC learns its policy by imitating the MPC planner^{Source 24}24 Wang et al. 2025, Bootstrapped Model Predictive Control. Yuhang Wang, Hanwei Guo, Sizhe Wang et al., ICLR 2025, and HumanoidBench, one of the three task suites, poses whole-body humanoid tasks^{Source 25}25 Sferrazza et al. 2024, HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation. Carmelo Sferrazza, Dun-Ming Huang, Xingyu Lin et al., 2024.

TD-M(PC)² found that planner-generated data causes "persistent value overestimation" and fixed it with a policy constraint, gaining most on 61-DoF humanoids^{Source 26}26 Lin et al. 2025, TD-M(PC)²: Improving Temporal Difference MPC Through Policy Constraint. Haotian Lin, Pengcheng Wang, Jeff Schneider et al., 2025. Fast-WAM, a video world-action model, keeps video co-training but encodes the current frame once and runs action denoising on a chunk, with success similar to its imagine-then-execute variants on LIBERO and RoboTwin 2.0. On Galaxea R1 Lite towel folding it reports 190 ms inference latency against 810 ms for Fast-WAM-IDM on one RTX 5090D V2 32GB GPU^{Source 8}8 Yuan et al. 2026, Fast-WAM: Do World Action Models Need Test-time Future Imagination?. Tianyuan Yuan, Zibin Dong, Yicheng Liu et al., 2026.

TD-MPC2's authors expect broad generalization to need far more tasks

TD-MPC2's authors expect broad generalization, such as transfer to new embodiments, "will likely require several orders of magnitude more tasks"^{Source 4}4 Hansen et al. 2023, TD-MPC2: Scalable, Robust World Models for Continuous Control. Nicklas Hansen, Hao Su, Xiaolong Wang, ICLR 2024, 2023. In November 2025 the same group's Newt trained one language-conditioned world model jointly across 200 tasks^{Source 27}27 Hansen et al. 2025, Learning Massively Multitask World Models for Continuous Control. Nicklas Hansen, Hao Su, Xiaolong Wang, 2025. In simulation, one world model trained on up to 90 insertion tasks reached 56% zero-shot success on unseen parts, against 7% for a model-free baseline^{Source 30}30 Hansen et al. 2026, Generalizable Robotic Insertion with World Models. Nicklas Hansen, Iretiayo Akinola, Yijie Guo et al., IROS 2026. DayDreamer's authors report that long hardware training causes wear needing human intervention or repair^{Source 5}5 Wu et al. 2022, DayDreamer: World Models for Physical Robot Learning. Philipp Wu, Alejandro Escontrela, Danijar Hafner et al., CoRL 2022.

Imagination tolerates slow models; online planning needs predictable latency

Imagination-only training, as in Dreamer, tolerates slow generation but can bias the policy silently. Online planning, as in TD-MPC2, needs predictable latency and a fallback, and a planner that never changes its policy prior's proposal pays model cost for nothing. A latent model can also supply a representation: a simulation-trained world model predicted real legged-locomotion trajectories well enough to inform a separate controller^{Source 28}28 Lai et al. 2024, World Model-based Perception for Visual Legged Locomotion. Hang Lai, Jiahang Cao, Jiafeng Xu et al., 2024. See the model-based and evaluation guides.