Model-based robot learning is the family of methods that learn how the world responds to actions and use that model to plan, to train a policy in imagination, or both. Its two steps are learning a dynamics model and integrating planning with learningSource 10.

A black and yellow Unitree A1 quadruped robot lying on its side on a grey floor, legs bent, with a cable trailing behind it.
With a Dreamer-style world model, a Unitree A1 that started on its back learned to roll over, stand up and walk in about one hour of real-world training. A frame from the DayDreamer experiment.
Photo: Philipp Wu et al. / arXiv. Licence: CC BY 4.0.

The loop predicts futures under candidate actions, scores them, acts, observes the real transition and updates the model. Sutton proposed integrated architectures for learning, planning and reacting in 1990Source 11.

Self-check

A learned dynamics model produces sharp videos but predicts nearly the same future for three different gripper actions. What is the diagnosis?
Read the reasoning
  • The model is useful for planning because its videos look photorealisticVisual fidelity cannot select among actions when the rollout ignores the candidate action. The model may be a good generator and a poor control model.
  • The model has action-conditioning collapse and cannot support planningPlanning requires counterfactual separation: different actions must produce meaningfully different predicted consequences. Without it, every candidate receives nearly the same evidence.
  • The planner needs a longer prediction horizonA longer rollout compounds the same missing action dependence. Horizon cannot recover information the one-step transition failed to encode.

A control model earns its name by changing its prediction when the candidate action changes.

Model-based means predicting the effect of an action

A 2026 survey defines a world model as "an action-conditioned predictive system"Source 13. Classical model-predictive control (MPC) uses explicit dynamics, such as MuJoCo's rigid-body and contact modelSource 6; model-based learning fits some or all of that transition to data.

A world model need not predict pixels: value-equivalent models predict only what planning needs, "without compromising performance"Source 14.

A model can plan each step or train a policy in imagination

A planner rolls sampled action sequences through the model, executes the best first action and replans, so a new goal or cost needs no retraining, at the price of many rollouts per real step (reward design and MPC). In 2016 Finn and Levine combined action-conditioned video prediction with MPC so a real robot could push novel objectsSource 15. TD-MPC2 trained one 317M-parameter agent on 80 tasksSource 4, and PointWorld, trained on about 2 million trajectories, plans with MPC at 0.1 s per inference on a real Franka without demonstrationsSource 30.

Dreamer instead trains a policy on trajectories imagined in a learned latent spaceSource 1, and DreamerV3 covers more than 150 tasks with one configurationSource 2. Dreamer 4 learns purely offline, a setting its authors tie to robotics, where interaction "can be unsafe and slow"Source 16. Latent dynamics covers these agents.

Uncertainty estimates keep policies from exploiting model errors

A policy trained on a model "tends to exploit regions where insufficient data is available", which ME-TRPO counters with an ensembleSource 17. PETS, a probabilistic ensemble, matches model-free asymptotic performance with 8 and 125 times fewer samples than SAC and PPO on half-cheetahSource 18. MBPO keeps rollouts short and "branched from real data"Source 19. In offline RL, MOPO penalizes reward by model uncertaintySource 20.

Robots see only part of the state, yet learn useful models in minutes to hours

A robot observation is partial: cameras miss forces, friction and compliance, and RGBD gave an 80% improvement in fabric-folding success over RGBSource 21. Data is narrow too: Hansen and Wang call world-model hallucination "inherently a data coverage issue" and adapt a model to unseen simulated environments with 50 trajectoriesSource 22.

Chenyuan Zhou, Yining Hong, Jiajun Wu and colleagues at Stanford and elsewhere (October 2026) condition a video world model on a 32-dimensional latent for hidden factors such as friction or mass, fitted at test time from one or two interactions while the model stays frozenSource 31. Averaged over four real-robot tasks, the adapted model's mean tracking error fell from 19.30 to 10.85 pixels against the same model without a latentSource 31. Actions it picked succeeded 70.80% of the time against 26.47% for the model without a latent, though the paper does not state trial countsSource 31.

PDDM learned contact-rich manipulation on a 24-DoF hand from 4 real hoursSource 23. DayDreamer ran Dreamer online on 4 robots without simulators; a quadruped learned to stand up and walk in 1 hourSource 3. SSRL adds a learned force ensemble to known Lagrangian dynamics and learned Go1 gaits from a few minutes of dataSource 24.

In September 2026, Claudio Canales, Marco Hutter and colleagues learned a probabilistic dynamics ensemble from scratch on an 11.5-ton hydraulic excavator. After 20 minutes, their sampling-based MPC tracked comparably to learned controllers trained on 100 to 150 minutes; after 40 it held sub-centimeter mean path error at high speedSource 8. Hansen, Fox and colleagues trained one TD-MPC2 model in simulation on up to 90 insertion tasks, with 56% zero-shot success on unseen objects against 7% model-freeSource 9.

Learning on the machine
An excavator learned its own arm in 20 minutes
Minutes of practice on the machine
Faint: the five models' guesses. Blue: where the shovel went20 minutes · matches earlier controllers

Checkpoints from the paper; the path and the guesses' spread are illustrative

Learning a model from its own moves, the excavator tracked paths after 20 minutes as well as earlier controllers trained for 100 to 150.
How this was made

Canales, Nan, Hutter and Ruiz-del-Solar learned a model of an 11.5-ton Menzi Muck M445 walking excavator's arm from scratch on the machine itself, with no demonstrations and no simulator. Its sensors measure the joints but not the oil pressures or the valves' inner state, so the model reads a short history of joint positions and valve commands. The model is an ensemble of five networks, refitted after every ten trajectories; a sampling-based planner tries thousands of command sequences through it every control cycle and sends the first command of the best one.

The drawing is that loop on a practice yard. A stylus on the shovel inks where the shovel really went (blue) against the path engraved on the board, and five faint lines ahead of it are the five models' guesses of where it goes next. The trainer's dial counts minutes of practice; its band marks the 100 to 150 minutes of machine data the earlier learned controllers were given. The paper's snapshots show large misses after 3 minutes, the path followed more closely after 10 and consistent tracking after 20. After 20 minutes the shovel tracked its timed path 2.7 cm off on average, against 2.8 and 4.5 cm for the earlier controllers; after 40 minutes every objective kept it under a centimetre off the path on average.

Illustrative: the path, the misses drawn at 3 and 10 minutes (shaped like the paper's snapshots, not measured) and how far the guesses fan out. At 20 and 40 minutes the misses are drawn at the paper's mean errors, too small to see at this scale. The machine is drawn with its arm in one plane, the telescopic extension sliding out as it reaches further.

Canales, Nan, Hutter and Ruiz-del-Solar (2026), Fig. 5 and Table II.

Model error is task-weighted

A two-millimeter error is irrelevant in free space and decisive at a tight connector. One-step prediction likelihood "is not always correlated with control performance", and task-effective models "do not necessarily need to be globally accurate"Source 12. The 2026 robot world-model survey stresses action-conditioned consistency and functional-utility evaluationSource 7; the evaluation guide gives a protocol.

Neural and hybrid simulators now train real-robot policies

Robotic World Model trains a neural-network simulator for policy optimizationSource 5, and RWM-U adds epistemic uncertainty to train real quadruped and humanoid policies entirely offlineSource 25. RISE improves a VLA inside a video world model scored by a progress-value model, gaining 35 to 45 points on three real tasksSource 26.

Two real tabletop robot images with instructions, wipe the table and remove the bowl from the plate, each turned into a row of generated frames set in a spacecraft interior and a disaster site, with a generalist robot policy acting in the loop.
A neural simulator can stage scenes no lab has. RoboWorld turns real robot images into spacecraft interiors and disaster sites and runs closed-loop policy evaluation inside the generated rollouts.
Paper figure: Byeongguk Jeon et al. / arXiv. Licence: CC BY 4.0.

Hybrids keep known physics and learn what it misses: TossingBot learns residuals on a physics-based throw and reached 500+ mean picks per hourSource 27, and ASAP fits a delta action model to real Unitree G1 dataSource 28. World models versus physics simulators compares the options.

Model-based learning pays off when real interaction is expensive

It fits when interaction is expensive, goals change faster than dynamics and planning fits the latency budget; it fits less well when observations omit contact state or resets are cheap. Plan only once predicted action rankings track real ones: in a 2026 agile-driving study, SAC won in simulation, yet "only TD-MPC2 consistently outperforms the MPPI baseline on the physical platform"Source 29.

Adrian Remonda, Nicklas Hansen, Xiaolong Wang and colleagues at Toyota Research Institute and UC San Diego (October 2026) pretrain a racing world model in simulation and correct its mismatch on the real carSource 32. On one track, with 44 minutes of real-car data, 1.13% of its buffer, all 8 runs succeeded, with a best sector time of 49.55 s against 52.47 s for an MPC reference; without the simulation priors, a real-data-only run was stopped for safetySource 32.