Robot Wiki

RL, Sim-to-Real & Locomotion

Reinforcement learning at scale, from massively parallel simulation to the transfer problem.

This domain overview lists 8 articles, ordered as why reinforcement learning took locomotion before manipulation, then the parallel-simulation machinery behind that result, then the transfer gap it left over.

  1. RL for Robotics

    Sample efficiency decides which reinforcement learning algorithms a robot can actually be trained with, from PPO in simulation to offline learning on a fixed dataset.

  2. Why RL Won Locomotion but Not Manipulation

    The MDP simulability gap: contact-rich manipulation resists the simulation that made walking routine.

  3. Massively Parallel Sim RL

    Isaac Lab, Newton, MJX, and Brax: GPU-parallel environments and the wall-clock economics of training.

  4. Sim-to-Real Transfer

    Domain randomization, teacher-student distillation, system identification, and real-to-sim correction.

  5. Legged Locomotion Lineage

    From ANYmal to Unitree and the MIT humanoid line: how learned gaits became the default.

  6. Humanoid Whole-Body Control

    Motion tracking from PHC to ASAP and GMT, and the three decompositions of 2026.

  7. Reward Design and the MPC Debate

    LLM-written rewards and curricula; where classical trajectory optimization still wins.

  8. Offline Reinforcement Learning for Robotics

    CQL, IQL, TD3+BC and Q-Transformer under the support constraint, with a decision rule for when behavior cloning is the stronger baseline.