Legged Locomotion Lineage

The previous modules built the machinery: massively parallel simulation that trains a walking policy in minutes, and the transfer techniques that get it onto hardware. This module is where that machinery was proven. Legged locomotionlegged locomotionLocomotion through legs rather than wheels or tracks: support comes from discrete, intermittent footholds, so balance is re-established at every step instead of resting on a continuous contact patch. It is the problem that made sim-to-real reinforcement learning a shipping technology instead of a demo. Conventional controllers built it from elaborate state machines that explicitly trigger motion primitives and reflexes, a design that grew more complex without approaching the generality of animal locomotion. The ETH Zurich line replaced that stack outright: Lee and colleagues trained an ANYmal controller by reinforcement learning in simulation and generalized it zero-shot to alpine terrain on proprioception alone, and Miki and colleagues extended the same recipe to perceptive locomotion in the wild, the template humanoid programs have since been rerunning on two legs. is the one domain where sim-to-real reinforcement learning is not a promise or a demo but the shipping default, and the chain of results that made it so is unusually well documented. Between 2019 and 2022, the ETH Zurich line around ANYmal took a quadruped from fragile lab behavior to blind alpine hiking, and every humanoid program since has tried to rerun the same recipe on two legs.

Underneath all of it sits an older idea that learning did not replace: the gait. Switch the diagram below from walk to trot to bound and watch the support readout, three feet always down, then two, then a flight phase with none. That progression is the stability story of legged locomotion in miniature.

The quadruped lineage

Hwangbo and colleagues learned an actuator network for ANYmal's series-elastic actuators. It predicts joint torque from current and past joint-position errors and velocities, using supervised data collected by exciting the real robot's legs and manually disturbing it. The network models actuation inside the simulator; the control policies are trained there and then deployed on the robot. Policies trained with ideal or analytical actuator models could not take a single step without falling in the reported ablations. A separately trained recovery policy flipped ANYmal upright from nine tested configurations, including upside-down and folded-leg poses. The authors report relaxing joint-velocity constraints after the first successful hardware recovery attempt Hwangbo 2019↓Learning agile and dynamic motor skills for legged robotsJemin Hwangbo, Joonho Lee, Alexey Dosovitskiy et al., Science Robotics 4(26), 2019.

Lee and colleagues used a privileged RL teacher to supervise both the actions and latent features of a temporal convolutional network student. Its deployed default uses two seconds of proprioceptiveproprioceptionSensing of the body's own state: joint positions and velocities, orientation, and contact, as opposed to exteroception, which senses the outside world through cameras or lidar. Proprioception is fast, cheap, and never occluded, and the ETH Zurich line showed how far it goes: Lee and colleagues trained an ANYmal controller that hiked mud, snow, rubble, and vegetation on proprioception alone, and Miki and colleagues kept it as the trusted channel the policy falls back on when its terrain map disagrees with its body. history from joint encoders and an IMU, alongside the command and current state. Trained on rigid, procedurally generated terrain, controllers for ANYmal-B and ANYmal-C traversed mud, snow, rubble, and vegetation outside the training conditions, without environment-specific tuning for robots of the same generation. The student has no external terrain map: it infers terrain properties from bodily feedback. This blindness still limits foresight: Lee and colleagues explicitly warn that a robot commanded off a cliff will walk off it Lee 2020↓Learning Quadrupedal Locomotion over Challenging TerrainJoonho Lee, Jemin Hwangbo, Lorenz Wellhausen et al., Science Robotics 5(47), 2020. The sim-to-real module covers the distillation mechanics.

Miki and colleagues combined height samples from an elevation map with proprioception in a GRU-based recurrent belief encoder. A learned attention gate controls how much exteroceptive information enters the belief state; the student is trained to imitate a privileged teacher and reconstruct terrain information from noisy observations. In covered-sensor stair and slippery-platform tests, the controller fell back on proprioception, revising its terrain estimate after contact rather than anticipating every obstacle. On ANYmal C, deployment used either two Robosense Bpearl LiDAR sensors or four Intel RealSense D435 cameras, with elevation mapping at 20 Hz and the policy at 50 Hz. Their Etzel mountain hike covered 2.2 km with 120 m of elevation gain in 78 minutes, compared with a hiking planner's 76 minutes; it had no falls but included stops to reattach a shoe and swap batteries. The paper's “hour-long” description is not an exact one-hour traversal. Its limitations include possible falls at occluded cliffs or stepping stones, so the result does not establish all-terrain robustness Miki 2022↓Learning robust perceptive locomotion for quadrupedal robots in the wildTakahiro Miki, Joonho Lee, Jemin Hwangbo et al., Science Robotics 7(62), 2022.

Rudin and colleagues report flat-terrain walking policies trained in under four minutes on a single workstation GPU. Separately, their documented simulation-and-deployment policy used 4,096 parallel ANYmal robots, batches of 98,304 RL transitions (24 steps per robot), and 1,500 policy updates in under twenty minutes on an i9-11900k CPU and NVIDIA RTX A6000 GPU. Their game-inspired terrain curriculumcurriculum learningTraining on a scheduled sequence of tasks that grow harder as the agent improves, instead of sampling the full difficulty range from the start. Rudin and colleagues raised or lowered terrain levels based on each robot's progress. Their documented curriculum-trained simulation-and-deployment policy used 4,096 robots, 98,304 transitions per batch and 1,500 updates in under twenty minutes on an i9-11900k CPU and RTX A6000 GPU; their separate flat-terrain headline is under four minutes on one workstation GPU. promotes robots that cross a tile boundary, demotes those that cover less than half the commanded distance, and returns robots that solve the hardest level to a random level Rudin 2021↓Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement LearningNikita Rudin, David Hoeller, Philipp Reist et al., CoRL 2021. The paper links its released training code, legged_gym; the pinned October 2021 repository identifies itself as the Isaac Gym environment used to train ANYmal on rough terrain Rudin 2021↓Isaac Gym Environments for Legged RobotsNikita Rudin, 2021.

Photo: ANYbotics / Wikimedia Commons. Licence: CC BY-SA 4.0.

Choi and colleagues extended the recipe to ground that moves. Deformable terrain like sand defeats rigid-contact simulators, so they put a computationally cheap granular-media model inside the training loop and paired it with a controller that identifies terrain properties from feel as the robot walks. Their Raibo quadruped ran on soft beach sand at 3.03 m/s with its feet fully buried during stance Choi 2023↓Learning Quadrupedal Locomotion on Deformable TerrainSuyoung Choi, Gwanghyeon Ji, Jeongsoo Park et al., Science Robotics 8(74), 2023.

A gait is a timing pattern

A quadruped's gait is a footfall-timing pattern, described by two numbers per leg: the phase offset (when in the stride cycle the foot strikes) and the duty factor $\beta$, the fraction of the cycle the foot spends on the ground. The diagram at the top of this module shows the four canonical patterns.

The walk spreads footfalls evenly and keeps three feet down at all times, so a static support polygon always exists. The trot, the working gait of most quadruped robots including ANYmal, strikes diagonal pairs together and balances dynamically on two feet. The bound alternates the front pair against the hind pair with a suspension interval in between. In the MIT Cheetah 2 control design, Park, Wensing and Kim plan stance time from stride length and desired speed, $T_{st} = L/v_d$, keep swing duration fixed, and modulate the duty cycle via vertical impulse scaling (the paper’s Sec. 4.3 is literally titled “Duty cycle modulation via vertical impulse scaling”). Experimentally the stride length grows with speed only up to 3 m/s and is fixed above it, so the stance-time scaling is one classical controller’s schedule rather than a universal timing law, and the abstract’s 6.4 m/s bounding result with cost of transport 0.47 is qualified by side-wall contact and roll instability Park 2017↓High-Speed Bounding with the MIT Cheetah 2: Control Design and ExperimentsHae-Won Park, Patrick M. Wensing, Sangbae Kim, International Journal of Robotics Research 36(2), 2017. The pronk throws all four feet down and up together and spends most of the cycle airborne.

Learned policies are not handed these patterns. Foot-air-time and footfall-style reward terms induce them, and the policy discovers whichever pattern the reward makes cheapest. The duty factors shown here are canonical nominal values; real controllers, classical and learned alike, modulate duty factor continuously with speed Park 2017↓High-Speed Bounding with the MIT Cheetah 2: Control Design and ExperimentsHae-Won Park, Patrick M. Wensing, Sangbae Kim, International Journal of Robotics Research 36(2), 2017.

From quadrupeds to humanoids

On Unitree's H1, H2O turns human pose estimates from an RGB camera into real-time whole-body teleoperationteleoperationDirect remote control of a robot by a human operator, used in robot learning mostly to record expert demonstrations. The ALOHA rig made the setup cheap and precise: the operator back-drives a leader arm that is a joint-for-joint twin of the follower the policy will run on, so the recorded motion is already in the robot's own action space. Mobile ALOHA extended the same leader-follower scheme to a mobile base for whole-body tasks.. It first fits and retargetsretargetingMapping motion recorded on one body, usually a human's, onto a robot with different proportions and joint limits, so human demonstrations become references the robot can track. Raw retargeting ignores the robot's physical constraints and can produce references that are morphologically infeasible, which is why systems such as H2O and ASAP pair retargeted human motion with reinforcement learning that restores balance and contact feasibility on the real body. AMASS human motions, then uses a privileged imitator to filter motions it cannot track. A separate tracking policy learns from the cleaned set with domain randomization and outputs 19 joint-position targets to a PD controller. Naive retargeting can produce infeasible references; it does not invariably destroy feasibility. The demonstrated system also uses robot-side motion capture for linear velocity, so "one RGB camera" describes the human-motion input, not the complete sensing setup He 2024↓Learning Human-to-Humanoid Real-Time Whole-Body TeleoperationTairan He, Zhengyi Luo, Wenli Xiao et al., IROS 2024.

On the G1 with fixed wrists, ASAP uses real rollouts to train an ankle delta-action model, freezes it in simulation for policy fine-tuning, and deploys the fine-tuned policy without the residual model. The paper illustrates jumps and Ronaldo-, LeBron- and Kobe-inspired motions. Its quantitative hardware comparison is kicking and LeBron's "Silencer" against Vanilla, not a success-rate study of every illustrated skill He 2025↓ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body SkillsTairan He, Jiawei Gao, Wenli Xiao et al., RSS 2025.

MIT's humanoid program attacked a different piece of the problem.

Jeon and colleagues compared reward formulations, not RL algorithms, for running with a simulated MIT Humanoid: an 18-DoF platform with its arms fixed and ten leg joints controlled. They held PPO-Clip and the IsaacGym training setup fixed while comparing baseline rewards alone, baseline plus direct shaping, and baseline plus potential-based shaping of orientation, height, and joint regularization. After tuning the baseline weights and keeping them fixed, they compared accumulated baseline reward rather than the differently scaled total rewards. Ten runs per nominal case converged to similarly performing behaviors within 1,000 policy iterations; the potential-based terms offered only marginally faster convergence but were less sensitive in a sweep from 0.1 to 10 times the nominal shaping weights. The practical variant set the potential-shaping discount to 1 while retaining PPO's discount of 0.99, which the authors explicitly say sacrifices policy invariance. This is evidence for easier reward tuning in that experiment, not proof that algorithm choice is irrelevant or that the learned controller is optimal or safe Jeon 2023↓Benchmarking Potential Based Rewards for Learning Humanoid LocomotionSe Hwan Jeon, Steve Heim, Charles Khazoom et al., ICRA 2023.

The full human-motion-tracking lineage, from PHC through GMT and the 2026 robustness results, gets its own module next.

Industrial adoption

Boston Dynamics describes a specific production lineage for Spot. Its earlier locomotion controller evaluated many MPCmodel predictive controlControl by constant re-planning: at each control step, optimize a short sequence of future actions against an explicit dynamics model, execute only the first action, and solve again from the freshly measured state. Because the plan is recomputed online, model error is rejected by feedback at every step instead of being frozen into a policy's weights, so a model-predictive controller shows less of the frozen-policy sim-to-real failure mode, though a wrong dynamics model still costs it closed-loop performance. Long assumed too slow for a robot's full dynamics, the method reached whole-body scale in 2026, when Zhang and colleagues solved iLQR against MuJoCo's physics fast enough to close the loop in real time on real hardware. controllers simultaneously: in less than a millisecond, dozens of predictive horizons with distinct step-trajectory references were evaluated, then a scoring system selected the highest-valued controller output Dynamics 2024↓Starting on the Right Foot with Reinforcement LearningBoston Dynamics, 2024.

In its 2024 account, Boston Dynamics describes adding a learned policy while retaining the existing model-based locomotion controller for the part it handles well. This removed the need to run multiple MPC instances in parallel, reducing computational complexity. The company reports that Spot became less likely to fall, including on extremely slick or irregular surfaces, and says existing customers receive the policy through a software update; it does not supply a numerical fall-rate reduction. The same article separately describes research into other architectures: its example combining a learned policy with a model-based controller not limited by periodic-gait assumptions climbs boxes over 70 cm high. That example is not stated as a capability of the shipped production policy Dynamics 2024↓Starting on the Right Foot with Reinforcement LearningBoston Dynamics, 2024.

The reported evaluation process first benchmarks policies in simulation, then tests them on an internal Spot robustness fleet operating 24/7 for a cumulative runtime of over 2,000 hours a week across indoor and outdoor conditions, stairs, floor materials, and simulated weather. This is a fleet total, not a per-robot runtime. Falls and other mobility issues reproducible in a physics simulation are recreated for either the training or evaluation set; the article describes thousands of simulated and on-robot hours before customer deployment Dynamics 2024↓Starting on the Right Foot with Reinforcement LearningBoston Dynamics, 2024.

On March 19, 2025, the RAI Institute described its Boston Dynamics collaboration on Atlas: a control policy tracks and controls retargeted human motion, with each maneuver created from data from about 150 million simulator runs and transferred zero-shot to hardware. The readable landing page uses the unit “runs”; it does not define their duration or specify calibration, system identification, or intervention procedures. Its zero-shot report is not evidence of calibration-free deployment Institute 2025↓Reinforcement Learning Accelerates Humanoid Behavior ProductionRobotics and AI Institute, 2025.

In August 2025, Boston Dynamics and Toyota Research Institute reported Large Behavior Models on Atlas for long-horizon manipulation involving locomotion: a 450M-parameter diffusion transformer trained with a flow-matchingflow matchingA generative modeling recipe that learns a time-dependent vector field carrying samples from a noise distribution to the data distribution, trained by regressing the model's field against the conditional flow rather than by estimating a score. pi0 brought it to robot control: a dedicated action expert is trained with the flow-matching objective to denoise continuous action chunks, and at inference the learned field is integrated as an ODE for a handful of steps, which keeps the policy fast enough for high-frequency control while keeping the multimodality that made diffusion policies attractive. objective and conditioned on proprioception, images, and a language prompt. Training draws on curated teleoperated demonstrations from hardware and simulation. The article describes actions controlling the full robot at 30 Hz, but not a network inference on every control tick: images arrive at 30 Hz, the network predicts 48 actions (1.6 seconds), and generally 24 actions (0.8 seconds at 1x speed) are executed per inference. Its teleoperation system is built on MPC, and the neural policies use the same robot control interface. These are reported research policies deployed on hardware, not an announcement that every Atlas behavior or the complete controls stack has been replaced Cousineau 2025↓Large Behavior Models and Atlas Find New FootingEric Cousineau, Scott Kuindersma, Lucas Manuelli et al., 2025.

What the lineage does not solve

Hold onto three limits. First, these controllers walk, and walking is not fetching: nothing in the legged success story transfers directly to contact-rich manipulation, where the first module in this domain showed the MDP itself resists simulation. Second, motion tracking inherits the human demonstrator's envelope, and retargeted references can be morphologically infeasible for the robot, which is the explicit motivation of the 2026 robustness work the humanoid module covers. Third, the lineage's center of gravity turned out to be reward design: the gait a policy learns is whichever pattern the reward makes cheapest, and getting that objective right is still hand labor, now with LLM assistants.