Why RL Won Locomotion but Not Manipulation

In 2021, Rudin and colleagues trained an ANYmal quadruped to walk on flat ground in under four minutes, and on uneven terrain in twenty minutes, on a single workstation GPU Rudin 2021↓Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement LearningNikita Rudin, David Hoeller, Philipp Reist et al., CoRL 2021. Policies trained this way are now the default production approach for quadruped locomotion and, increasingly, for bipedal humanoids. General manipulation went the other direction: the field's strongest systems are trained by imitation on real, teleoperated demonstrations. In 2026 there is still no manipulation equivalent of the four-minute walking run.

The reason sits in the contact, and it is measurable. Drag the contact-model error slider below from its default 2 mm: the peg insertion is already jammed at that value while the quadruped notices nothing, and locomotion only fails past 20 mm, forty times the error that kills the insertion.

The asymmetry is in the MDP, not the algorithm

The tempting explanation is that RL is somehow bad at manipulation. The reality-gap survey, the field's authoritative 2026 statement on sim-to-real, points somewhere else: the manipulation MDP is not cheaply simulatable Aljalbout 2025↓The Reality Gap in Robotics: Challenges, Solutions, and Best PracticesElie Aljalbout, Jiaxu Xing, Angel Romero et al., Annual Review of Control, Robotics, and Autonomous Systems 2026 (accepted), 2025. The asymmetry is in the problem, not in the machinery, and the RL for Robotics module covers that machinery side: what a sample budget buys, why robotics defaulted to PPO anyway, and where off-policy methods pay off. Locomotion asks the simulator for proprioceptionproprioceptionSensing of the body's own state: joint positions and velocities, orientation, and contact, as opposed to exteroception, which senses the outside world through cameras or lidar. Proprioception is fast, cheap, and never occluded, and the ETH Zurich line showed how far it goes: Lee and colleagues trained an ANYmal controller that hiked mud, snow, rubble, and vegetation on proprioception alone, and Miki and colleagues kept it as the trusted channel the policy falls back on when its terrain map disagrees with its body., a terrain heightfield, and a decent actuator model, all of which physics engines handle well. Manipulation asks for per-object geometry, friction, compliance, and deformation, resolved to millimeter-accurate multi-point contact, plus a visual pipeline good enough to stand in for object state. Each of those is a known hard problem; a single task needs all of them at once.

This one fact explains why the two literatures read so differently. The locomotion literature is "PPOPPOProximal policy optimization, the on-policy reinforcement learning algorithm the locomotion literature standardized on. Schulman and colleagues introduced it as a policy gradient method that alternates between sampling data through interaction with the environment and optimizing a surrogate objective with stochastic gradient ascent, where the surrogate is what allows several epochs of minibatch updates on each batch of samples instead of the single gradient step standard policy gradient takes. It keeps most of trust region policy optimization's reliability without the second-order machinery, and it pairs naturally with massively parallel simulation, where thousands of robots supply the enormous on-policy batches it consumes., domain randomizationdomain randomizationTraining a policy across a distribution of simulators with randomized parameters, visual appearance, dynamics, or both, so the real world reads as one more sample from that distribution. Tobin et al. randomized rendering and transferred an object detector to a real arm; Peng et al. randomized dynamics and transferred a pushing policy with no real-world training. The price is conservatism: a policy that does well across a wide distribution is optimal for no single instance of it., curriculumcurriculum learningTraining on a scheduled sequence of tasks that grow harder as the agent improves, instead of sampling the full difficulty range from the start. Rudin and colleagues raised or lowered terrain levels based on each robot's progress. Their documented curriculum-trained simulation-and-deployment policy used 4,096 robots, 98,304 transitions per batch and 1,500 updates in under twenty minutes on an i9-11900k CPU and RTX A6000 GPU; their separate flat-terrain headline is under four minutes on one workstation GPU.." The manipulation literature is "collect a hundred thousand demonstrations and train a diffusion transformer," because real demonstration data sidesteps the simulability problem entirely: a real trajectory needs no contact model at all (the behavior cloning module covers what that choice costs elsewhere).

Six properties of the two MDPs

The asymmetry decomposes into six specific properties. Each row is a place where the locomotion MDP is cheap to simulate faithfully and the manipulation MDP is not.

The Isaac Lab v1 paper says dexterous manipulation with multi-fingered hands "remains challenging compared to standard parallel-jaw grasping due to the high-dimensional action space and fine-grained control required". Its discussion names accurate contact modeling, vision and proprioception, domain randomization, and tiled rendering as support for this work NVIDIA 2025↓Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot LearningNVIDIA, Mayank Mittal, Yunrong Guo et al., 2025. The humanoid sim-to-real RL literature is blunter. Lin and colleagues note that previous successes in dexterous manipulation involve "much more laborious real-to-sim engineering efforts" that are task-specific or hardware-specific, unlike the locomotion successes Lin 2025↓Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on HumanoidsToru Lin, Kartik Sachdev, Linxi Fan et al., CoRL 2025.

Contact geometry is where the gap lives

Row two of the table deserves its own look, because contact is the property that breaks simulators first. The figure at the top of this module injects a contact-model error, the simulator's idea of where a surface is being off by epsilon, into two scenes: a quadruped stance and a peg insertion.

Three things to take from it. First, contact count: four near-point foot contacts versus fourteen simultaneous distributed contacts, each one a place where a solver approximation becomes a wrong force. Second, patch character: a foot presses a near-point patch into a rigid substrate, while a peg slides through continuous wall contact that simple point-contact models do not represent. Third, irreversibility: the quadruped's gait is an attractor that high-bandwidth feedback pulls it back into, so a wrong contact force is corrected on the next step. A jammed peg has no next step.

It is a tendency, not a law

Sim-to-real RL has produced real dexterous results, and honesty requires listing them. In OpenAI’s 2019 paper, a policy trained in simulation with automatic domain randomization controlled a modified five-fingered Shadow Dexterous Hand. For one fixed fair-scramble sequence, its best policy completed all 43 subgoals in two of ten real trials using vision for cube pose and the sensor-equipped Giiker cube for face angles. The condition using vision for face angles completed the full sequence in zero of ten trials. These tests started from a solved cube and applied a fixed scramble, rather than sampling every possible scramble; a separate solver supplied the subgoal sequence OpenAI 2019↓Solving Rubik's Cube with a Robot HandOpenAI, Ilge Akkaya, Marcin Andrychowicz et al., 2019.

Isaac Lab v1 describes DextrAH-RGB's privileged-state RL teacher distilled into a stereo-RGB student for a KUKA arm with an Allegro hand NVIDIA 2025↓Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot LearningNVIDIA, Mayank Mittal, Yunrong Guo et al., 2025. See the sim-to-real explanation.

Play2Perfect trains CAD-derived sparse-reward assembly tasks and transfers zero-shot, including 60% success on insertions with 0.5 mm clearance Lum 2026↓Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?Tyler Ga Wei Lum, Kushal Kedia, C. Karen Liu et al., 2026. What these results share is per-task engineering. The Rubik’s-cube system used customized hand hardware, manually tuned motor-torque limits, and a hand simulation calibrated against recorded physical joint trajectories. ADR started from a calibrated environment and adjusted randomization ranges automatically using performance thresholds; the ranges were not all hand-tuned OpenAI 2019↓Solving Rubik's Cube with a Robot HandOpenAI, Ilge Akkaya, Marcin Andrychowicz et al., 2019. Play2Perfect's clearance number comes from a pipeline built around its task family.

The honest snapshot comes from the imitation side

The sharpest 2026 evidence that manipulation remains the bottleneck is not an RL result at all. Gemini Robotics 2, the strongest frontier system with published task-level numbers, reports multi-finger household tasks spanning 32% (dustpan) to 92% (unscrewing a bulb), 89.6% on gripper-based precise insertion, and whole-body pick success of 68.4% from a table, 45.7% from the floor, and 76.3% from a shelf Parada 2026↓Gemini Robotics 2 brings whole body intelligence to robotsCarolina Parada, 2026. A 32% task on the frontier's best system is what an unsolved MDP looks like.