Sim-to-Real Transfer

The previous module ended with a policy that walks after minutes of GPU training. That policy is useless as trained: it has only ever seen one simulator, with one set of contact parameters, one actuator model, one rendering stack. The real robot differs from all of them in ways nobody can fully enumerate. Closing that difference is sim-to-real transfer, and it is where most of the engineering lives. The RL algorithm is usually the least interesting part of a successful sim-to-real system.

Four method families do the work. Domain randomizationdomain randomizationTraining a policy across a distribution of simulators with randomized parameters, visual appearance, dynamics, or both, so the real world reads as one more sample from that distribution. Tobin et al. randomized rendering and transferred an object detector to a real arm; Peng et al. randomized dynamics and transferred a pushing policy with no real-world training. The price is conservatism: a policy that does well across a wide distribution is optimal for no single instance of it. trains over a distribution of simulators so the real world reads as one more sample. Teacher-student distillationteacher-student distillationTraining two policies in sequence to work around partial observability: a teacher trains with privileged simulator state, such as exact terrain friction or object pose, and a student then learns to imitate the teacher using only the observations available at deployment. RMA instead trained a base policy with a privileged encoder, then trained a separate state-action-history module to predict its latent extrinsics for asynchronous online inference. Lee and colleagues distilled a privileged ANYmal teacher into a proprioceptive student that hikes challenging terrain without ever seeing it. trains on privileged simulator state, then compresses that knowledge into a policy that sees only what the robot sees. System identificationsystem identificationMeasuring a real robot's dynamics and correcting the simulator's parameters to match, so the sim-to-real gap shrinks by calibration rather than by randomization. Hwangbo and colleagues replaced a miscalibrated analytic actuator model with a learned network mapping joint-command history to realized torque after identifying actuator error as the dominant transfer obstacle on ANYmal. The method attacks the dynamics component of the gap directly, where domain randomization only averages over it. measures the dynamics gap and corrects the model directly. Real-to-sim reconstruction rebuilds the deployment scene inside the simulator so the visual gap shrinks to rendering error. The 2026 reality-gap survey organizes the field into nearly these same buckets, adding sim-real co-training and state/action abstraction as cross-cutting levers Aljalbout 2025↓The Reality Gap in Robotics: Challenges, Solutions, and Best PracticesElie Aljalbout, Jiaxu Xing, Angel Romero et al., Annual Review of Control, Robotics, and Autonomous Systems 2026 (accepted), 2025.

The figure below is a deterministic toy, not a learned policy or a measured robot benchmark. Drag the selected friction to compare the two authored curves. Widening the half-width lowers the plateau because that relationship is built into this model, not because the cited papers establish a universal tradeoff.

Domain randomization: train on a distribution of simulators

These two papers address different parts of transfer. Tobin and colleagues randomized textures, lighting and camera pose for tabletop object localization. They summarize real-world localization accuracy as around 1.5 cm, evaluated on 480 webcam images of eight geometric objects in object-only, distractor and partial-occlusion conditions. The paper reports the best network after a small hyperparameter search; the approximate 1.5 cm summary is not a bound for every object or condition. Most experiments used ImageNet initialization, but an ablation found that random initialization could achieve nearly the same transfer performance with enough simulated training data. The detector was also used with a motion planner on a Fetch robot Tobin 2017↓Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real WorldJosh Tobin, Rachel Fong, Alex Ray et al., IROS 2017. Peng and colleagues randomized link mass, joint damping, puck mass/friction/damping, table height, controller gains, action timing and observation noise. A recurrent policy trained only in simulation transferred to a seven-DoF Fetch arm pushing a puck to a target. Real evaluation used motion capture for puck position, 200 control steps per episode, and success within 7 cm of the target at the end of the episode; target and initial puck positions were randomized within a 30 cm by 30 cm area. The authors report substantial simulated-versus-real joint-trajectory mismatch after limited calibration, not an absence of calibration Peng 2018↓Sim-to-Real Transfer of Robotic Control with Dynamics RandomizationXue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba et al., ICRA 2018. With enough variability in the simulator, the real world may appear to the model as just another variation, as Tobin's abstract puts it.

OpenAI's Rubik's-cube system added the curriculumcurriculum learningTraining on a scheduled sequence of tasks that grow harder as the agent improves, instead of sampling the full difficulty range from the start. Rudin and colleagues raised or lowered terrain levels based on each robot's progress. Their documented curriculum-trained simulation-and-deployment policy used 4,096 robots, 98,304 transitions per batch and 1,500 updates in under twenty minutes on an i9-11900k CPU and RTX A6000 GPU; their separate flat-terrain headline is under four minutes on one workstation GPU. version: automatic domain randomization widens the randomization range as the agent succeeds, so the distribution grows with the policy's competence OpenAI 2019↓Solving Rubik's Cube with a Robot HandOpenAI, Ilge Akkaya, Marcin Andrychowicz et al., 2019. Isaac Lab ships ADR as a configurable curriculum with reference configs in its dexterous-manipulation examples NVIDIA 2025↓Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot LearningNVIDIA, Mayank Mittal, Yunrong Guo et al., 2025.

Two details rarely make it into the textbook version. First, the plumbing: in the 2025 Isaac Lab v1 paper, simulation state is GPU-resident but mass, friction, contact offsets, and joint armature are modified through PhysX CPU APIs. Most other physics parameters can be randomized at runtime, while mesh scale and collider type can change only before simulation starts. CPU parameter writes do not mean randomization is restricted to episode resets
Source: NVIDIA 2025↓Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot LearningNVIDIA, Mayank Mittal, Yunrong Guo et al., 2025. Second, the objective: Peng and colleagues maximize expected return over a distribution of dynamics models and evaluate randomization ablations on their Fetch pushing task Peng 2018↓Sim-to-Real Transfer of Robotic Control with Dynamics RandomizationXue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba et al., ICRA 2018. The friction chart below instead imposes its peak-versus-width relation by construction; it is not a measured performance frontier.

The figure draws a point-trained-policy toy curve and a distribution-trained-policy toy curve. No policy is trained here. The shaded interval marks the assumed training range, not uncertainty or a confidence interval.

Teacher-student distillation: privilege in, proprioception out

Domain randomization still asks the policy to act on what the robot can sense. The teacher-student pattern separates those problems. Phase one trains a teacher with privileged, simulator-only observations: terrain geometry under the feet, friction coefficients, applied disturbances. Phase two trains a student that sees only deployable sensing, usually a short history of proprioceptionproprioceptionSensing of the body's own state: joint positions and velocities, orientation, and contact, as opposed to exteroception, which senses the outside world through cameras or lidar. Proprioception is fast, cheap, and never occluded, and the ETH Zurich line showed how far it goes: Lee and colleagues trained an ANYmal controller that hiked mud, snow, rubble, and vegetation on proprioception alone, and Miki and colleagues kept it as the trusted channel the policy falls back on when its terrain map disagrees with its body., to imitate the teacher's actions.

Lee and colleagues trained a privileged RL teacher with simulator-only terrain and contact information, then distilled its actions and latent features into a temporal convolutional network student. The deployed student uses two seconds of proprioceptive history from joint encoders and an IMU, together with the command and current state; it does not consume an external terrain map. Training used rigid, procedurally generated terrain, while ANYmal-B and ANYmal-C deployments included mud, snow, rubble, and vegetation. Robots of the same generation used the same controller across those environments without environment-specific tuning Lee 2020↓Learning Quadrupedal Locomotion over Challenging TerrainJoonho Lee, Jemin Hwangbo, Lorenz Wellhausen et al., Science Robotics 5(47), 2020.

RMA uses a different two-stage construction: a base policy and an environment-factor encoder are trained jointly with RL, then an adaptation module is trained by supervised regression on simulator state-action histories to predict the latent extrinsics. At deployment the base policy combines the current state, previous action and estimated extrinsics to predict desired joint positions. The adaptation module uses 50 state-action steps (0.5 seconds of history); on the tested Unitree A1 setup it updates extrinsics asynchronously at about 10 Hz, while the base policy runs at 100 Hz using the latest estimate. These are inference processes, not online gradient updates. The paper reports adaptation in fractions of a second and deployment without real-world fine-tuning, but also reports failures after large perturbations or multiple leg obstructions. The latent is a behavior-relevant encoding, not a guarantee of recovering each physical parameter Kumar 2021↓RMA: Rapid Motor Adaptation for Legged RobotsAshish Kumar, Zipeng Fu, Deepak Pathak et al., RSS 2021.

In its DextrAH-RGB example, the Isaac Lab v1 paper reports an RL teacher trained with privileged state on a KUKA arm and Allegro hand, then distilled into a network taking stereo RGB pairs as input
Source: NVIDIA 2025↓Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot LearningNVIDIA, Mayank Mittal, Yunrong Guo et al., 2025.

The failure mode has a name. The Isaac Lab paper calls it an information gap due to input mismatch: the teacher observes privileged state information while the student sees only partial observations, so the student must reconstruct the unobserved states from images, a challenge that becomes particularly pronounced under high camera occlusion NVIDIA 2025↓Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot LearningNVIDIA, Mayank Mittal, Yunrong Guo et al., 2025.

The panel below is an authored toy, not measured robot data. It uses chosen terrain and noise, not a trained teacher or student. Its 24 terrain cells combine two sine waves with a step and a depression. Two seeded sequences (123456789 and 987654321) fix the noise and channel-dropout thresholds. A Gaussian blur of width three cells, a noise coefficient of 0.06, and a same-sign occlusion error of 0.12 define the reconstruction error; the degradation control scales that error from zero to one and starts at 0.15. The reconstruction is calculated directly from this terrain and error field, not inferred from the displayed input strip.

Push the degradation slider up. The input strip loses channels and gains noise, the constructed reconstruction changes, and both error readouts rise by design. The action-divergence value is 2.2 times the unrounded reconstruction MAE, displayed in normalized units; it is not a comparison of learned actions or a measured cost of distillation. At zero degradation the local reconstruction is exact. Reset restores 0.15. The cited input-mismatch discussion motivates the illustration, but does not establish these coefficients or a universal error floor independent of training.

System identification: measure the gap instead

Randomization averages over model error. System identification removes the error by measurement. Hwangbo and colleagues learned an actuator network for ANYmal's series-elastic actuators: it predicts joint torque from joint-position errors and velocities at the current time and 0.01 and 0.02 seconds earlier. They fitted it to real-robot excitation data, then used it inside the simulator to train control policies. In their locomotion ablations, policies trained with ideal or analytical actuator models could not take a single step without falling Hwangbo 2019↓Learning agile and dynamic motor skills for legged robotsJemin Hwangbo, Joonho Lee, Alexey Dosovitskiy et al., Science Robotics 4(26), 2019.

ASAP starts with phase-conditioned tracking policies trained in IsaacGym on retargetedretargetingMapping motion recorded on one body, usually a human's, onto a robot with different proportions and joint limits, so human demonstrations become references the robot can track. Raw retargeting ignores the robot's physical constraints and can produce references that are morphologically infeasible, which is why systems such as H2O and ASAP pair retargeted human motion with reinforcement learning that restores balance and contact feasibility on the real body. human motions. It runs the pretrained policies on a Unitree G1 and records trajectories using motion capture and onboard sensors. A second PPO policy takes the simulated state and a recorded action and outputs a delta action. Adding that delta before the simulator step lets its reward penalize disagreement with the recorded real trajectory; the delta is learned through RL, not supplied as a measured residual-action label. The delta action model is then frozen while the tracking policy is fine-tuned in the modified simulator. Only the fine-tuned tracking policy is deployed: the delta model stays in simulation. The hardware study learns corrections for four ankle DoFs, not the full 23-DoF simulation model He 2025↓ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body SkillsTairan He, Jiawei Gao, Wenli Xiao et al., RSS 2025.

The evaluation separates IsaacGym-to-IsaacSim and IsaacGym-to-Genesis transfer from the G1 hardware study. Table III tests open-loop replay; Table IV evaluates closed-loop motion tracking. The simulation suite has 43 motion sequences grouped by difficulty, and reports mean tracking errors across motion sequences. SysID and delta-dynamics are explicit simulation baselines, but ASAP does not win every metric: in Table IV's easy IsaacSim group, its mean global joint-position error is 106 mm versus SysID's 105 mm. On hardware, Table V compares ASAP only with Vanilla on kicking and LeBron James's "Silencer": mean global joint-position error falls from 61.2 to 50.2 mm and from 159 to 112 mm, respectively. Basic domain randomization is used in pretraining; the separate random-action-noise fine-tuning ablation is an IsaacGym-to-Genesis test, not a hardware comparison against every baseline He 2025↓ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body SkillsTairan He, Jiawei Gao, Wenli Xiao et al., RSS 2025.

Generalization is measured, not guaranteed. ASAP reports improved tracking on the held-out "Silencer" motion and better out-of-distribution replay with more training data; neither result establishes arbitrary unseen-skill coverage. Its stated hardware limits are motor overheating and damage, motion-capture dependence during data collection, and the data required to learn a full 23-DoF delta model. These are empirical limitations, not a mathematical rule that the correction is valid only near the pretrained policy's trajectories He 2025↓ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body SkillsTairan He, Jiawei Gao, Wenli Xiao et al., RSS 2025.

Real-to-sim: rebuild the scene, keep the physics

The newest family attacks the visual gap by reconstructing the actual deployment scene. SplatSim replaces mesh primitives in the rendering pipeline with Gaussian splats reconstructed from the deployment scene; PyBullet supplies the physics. Across four UR5 manipulation tasks, with 40 trials per task, the authors report 86.25% average zero-shot real-world success for diffusion policies trained on simulated demonstrations, versus 97.5% for policies trained on real-world demonstrations. This is the result with training augmentations, not rendering alone Qureshi 2024↓SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian SplattingMohammad Nomaan Qureshi, Sparsh Garg, Francisco Yandun et al., 2024.

The setup uses a Robotiq 2F-85 gripper and two RealSense D455 cameras. Preparing the splats requires manual robot segmentation, CAD-derived link bounds and ICP alignment; robot kinematics and simulated object poses drive rendering. Figure 2 lists RGB observations plus end-effector position and orientation, while Section IV-A says the policy relies solely on RGB at test time. Those descriptions disagree; they do not establish an unqualified image-only input specification Qureshi 2024↓SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian SplattingMohammad Nomaan Qureshi, Sparsh Garg, Francisco Yandun et al., 2024. In its August 2025 v2, RoboGSim combines a Gaussian Reconstructor, Digital Twins Builder, Scene Composer and Interactive Engine in a real2sim2real system. Multi-view images and supplied robot MDH parameters feed reconstruction; mesh assets and measured layout alignment connect the scene to Isaac Sim. The synthesizer composes novel views, objects, scenes and trajectories. In closed-loop evaluation, a policy acts on splat-rendered images; Isaac Sim handles inverse kinematics, collisions and other physical interactions, and the resulting state drives the next rendering Li 2024↓RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting SimulatorXinhai Li, Jialin Li, Ziheng Zhang et al., 2024.

This evaluation is not interchangeable with real-robot testing. On the UR5 ring-toss task, each model had ten trials with up to three grasp attempts per trial. Table 2 reports 90% placement for the real-data-trained model on the real robot but 30% in RoboGSim. Simulated evaluation avoids executing those actions on hardware; it is not a demonstrated safety guarantee for a deployed robot. The paper examines novel-pose rendering and trajectory replay separately from closed-loop policy evaluation Li 2024↓RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting SimulatorXinhai Li, Jialin Li, Ziheng Zhang et al., 2024. The Newton 1.0 release post describes a Warp-based tiled camera sensor whose ray-tracing backend supports both triangle meshes and Gaussian splats Reist 2026↓Newton Adds Contact-Rich Manipulation and Locomotion Capabilities for Industrial RoboticsPhilipp Reist, Miguel Zamora Mora, JC Chang et al., 2026.

Keep the division of labor straight: the splat supplies appearance, and a conventional physics engine still supplies dynamics. A 3DGS twin is a learned renderer bolted onto a simulator, not a learned simulator, and it reconstructs a static scene: articulated objects, deformables, and lighting changes each require extra machinery on top. Confusing the two is the most common misreading of the real-to-sim literature.

The levers nobody demos

The survey adds two cross-cutting options that rarely get headline systems. Sim-real co-training mixes real data into an otherwise simulated training set, so the real samples anchor the representation while simulation supplies coverage. State and action abstraction chooses the interface between policy and robot (end-effectorend effectorThe last link of a robot arm and whatever is attached to it: the gripper, hand, or tool whose pose the arm exists to place. Kinematics is conventionally written as the map from joint angles to the end-effector pose, and most action spaces in learned manipulation are defined as end-effector deltas rather than joint targets, because a task is specified in the space where the hand meets the world. velocities or foot positions rather than raw torques) so that a conventional low-level controller absorbs much of the gap Aljalbout 2025↓The Reality Gap in Robotics: Challenges, Solutions, and Best PracticesElie Aljalbout, Jiaxu Xing, Angel Romero et al., Annual Review of Control, Robotics, and Autonomous Systems 2026 (accepted), 2025. The survey's own emphasis is that the action-space choice plays a crucial role in reducing the sim-to-real gap across navigation, locomotion, and manipulation, which matches the quiet pattern in successful deployments: most of them randomize some things, identify others, and abstract away the rest.

Where each family breaks

Each family's failure mode is worth holding onto. The friction chart's conservatism cost is an authored assumption, not an empirical result of the cited papers. The teacher-student panel likewise constructs its error rather than measuring a distillation limit. ASAP shows some held-out motion transfer, but broader coverage and affordable real-robot data collection remain open He 2025↓ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body SkillsTairan He, Jiawei Gao, Wenli Xiao et al., RSS 2025. Real-to-sim twins freeze the scene they captured.