Humanoid Whole-Body Control

Humanoid whole-body control is the control of every actuated joint of a humanoid, from legs to hands, as one coupled problem. A dominant paradigm treats whole-body controlwhole-body controlWhole-body control is treating all of a robot's actuated degrees of freedom (legs, torso, arms, hands) as one coupled problem instead of an arm controller stacked on a separate locomotion controller. Model-based versions optimize against the full rigid-body dynamics. Zhang et al. ran whole-body MPC in real time on hardware with plain iLQR and MuJoCo, for dynamic quadruped locomotion and full-sized humanoid walking. Learned versions track retargeted human motion, as H2O did for real-time whole-body teleoperation, with reinforcement learning restoring the balance and contact feasibility raw retargeting loses. as reinforcement-learned tracking of human motion retargetedretargetingRetargeting is mapping motion recorded on one body, usually a human's, onto a robot with different proportions and joint limits, so human demonstrations become references the robot can track. Raw retargeting ignores the robot's physical constraints and can produce morphologically infeasible references. H2O and ASAP therefore add reinforcement learning that restores balance and contact feasibility on the real body. to the robot^{Source 14}14 Yang et al. 2025, OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction. Lujie Yang, Xiaoyu Huang, Zhen Wu et al., 2025, though retargeted motion can be physically infeasible^{Source 2}2 He et al. 2024, Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation. Tairan He, Zhengyi Luo, Wenli Xiao et al., IROS 2024.

Photo: Damian B Oh / Wikimedia Commons. Licence: CC BY-SA 4.0.

The tracking lineage

DeepMimic adapted standard RL to imitate motion clips, including retargeted ones, for animated characters and an Atlas robot model^{Source 15}15 Peng et al. 2018, DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. Xue Bin Peng, Pieter Abbeel, Sergey Levine et al., 2018.

Photo: Applearts123 / Wikimedia Commons. Licence: CC BY-SA 4.0.

PHC tracks motion and recovers from falls for simulated avatars, with no physical robot: its controller imitates 98.9% of 11,313 filtered AMASS training clips, and PMCP trains new primitives on progressively harder failed clips^{Source 1}1 Luo et al. 2023, Perpetual Humanoid Control for Real-time Simulated Avatars. Zhengyi Luo, Jinkun Cao, Alexander Winkler et al., ICCV 2023.

On the Unitree H1, H2O's privileged motion-filtering imitator is "similar to PHC" and follows PULSE^{Source 16}16 Luo et al. 2023, Universal Humanoid Motion Representations for Physics-Based Control. Zhengyi Luo, Jinkun Cao, Josh Merel et al., arXiv preprint (ICLR 2024), 2023. A second policy learns whole-body tracking from the cleaned retargeted set with domain randomization and follows human keypoints estimated from an RGB camera, but the deployed teleoperationteleoperationTeleoperation is direct remote control of a robot by a human operator, used in robot learning mostly to record expert demonstrations. In the cheap, precise ALOHA rig, the operator back-drives a leader arm twinning the follower joint for joint, so recordings are in the robot's own action space. Mobile ALOHA extended this leader-follower scheme to a mobile base for whole-body tasks. system, a joint-target policy over a PD controller, also uses robot-side motion capture for linear velocity^{Source 2}2 He et al. 2024, Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation. Tairan He, Zhengyi Luo, Wenli Xiao et al., IROS 2024.

OmniH2O's student, distilled from a privileged RL teacher, tracks head-and-hand goals from proprioceptive history with no explicit global-linear-velocity input, though root odometry still maps goals into the robot frame^{Source 3}3 He et al. 2024, OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning. Tairan He, Zhengyi Luo, Xialin He et al., CoRL 2024. A visual policy trained on teleoperated demonstrations can drive the tracker, evaluated on four of six recorded tasks at ten runs each, and the authors report no safety guarantees for extreme disturbances or out-of-distribution goals^{Source 3}3 He et al. 2024, OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning. Tairan He, Zhengyi Luo, Xialin He et al., CoRL 2024.

HumanPlus estimates the operator's body and hand poses with one RGB camera, and its Humanoid Shadowing Transformer, trained with PPO on retargeted motion in simulation, runs zero-shot on a customized H1 from IMU and joint encoders. Two head-mounted cameras record the robot's view during shadowing, and supervised behavior cloningbehavior cloningBehavior cloning is supervised learning of a policy from expert demonstrations, with observations as inputs and the expert's actions as labels. The recipe predates deep learning: ALVINN (1988) used camera and laser-range inputs and trained on simulated road images before NAVLAB tests. Scaled up, it still underlies most learned manipulation policies; its known closed-loop failure mode is covariate shift. then trains a Humanoid Imitation Transformer from binocular images and proprioception; occlusion, fixed retargeting and limited locomotion remain constraints^{Source 4}4 Fu et al. 2024, HumanPlus: Humanoid Shadowing and Imitation from Humans. Zipeng Fu, Qingqing Zhao, Qi Wu et al., 2024.

ExBody2 tracks local keypoints with separate velocity tracking, because drifting global keypoint targets can accumulate tracking errors. It filters references by an initial policy's lower-body tracking error and distills a privileged PPO teacher into a history-based student that walks, squats and dances on Unitree G1, though transitions between specialists remain unresolved^{Source 5}5 Ji et al. 2024, ExBody2: Advanced Expressive Humanoid Whole-Body Control. Mazeyu Ji, Xuanbin Peng, Fangchen Liu et al., 2024. ASAP, the delta-action method in the sim-to-real module, fine-tunes against a simulator corrected from real G1 rollouts for jumps and sports-style motions^{Source 6}6 He et al. 2025, ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills. Tairan He, Jiawei Gao, Wenli Xiao et al., RSS 2025.

KungfuBot trains one policy per processed reference motion and tightens the tracking reward's error tolerance as training proceeds, never loosening it; its quantitative hardware test is ten Tai Chi trials with the G1 root fixed for evaluation^{Source 7}7 Xie et al. 2025, KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills. Weiji Xie, Jinrui Han, Jiakun Zheng et al., NeurIPS 2025. GMT instead trains a single tracking controller, distilled from a soft mixture-of-experts teacher whose gating network combines expert actions; its baselines and ablations run in simulation, and it neither gets up after falls nor targets slopes and stairs^{Source 8}8 Chen et al. 2025, GMT: General Motion Tracking for Humanoid Whole-Body Control. Zixuan Chen, Mazeyu Ji, Xuxin Cheng et al., 2025. Learned get-up policies lift a real G1 from lying face up or face down^{Source 17}17 He et al. 2025, Learning Getting-Up Policies for Real-World Humanoid Robots. Xialin He, Runpei Dong, Zixuan Chen et al., Robotics: Science and Systems (RSS) 2025.

BeyondMimic learns cartwheels, spin-kicks, flip-kicks and sprinting with one setup and shared hyperparameters and transfers zero-shot to hardware^{Source 18}18 Liao et al. 2025, BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion. Qiayuan Liao, Takara E. Truong, Xiaoyu Huang et al., 2025. ZEST trains motion-imitation policies with RL from motion capture, monocular video and animation, and deploys them zero-shot on Atlas, the Unitree G1 and Spot^{Source 29}29 Sleiman et al. 2026, ZEST: Zero-shot Embodied Skill Transfer for Athletic Robot Control. Jean Pierre Sleiman, He Li, Alphonsus Adu-Bredu et al., 2026.

Three decompositions ship in 2026

HOVER distills full-body kinematic motion imitation across control modes into one policy^{Source 19}19 He et al. 2024, HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots. Tairan He, Wenli Xiao, Toru Lin et al., arXiv preprint (ICRA 2025), 2024. Figure, NVIDIA and Google DeepMind each shipped whole-body control in 2026 with the learning boundary in a different place.

A unified motion-tracking policy: Figure Helix 02 S0

Figure announced Helix 02 on 27 January 2026. It keeps the tracking decomposition: S2 reasons over scenes and language and emits latent goals; S1, a 200 Hz visuomotor transformer, maps head and palm cameras, fingertip tactile sensors and full-body proprioceptionproprioceptionProprioception is sensing the body's own state (joint positions and velocities, orientation, contact), as opposed to exteroception of the outside world through cameras or lidar. It is fast, cheap and never occluded. At ETH Zurich, Lee et al.'s ANYmal controller hiked mud, snow, rubble and vegetation on proprioception alone. Miki et al. kept it as the trusted fallback when the terrain map disagrees with the body. to whole-body joint targets; and S0, a 10M-parameter controller at 1 kHz, turns joint state and base motion into actuator commands. S0 trained on over 1,000 hours of joint-level retargeted human motion and sim-to-real RL across more than 200,000 parallel environments, and the headline demo is a 4-minute dishwasher unload and reload^{Source 10}10 Figure AI 2026, Introducing Helix 02: Full-Body Autonomy. Figure AI, 2026.

Figure says it later conditioned S0 on camera perception^{Source 20}20 Figure AI 2026, Ramping Figure 03 Production. Figure AI, Figure AI news, 2026, and reports that Helix 2.5, pretrained from random initialization on its Index human-behavior dataset, succeeded on 56% of blind zero-shot trials in 30 unseen homes, against 9% when trained from scratch^{Source 21}21 Figure AI 2026, Helix 2.5: Zero-Shot 30-Home Generalization. Figure AI, Figure news, 2026.

A VLA above a learned controller: GR00T + GEAR-SONIC

NVIDIA keeps the VLA and whole-body controller as separate learned systems, and released GR00T N1.7 as early access on 18 April 2026^{Source 28}28 NVIDIA 2026, n1.7 release. NVIDIA, GitHub release n1.7-release, NVIDIA/Isaac-GR00T, 2026. GR00T N1.7 is a 3B-parameter VLA with a Cosmos-Reason2-2B backbone and a flow-matchingflow matchingFlow matching is a generative modeling recipe that learns a time-dependent vector field carrying noise to data, by regressing against the conditional flow instead of estimating a score. In pi0, which brought it to robot control, an action expert trained this way denoises continuous action chunks. Integrating the field as an ODE for a handful of steps is fast enough for high-frequency control and keeps diffusion policies' multimodality. action head; with the UNITREE_G1_SONIC embodiment it predicts latent action tokens that the GEAR-SONIC controller, running at 50 Hz, decodes into joint commands for legs, arms and hands^{Source 11}11 NVIDIA 2026, NVIDIA Isaac GR00T. NVIDIA, 2026 ^{Source 12}12 NVIDIA 2026, GR00T-WholeBodyControl (GEAR-SONIC whole-body controller workflow). NVIDIA, 2026. SONIC (Science Robotics) scales tracking from 1.2M to 42M parameters on 700 hours of motion capture, with one token space serving VR teleoperation and VLAs^{Source 22}22 Luo et al. 2025, SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control. Zhengyi Luo, Ye Yuan, Tingwu Wang et al., arXiv preprint (Science Robotics 11(117), 2026), 2025. Code is under Apache 2.0 and weights under the NVIDIA Open Model License, and every checkpoint loads the gated Cosmos-Reason2-2B backbone^{Source 11}11 NVIDIA 2026, NVIDIA Isaac GR00T. NVIDIA, 2026.

Qiayuan Liao, Pieter Abbeel, Koushil Sreenath and colleagues at UC Berkeley (October 2026) train a tracker with analytic gradients through short rollouts of a differentiable world model fit to logged simulator transitions^{Source 32}32 Liao et al. 2026, SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model. Qiayuan Liao, Yiyang Shao, Junfeng Long et al., 2026. Their SuperMimic passed the return of the released SONIC checkpoint after 3 hours on one H100 GPU, where SONIC trained for 7 days on 128 L40 GPUs, and failed on 25 of its 129,514 training motions against 179 for SONIC^{Source 32}32 Liao et al. 2026, SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model. Qiayuan Liao, Yiyang Shao, Junfeng Long et al., 2026. The authors note that the GPU models differ and that SuperMimic trained on the public part of SONIC's data, about 2.3 times less motion^{Source 32}32 Liao et al. 2026, SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model. Qiayuan Liao, Yiyang Shao, Junfeng Long et al., 2026.

Photo: RuinDig/Yuki Uchida / Wikimedia Commons. Licence: CC BY 4.0.

One VLA, feet to fingertips: Gemini Robotics 2

Google DeepMind puts everything in one model: its 30 July 2026 announcement reports that Gemini Robotics 2 controls full humanoids "from feet to fingertips", fetching and shelving a watering can on Apptronik's Apollo 2. One checkpoint runs Apollo 2 with SharpaWave hands, Apollo 2 with Inspire hands and the Franka Duo with Robotiq grippers, and DeepMind reports adapting to new bi-arm robots in a few hours, with fewer than 200 examples in most cases^{Source 13}13 Parada 2026, Gemini Robotics 2 brings whole body intelligence to robots. Carolina Parada, 2026. Tap a body region in the figure below to see how a humanoid's motors are shared between legs, arms and hands.

Where do a humanoid robot's motors go?

A humanoid is shaped like us so it can use our doors, stairs and tools.

BackNext

Most of a humanoid's motors keep it standing and walking, and its battery decides how long it can work.

How this was made

The steps in words

1. A humanoid is shaped like us so it can use our doors, stairs and tools.
2. It has 23 motors, one in each joint, where the body bends or turns. Guess where most of them are.
3. The legs carry the whole body, about 35 kilograms, and keep it from falling, so they get the strongest motors.
4. The chest cover swings open: the battery and computer sit inside. One charge lasts only about two hours, which limits its working time.
5. Each motor adds one way to move, called a degree of freedom. Hands with fingers need about as many motors as the legs.

Parts

• Legs
• Arms
• Waist
• Hands
• Torso and head

Gemini Robotics 2 results

DeepMind's whole-body and gripper rates average several tasks per skill category; multi-finger rates are per task. Whole-body picking on Apollo 2 with Inspire hands averages 76.3% from a shelf, 68.4% from a table and 45.7% from the floor, against 89.6% for precise gripper insertion on the Franka Duo. Multi-finger dexterity on the SharpaWave hand spans 92% for unscrewing a bulb to 32% for a dustpan, and DeepMind says multi-finger dexterous manipulation "remains challenging"^{Source 13}13 Parada 2026, Gemini Robotics 2 brings whole body intelligence to robots. Carolina Parada, 2026.

DeepMind says its robots "have more to advance in movement speed"^{Source 13}13 Parada 2026, Gemini Robotics 2 brings whole body intelligence to robots. Carolina Parada, 2026.

What tracking does not solve

A tracker does not choose its own motion. Zhang and colleagues pair a terrain-aware diffusion motion generator with a tracker fine-tuned on its imperfect references, crossing boxes, hurdles and stairs on a G1^{Source 23}23 Zhang et al. 2026, Learning Whole-Body Humanoid Locomotion via Motion Generation and Motion Tracking. Zewei Zhang, Kehan Wen, Michael Xu et al., 2026. VIRAL's simulation-trained RGB policy ran continuous loco-manipulation for up to 54 cycles on a G1^{Source 30}30 He et al. 2025, VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation. Tairan He, Zi Wang, Haoru Xue et al., arXiv preprint, 2025, and Boston Dynamics reports that a policy trained for 50 to 70 pound loads moved a fridge of over 100 pounds^{Source 24}24 Rodriguez et al. 2026, Training a Humanoid Robot for Hard Work. Alberto Rodriguez, Shane Rozen-Levy, Vinay Kamidi, Boston Dynamics blog, 2026.

Yutang Lin, Guanya Shi and C. Karen Liu at Amazon FAR, Carnegie Mellon and Stanford (October 2026) plan in the latent command space of a frozen behavior foundation model, which closes the balance loop at 50 Hz while their FIBER planner replans at 5 Hz^{Source 33}33 Lin et al. 2026, Action-Aligned Latent Planning for Versatile Humanoid Loco-Manipulation. Yutang Lin, Guanya Shi, C. Karen Liu, 2026. FIBER projects separately trained policies for box pickup, door opening and grasping into that space and trains one perceptive planner on their rollouts; across 128 simulated variations per task it succeeded 42.0% of the time, against 11.6% for replaying the projected latents, 3.5% for a kinematic planner and 95.4% for the task experts^{Source 33}33 Lin et al. 2026, Action-Aligned Latent Planning for Versatile Humanoid Loco-Manipulation. Yutang Lin, Guanya Shi, C. Karen Liu, 2026. Trained only in simulation, FIBER succeeded in 6, 4 and 2 of 10 real G1 trials on the three tasks^{Source 33}33 Lin et al. 2026, Action-Aligned Latent Planning for Versatile Humanoid Loco-Manipulation. Yutang Lin, Guanya Shi, C. Karen Liu, 2026.

Zeji Yi, Guanya Shi and colleagues at Carnegie Mellon and Amazon FAR (October 2026) split collision avoidance between a perception-conditioned diffusion policy that proposes whole-body motions and an RL controller that tracks them and keeps balance^{Source 34}34 Yi et al. 2026, SWAP: Test-Time Steerable Humanoid Whole-Body Collision Avoidance. Zeji Yi, Guanqi He, Xiaofeng Guo et al., 2026. Their SWAP labels training data in hindsight, placing obstacles only where an executed motion did not sweep^{Source 34}34 Yi et al. 2026, SWAP: Test-Time Steerable Humanoid Whole-Body Collision Avoidance. Zeji Yi, Guanqi He, Xiaofeng Guo et al., 2026. Across five simulated tasks it averaged 88.3% success with test-time steering, 72.4% without it and 17.4% when the diffusion output drove the joints directly^{Source 34}34 Yi et al. 2026, SWAP: Test-Time Steerable Humanoid Whole-Body Collision Avoidance. Zeji Yi, Guanqi He, Xiaofeng Guo et al., 2026. On a real G1, with motion capture for localization and the moving obstacles, it succeeded in 18 of 25 trials; the authors trace the failures to LiDAR occlusion and tracking mismatch^{Source 34}34 Yi et al. 2026, SWAP: Test-Time Steerable Humanoid Whole-Body Collision Avoidance. Zeji Yi, Guanqi He, Xiaofeng Guo et al., 2026.

HITTER pairs a planner that predicts the ball and picks the strike with an RL whole-body controller, and reached up to 106 consecutive table-tennis shots against a human^{Source 26}26 Su et al. 2025, HITTER: A HumanoId Table TEnnis Robot via Hierarchical Planning and Learning. Zhi Su, Bike Zhang, Nima Rahmanian et al., 2025. Chris Paxton of Agility Robotics wrote in October 2026 that many such systems still see the ball through external motion trackers, which leaves onboard perception as the limit^{Source 27}27 Paxton 2026, What Do Robot Sports Teach Us?. Chris Paxton, It Can Think!, 2026.

Retargeting is lossy: artifacts such as foot sliding and self-penetration significantly reduce tracking-policy robustness, particularly for dynamic or long sequences^{Source 25}25 Araujo et al. 2025, Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking. Joao Pedro Araujo, Yanjie Ze, Pei Xu et al., 2025, and noisy, inconsistent transferred references motivated a 2026 robustness method with a fall-recovery curriculum^{Source 9}9 Ma et al. 2026, Robust and Generalized Humanoid Motion Tracking. Yubiao Ma, Han Yu, Jiayin Xie et al., 2026.

Nicholas Ioannidis, Xue Bin Peng and colleagues at Simon Fraser University, FieldAI and NVIDIA (October 2026) replace the weighted objectives of retargeting with a priority order: foot placement first, then contacts with objects or other characters, then the remaining pose objectives^{Source 31}31 Ioannidis et al. 2026, PriMoRetarget: Humanoid Motion Transfer Through Prioritized Motion Retargeting. Nicholas Ioannidis, Dun Yang, Max Smith et al., 2026. On two-person interactions from Inter-X their PriMoRetarget kept 78.8% of source hand contacts, and PPO policies trained on its output tracked the motions on a simulated Unitree G1 93.6% of the time^{Source 31}31 Ioannidis et al. 2026, PriMoRetarget: Humanoid Motion Transfer Through Prioritized Motion Retargeting. Nicholas Ioannidis, Dun Yang, Max Smith et al., 2026. The strongest of three baselines, PHC, reached 35.2% and 72.8%; GMR and OmniRetarget scored lower^{Source 31}31 Ioannidis et al. 2026, PriMoRetarget: Humanoid Motion Transfer Through Prioritized Motion Retargeting. Nicholas Ioannidis, Dun Yang, Max Smith et al., 2026. Collaborative manipulation by several robots stayed harder, at 62.6% tracking success^{Source 31}31 Ioannidis et al. 2026, PriMoRetarget: Humanoid Motion Transfer Through Prioritized Motion Retargeting. Nicholas Ioannidis, Dun Yang, Max Smith et al., 2026.

Chaitanya Chawla, Kartik Sharma, Guanya Shi and colleagues at Carnegie Mellon (October 2026) check reachability during generation: their EmbodyHOI steers a frozen generator of human hand-object motion with a differentiable inverse-kinematics cost for the target robot while it samples^{Source 35}35 Chawla et al. 2026, EmbodyHOI: Embodiment-Aware Generation of Hand-Object Interactions for Humanoid Dexterous Manipulation. Chaitanya Chawla, Kartik Sharma, Ananya Bal et al., 2026. On the Unitree G1 this cut the wrist error left after retargeting from 8.9 cm to 0.6 cm, and simulated policies trained on the guided references succeeded on 79% of 50 randomly drawn tasks, against 68% for unguided references^{Source 35}35 Chawla et al. 2026, EmbodyHOI: Embodiment-Aware Generation of Hand-Object Interactions for Humanoid Dexterous Manipulation. Chaitanya Chawla, Kartik Sharma, Ananya Bal et al., 2026. Fixing the arms with inverse kinematics after generation dropped success on a four-task suite to 32.8%, against 79.7% with guidance, which costs 32 times an unguided sample^{Source 35}35 Chawla et al. 2026, EmbodyHOI: Embodiment-Aware Generation of Hand-Object Interactions for Humanoid Dexterous Manipulation. Chaitanya Chawla, Kartik Sharma, Ananya Bal et al., 2026.

Figure and NVIDIA keep balance and contact beneath the foundation model, behind tracked joint targets or a latent token^{Source 10}10 Figure AI 2026, Introducing Helix 02: Full-Body Autonomy. Figure AI, 2026 ^{Source 11}11 NVIDIA 2026, NVIDIA Isaac GR00T. NVIDIA, 2026; DeepMind puts them inside one policy^{Source 13}13 Parada 2026, Gemini Robotics 2 brings whole body intelligence to robots. Carolina Parada, 2026.