Behavior Cloning Foundations

Behavior cloning is the oldest way to make a robot imitate a person: record what an expert did, treat each observation as an input and the expert's action as the label, and fit a policy $\pi(a \mid s)$ by supervised learning. The idea predates deep learning. In 1988, ALVINN trained a three-layer network to steer a van from camera images Pomerleau 1988ALVINN: An Autonomous Land Vehicle in a Neural NetworkDean A. Pomerleau, NeurIPS 1988, 1988, and the same recipe, scaled up, still sits underneath most modern manipulation policies.

The appeal is obvious: no reward function to design, no exploration to manage, just regression onto demonstrations. The failure mode is equally specific, and nearly every architectural innovation in learned manipulation since 2023 is a partial answer to it. Supervised learning assumes that test inputs come from the same distribution as training inputs. A robot running in closed loop violates that assumption with its own actions.

Covariate shift, stated precisely

Training data is drawn from the expert's state distribution $d_{\pi^*}$. At deployment the robot visits its own distribution $d_\pi$, and the two diverge the moment the policy makes its first mistake. A small error moves the robot into a state slightly outside the demonstrations; from that unfamiliar state the policy is more likely to err again; each error pushes it further out. Ross, Gordon, and Bagnell made this precise: a policy with per-step error $\varepsilon$ on the expert distribution can incur total cost growing quadratically in the episode length $T$, and they gave a matching lower bound showing the quadratic term is unavoidable under i.i.d. training Ross 2011A Reduction of Imitation Learning and Structured Prediction to No-Regret Online LearningStéphane Ross, Geoffrey J. Gordon, J. Andrew Bagnell, AISTATS 2011, 2011.

$$\underbrace{O(\varepsilon T^2)}_{\text{i.i.d. behavior cloning}} \qquad \underbrace{O(\varepsilon T)}_{\text{on-policy supervision}}$$

The second bound is what becomes available when the training distribution covers the states the policy actually visits, which is the DAgger result discussed below Ross 2011A Reduction of Imitation Learning and Structured Prediction to No-Regret Online LearningStéphane Ross, Geoffrey J. Gordon, J. Andrew Bagnell, AISTATS 2011, 2011. The gap between the two is the difference between a policy that demos well and one that survives long episodes. Physical Intelligence restated the same point in operational terms in 2025: it is relatively easy to get a learned policy to succeed at a task some of the time, and much harder to make it succeed reliably Physical Intelligence 2025π*0.6: a VLA that Learns from ExperiencePhysical Intelligence, 2025.

Watch the error compound

The interactive below simulates the mechanism. A policy with per-step error $\varepsilon$ rolls out along a demonstrated path; each error persists and slightly amplifies the next one, so the accumulated deviation tracks the quadratic bound. Two comparisons matter. First, switching from per-timestep prediction to committing to a chunk of 25 actions cuts the number of closed-loop decisions by 25x, and the accumulated deviation drops with it; this is the trick the next module's ACT exploits Zhao 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost HardwareTony Z. Zhao, Vikash Kumar, Sergey Levine et al., RSS 2023, 2023. Second, enabling DAgger-style relabeling, where an expert periodically corrects the states the policy actually visits, keeps the deviation bounded and the cost near-linear in $T$.

Fitting demonstrations is hard before deployment even starts

Compounding error is the deployment problem. Three training-side problems compound it:

Multimodality. Human demonstrations rarely contain one strategy. If half the demonstrators pass left of an obstacle and half pass right, a unimodal policy trained by mean-squared error regresses toward the mean of the two modes, which is a path straight into the obstacle. Representing the demonstration distribution faithfully, rather than its average, is the problem diffusion policies and flow-matching heads were built to solve Chi 2023Diffusion Policy: Visuomotor Policy Learning via Action DiffusionCheng Chi, Zhenjia Xu, Siyuan Feng et al., RSS 2023, 2023.

Temporally correlated confounders. Demonstrations contain pauses and idle segments that are unpredictable from a single Markovian state. A single-step policy cannot tell "waiting" from "acting," and both ACT and Diffusion Policy document robots freezing in place because of it Zhao 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost HardwareTony Z. Zhao, Vikash Kumar, Sergey Levine et al., RSS 2023, 2023 Chi 2023Diffusion Policy: Visuomotor Policy Learning via Action DiffusionCheng Chi, Zhenjia Xu, Siyuan Feng et al., RSS 2023, 2023. The fix is temporal context, not a larger backbone.

The demonstration ceiling. Behavior cloning can at best match the demonstrator. If the teleoperator is slow, cautious, or inconsistent, the policy inherits all of it, and the covariate-shift analysis above means it will usually do worse.

DAgger: relabel the states you actually visit

DAgger (Dataset Aggregation) attacks the quadratic bound directly Ross 2011A Reduction of Imitation Learning and Structured Prediction to No-Regret Online LearningStéphane Ross, Geoffrey J. Gordon, J. Andrew Bagnell, AISTATS 2011, 2011. The loop:

Because the training set grows to cover the distribution of states the learned policy induces, the mismatch that drives compounding error shrinks each iteration. Formally, DAgger reduces imitation learning to no-regret online learning, which is where the $O(\varepsilon T)$ bound comes from Ross 2011A Reduction of Imitation Learning and Structured Prediction to No-Regret Online LearningStéphane Ross, Geoffrey J. Gordon, J. Andrew Bagnell, AISTATS 2011, 2011.

The catch is the expert. Classic DAgger needs a supervisor who can look at an arbitrary mid-failure robot state and name the correct action, which is unnatural and slow for a human teleoperator. The variant that survives in practice is intervention-based: the operator watches the rollout and takes over only when the robot starts failing, as in HG-DAgger Kelly 2019HG-DAgger: Interactive Imitation Learning with Human ExpertsMichael Kelly, Chelsea Sidrane, Katherine Driggs-Campbell et al., ICRA 2019, 2019. Fourteen years later, the same pattern shows up as the "coaching" stage of Physical Intelligence's Recap, where human corrections on visited states fine-tune a vision-language-action policy Physical Intelligence 2025π*0.6: a VLA that Learns from ExperiencePhysical Intelligence, 2025.

Where this leads

Every module in this section is, in part, a different answer to the bound above. Action chunking shrinks the effective horizon by predicting sequences instead of single steps. Diffusion policies fix the multimodality that MSE regression destroys. RL fine-tuning and Recap-style coaching put DAgger's on-policy supervision into a form that scales to frontier VLAs. The terminology changed over fifteen years; the quadratic term never went away.