Knowledge Insulation

RT-2 represents robot actions as tokens, using the discretization introduced for RT-1: each continuous action dimension is divided into 256 uniform bins. Its robot inputs pair a camera image with a task instruction Brohan 2023↓RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic ControlAnthony Brohan, Noah Brown, Justice Carbajal et al., 2023. π0 instead adds a continuous flow-matchingflow matchingA generative modeling recipe that learns a time-dependent vector field carrying samples from a noise distribution to the data distribution, trained by regressing the model's field against the conditional flow rather than by estimating a score. pi0 brought it to robot control: a dedicated action expert is trained with the flow-matching objective to denoise continuous action chunks, and at inference the learned field is integrated as an ODE for a handful of steps, which keeps the policy fast enough for high-frequency control while keeping the multimodality that made diffusion policies attractive. action expert to a pretrained VLMvision-language modelA model pretrained jointly on web-scale image and text data, so visual recognition and language semantics live in one set of weights. RT-2's bet was that this pretraining is an asset for robot control: co-fine-tuning a vision-language model on robot trajectories and its original web data together transfers semantic knowledge, recognizing objects and following instructions the robot data never covered, into the policy. The vision-language-action models that followed all start from such a backbone. Black 2024↓π0: A Vision-Language-Action Flow Model for General Robot ControlKevin Black, Noah Brown, Danny Driess et al., RSS 2025, 2024. Knowledge Insulationknowledge insulationPhysical Intelligence's training recipe for keeping action learning from damaging a vision-language-action model's pretrained backbone. The backbone trains on discrete action tokens with its usual autoregressive loss, a separate flow-matching expert produces the continuous actions, and a stop-gradient keeps the expert's gradients out of the backbone. On the paper's bussing task, a model trained this way reaches a given performance level in 7.5 times fewer steps than π0 and follows language better, in and out of distribution., studied by Physical Intelligence in 2025, combines the two: discrete actions train the backbone’s representations while a continuous-action expert trains alongside it without sending gradients back into the backbone.

Driess 2025↓Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025 The paper distinguishes the earlier two-stage π0.5 recipe from simultaneous single-stage training.

Driess 2025↓VLAs that Train Fast, Run Fast, and Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025 The research note calls this single-stage model π0.5 + KI.

The barrier is below. Switch the pass direction to backward and then turn the stop-gradient off: the damage the recipe exists to prevent runs straight back into the pretrained backbone.

The 7.5x figure compares training steps to similar table-bussing performance, not inference latency or equal wall-clock training budgets. The paper reports about 20% extra training cost from the combined objectives. Its Appendix B describes a 2B language-model backbone within the complete VLM and a 300M action expert; the research note labels the complete VLM as 3B Driess 2025↓Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025 Driess 2025↓VLAs that Train Fast, Run Fast, and Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025.

What naive joint training breaks

The paper studies a pretrained VLM with a newly initialized continuous-action expert. It finds that expert gradients can interfere with the image encoder and language backbone during adaptation, slowing learning and weakening language following. The proposed explanation is interference with pretrained representations, not a measured claim that every early gradient is large or that the model can never recover Driess 2025↓Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025.

Figure 2 illustrates the problem with π0: instructed to bus a spoon into the bin, it grasps trash instead. The research note describes the target as the dish container and explicitly calls its explanation a hypothesis: disrupted language processing may lead the model to learn other correlations first Driess 2025↓Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025 Driess 2025↓VLAs that Train Fast, Run Fast, and Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025. The note’s phrase "not suitable for high-frequency, precise, or fluent motions" refers to the earlier simple fixed-bin action representation, not to every possible tokenized-action method Driess 2025↓VLAs that Train Fast, Run Fast, and Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025.

The recipe, in three parts

Knowledge Insulation separates the learning signals Driess 2025↓Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025:

1. The action expert trains with flow matching to generate continuous action chunks at inference.
2. The backbone learns from discrete FAST action tokensaction tokenizationExpressing continuous robot actions as discrete tokens so a language model's machinery can produce them. RT-2 discretized each action dimension into 256 uniform bins and mapped the bin indices onto rarely used tokens of the model's existing vocabulary, which let the policy train with ordinary next-token prediction. The representation is simple and inherits the backbone's web knowledge, but 256 bins are coarse and autoregressive decoding is slow, the two weaknesses later work attacked with continuous experts and parallel decoding., alongside general vision-language and robot-planning data. FAST normalizes actions, applies a DCT separately along time for each action dimension, scales and rounds the coefficients, then uses byte-pair encoding to compress the flattened sequence. Coefficient quantization is lossy; BPE losslessly compresses the resulting integer sequence Pertsch 2025↓FAST: Efficient Action Tokenization for Vision-Language-Action ModelsKarl Pertsch, Kyle Stachowicz, Brian Ichter et al., 2025.
3. Stop-gradient blocks the expert’s updates to the backbone. The expert can attend to the image/language/state prefix and its own continuous-action tokens. It does not attend to FAST action tokens, and the backbone does not attend to the expert. The attention equations apply stop-gradient to the backbone keys and values read by the expert, leaving forward information available while blocking that backward path Driess 2025↓Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025.

The figure at the top of this module walks both passes through the two stacks, including the failure mode with the barrier removed.

Why the barrier alone is not enough

Stopping expert gradients is not the same as adapting the backbone for robot control. Without another robot-learning signal, the pretrained backbone’s representations remain unadapted to the task. In the paper’s frozen-backbone ablation, shirt-folding performance is reported as 0%; that is an outcome for this tested configuration, not proof that all frozen-backbone policies are impossible Driess 2025↓Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025.

FAST-token supervision supplies the missing action-learning signal while the gradient barrier blocks updates from the newly initialized expert. The paper also finds that vision-language co-training can give the joint-training baseline good language following even without stop-gradient. The evidence supports the combined recipe, not a claim that continuous gradients always destroy knowledge or that cross-entropy guarantees perfect preservation Driess 2025↓Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025.

What it buys

The paper separates two table-bussing comparisons. For a generalist trained on many embodiments, π0 needs 7.5 times as many training steps as π0.5 + KI to reach similar performance. For specialist models trained on one embodiment, the authors report that π0-FAST needs twice the wall-clock time to solve the task. That second result is task-completion time, not a generic twofold inference-latency multiplier or a comparison across all robot tasks Driess 2025↓Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025 Driess 2025↓VLAs that Train Fast, Run Fast, and Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025.

The table-bussing task starts with 12 objects and scores correct placement into the dish and trash receptacles. The paper reports ten episodes per real-world task and policy. In a separate mobile-manipulation object-generalization experiment, the robot moves unseen objects from a counter into an already-open drawer; the research note says web-data co-training boosted generalization most in that comparison. These are different evaluation populations, not one combined benchmark Driess 2025↓Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025 Driess 2025↓VLAs that Train Fast, Run Fast, and Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025.

Language following also improves in the items-in-drawer comparison, but remains imperfect. The paper notes that VLM co-training can help the joint-training baseline follow language even without stop-gradient Driess 2025↓Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025.

Both, in different places

In the π0.5 + KI recipe, action representation has two roles: FAST tokens provide training supervision for the backbone, while the flow-matching expert generates continuous actions at inference. The expert’s gradients do not update the backbone Driess 2025↓Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize BetterDanny Driess, Jost Tobias Springenberg, Brian Ichter et al., 2025. A comparison matrix should distinguish training representation from runtime output rather than force this model into a single binary category.

Related later models

The π0.6 model card describes a Gemma3 4B backbone and an approximately 860M action expert with the same number of layers; its architecture figure also labels a SigLIP 400M vision encoder. The backbone predicts FAST tokens, while the expert predicts continuous actions without sending its gradients back into the backbone. With five denoising steps and three camera inputs, producing a chunk takes 63 ms on one H100 Physical Intelligence 2025↓π0.6 Model CardPhysical Intelligence, 2025. π0.7 retains a Gemma3-based VLM, a MEM-style video-history encoder and a flow-matching expert. Its prompt adds language, episode metadata and subgoal images; it also uses FAST supervision with action-expert gradients stopped at the backbone Ai 2026↓π0.7: a Steerable Generalist Robotic Foundation Model with Emergent CapabilitiesBo Ai, Ali Amin, Raichelle Aniceto et al., 2026.