Manipulation & Learned Policies
Knowledge Insulation
Training the VLM backbone on discrete tokens while a flow-matching expert learns actions behind a stop-gradient.
Last reviewed 2026-08-08
For three years, the field framed action representation as a fork. The RT-1 and RT-2 generation discretized actions into bins and predicted them as tokens Brohan 2023. pi0 went the other way, bolting a continuous flow-matching action expert onto a pretrained VLM Black 2024. Knowledge Insulation, published by Physical Intelligence in 2025, is the paper that dissolved the fork: the modern recipe uses both representations at once, in different places, with a gradient barrier between them Driess 2025.
What naive joint training breaks
Start from pi0's architecture: a pretrained vision-language backbone and a small, randomly initialized action expert, trained jointly on robot data. The problem is the loss mismatch. The backbone was built by cross-entropy on text and images. The expert is trained by a continuous flow-matching denoising objective, and its early gradients, large and unstructured, flow straight into the backbone through the attention connections and rewrite representations that took a web-scale pretraining run to build Driess 2025.
The observable symptom is that the robot stops listening. The paper's example: told to put a spoon in the dish container, the jointly trained model grabs the trash instead. The proposed mechanism is that the language pathway is disrupted early in training, so the model falls back on easier visual correlations (containers look like targets) and never recovers its instruction following Driess 2025 Physical Intelligence 2025. Physical Intelligence's verdict on the alternative, making the backbone itself emit actions as text tokens, is blunt: it "is not suitable for high-frequency, precise, or fluent motions," like controlling your arm by verbally saying which muscles should contract Physical Intelligence 2025.
The recipe, in three parts
Knowledge Insulation keeps both losses and gives each one a place where it does no harm Driess 2025:
- The action expert trains with flow matching on the continuous actions that actually execute. Nothing changes here; flow matching's fast, few-step inference is the property worth keeping.
- The backbone trains on discrete FAST action tokens via ordinary cross-entropy, plus web vision-language data and high-level subtask data. FAST compresses an action chunk through a DCT and byte-pair encoding into a short token stream Pertsch 2025, so the backbone keeps learning about actions, but with the same kind of objective it was pretrained on.
- A stop-gradient severs the backward pass at the interface. In the forward pass the expert still attends into backbone activations at every layer, so it reads everything the backbone knows. In the backward pass, no gradient from the flow-matching loss reaches the backbone. The insulation is one-directional: information flows forward, damage does not flow back.
The interactive below walks both passes through the two stacks. Run the backward pass with the stop-gradient off to see the failure mode the recipe exists to prevent.
backbone supervision: no gradients (inference)
measured effect: 7.5x fewer training steps to the same bussing-task performance vs pi0
Schematic: 8 layers drawn per stack for legibility, and the 0-100 language-following score is an illustrative rendering of the paper's qualitative finding (an uninsulated model told to put a spoon in the dish container grabs the trash instead), not a published curve. The sourced figures are the parameter counts and the 7.5x training-step ratio. Source: Driess 2025
Why the barrier alone is not enough
The ablation that makes this a recipe rather than a one-liner: stop-gradient by itself fails. A backbone that is fully insulated and never sees robot data keeps its language abilities, but its representations are not adapted for motor control, and the action expert cannot use them. In the paper's configuration, shirt folding is unsolvable Driess 2025.
This is where the FAST-token supervision earns its place. The backbone still needs to learn from robot data; the question is which loss delivers that learning. Continuous denoising gradients corrupt it. Cross-entropy on discrete action tokens adapts it for control while matching its pretraining objective, so the knowledge survives the adaptation. Freezing the backbone protects its knowledge but gives the expert nothing usable, and joint training adapts the backbone but corrupts it. The FAST-token objective gets both properties at once Driess 2025.
What it buys
On the bussing task, pi0.5 trained with Knowledge Insulation reaches a given performance level in 7.5x fewer training steps than pi0, while keeping pi0's flow-matching inference speed, an advantage pi0-FAST gave up (autoregressive decoding made it roughly 2x slower to complete tasks in a head-to-head comparison). It also beats both pi0 and pi0-FAST on language following, in-distribution and out-of-distribution Driess 2025 Physical Intelligence 2025. Of the co-training data sources, plain web vision-language data contributes the largest single boost to object generalization, which is consistent with the diagnosis: the backbone's pretrained knowledge is the asset the recipe is designed to preserve Driess 2025.
Both, in different places
The taxonomy consequence is worth stating flatly, because comparison tables keep getting it wrong. Discrete tokens versus continuous actions stopped being a binary fork in 2025. From pi0.5 onward, the answer is both, in different places: discrete FAST tokens supervise the backbone as a training objective, the flow-matching expert produces the continuous actions that execute, and a gradient barrier sits between them Driess 2025. Any comparison matrix that forces a single binary choice on the action-representation axis misrepresents every frontier model from pi0.5 onward. The comparison matrix module therefore records representation per component, not per model.
The default recipe from pi0.5 onward
Subsequent generations treat the recipe as settled infrastructure. pi0.5's open-world generalization results rest on it Black 2025. pi0.6 scales it: a Gemma3 4B backbone with a SigLIP vision encoder, an 860M expert with the same layer count as the backbone, Knowledge Insulation end to end, and 5 denoising steps at inference Physical Intelligence 2025. pi0.7 keeps the same structure and spends its training budget on richer conditioning context instead Physical Intelligence 2026. When a new VLA paper reports that its backbone retains its VQA accuracy after action training, this is the mechanism doing the work.
How to read this module
The mechanism, ablations, and the 7.5x figure come from the Knowledge Insulation paper (NeurIPS 2025) and Physical Intelligence's research note. The interactive's 8-layer stacks and 0-100 language meter are illustrative renderings of a qualitative finding, not published measurements. The sourced numbers are the parameter counts (3B backbone, 300M expert in the paper's configuration; 4B and 860M in pi0.6) and the training-step ratio.
The pi line module traces how this recipe arrived between pi0-FAST and pi0.5 and what the closed generations built on top of it. The VLA models module covers the discrete-token generation that came before, including why naive binning fails at 50 Hz and what FAST changed.