Real-Time Execution

A policy that needs 60 ms per inference cannot finish a new inference within every 20 ms tick of a 50 Hz controller. Chunked execution can still provide commands at 50 Hz. The control period at 50 Hz is 20 ms, and every component in the loop, cameras, network, preprocessing, model, spends from that budget. The earlier modules treated action chunkingaction chunkingPredicting a sequence of future actions in one inference instead of a single action per timestep. Introduced with ACT on the ALOHA platform, chunking cuts the number of sequential decisions in an episode by the chunk length, which reduces compounding error and gives the policy the temporal context it needs to represent pauses, at the cost of committing to a plan between replans. as an algorithmic choice; this module is about what happens when the algorithm meets the clock. Real-time execution turns out to be a first-class architectural problem, not an engineering afterthought: the way a policy hands off from one chunk to the next while inference is still running changes what the robot physically does, and the naive ways to do it fail in documented, measurable ways Black 2025↓Real-Time Execution of Action Chunking Flow PoliciesKevin Black, Manuel Y. Galliker, Sergey Levine, 2025.

The teaching model below compares one synchronous inference with a 20 ms tick. Its slider uses illustrative scaling and a deliberately chosen 3.0B reference coordinate; it is not a hardware profiler or a proof that a chunked 50 Hz robot controller cannot run.

Where the milliseconds go

The Real-Time Chunkingreal-time chunkingAn inference-time scheme that keeps chunked policies responsive when the model is slower than the control loop. While the robot finishes the current chunk, the policy conditions its next prediction on the actions that will already have executed by the time inference returns, holding that committed prefix fixed and generating only the remainder. Black, Galliker, and Levine introduced it for flow-matching policies, where it tracks moving targets under inference delays that break both naive chunk execution and temporal ensembling. paper reports a π0.5 latency breakdown with inference on an RTX 4090 in bfloat16, five denoising steps and a wired LAN. Across 50 inference calls during a real episode, mobile total latency is 138.98 ± 6.71 ms: model 96.89 ± 0.16, network 21.20 ± 3.12, image resizing 11.22 ± 5.00 and other work 9.67 ± 3.20 ms. The non-mobile total is 108.76 ± 2.34 ms; the ± values are one standard deviation. Image resizing uses an Intel i7-1260P NUC in the mobile case and an AMD Ryzen 9 7950X workstation in the non-mobile case Black 2025↓Real-Time Execution of Action Chunking Flow PoliciesKevin Black, Manuel Y. Galliker, Sergey Levine, 2025. At 50 Hz, the mean totals span about 6.95 and 5.44 control periods. Their reciprocals, about 7.20 and 9.19 inferences per second, do not establish the actual replanning or robot-control rate; the RTC experiment uses chunked execution.

Jiang, Clemons, Sankaralingam and Kozyrakis study the scaling picture with VLA-Perf, a roofline-based analytical model, not a hardware-wide profiling campaign Jiang 2026↓How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-PerfWenqi Jiang, Jason Clemons, Karu Sankaralingam et al., 2026. Its predictions assume optimistic software and attainable peak compute and memory bandwidth. The paper compares them with an optimized π0 Triton implementation on an RTX 4090: the implementation reaches 73.3 to 82.6% of predicted throughput under the tested one-, two- and three-camera settings, with 10 flow-matching steps, chunk size 63 and an empty language prompt.

In its separate B100 π0 model sweep, increasing denoising steps from 10 to 50 at chunk size 50 multiplies action-expert latency by 5, but total VLA latency by 2.15. Increasing chunk size from 50 to 250 at 10 steps raises expert latency by 40% and total latency by 11% Jiang 2026↓How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-PerfWenqi Jiang, Jason Clemons, Karu Sankaralingam et al., 2026. Chunk size matters less in this experiment; it is not free, and expert latency is not the whole pipeline.

The hardware wall

VLA-Perf predicts π0 inference at 162.5 Hz on H100 and 19.0 Hz (52.57 ms) on Jetson Thor, excluding network latency Jiang 2026↓How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-PerfWenqi Jiang, Jason Clemons, Karu Sankaralingam et al., 2026. The baseline is 2.7B parameters: a 0.4B vision encoder, 2B VLM and 0.3B action expert. It models three 224×224 cameras, 32 language tokens (800 input tokens total), 14 action dimensions, chunk size 50, 10 denoising steps and batch size one, using BF16 or FP16 where BF16 is unsupported. These are inferences per second, not robot/controller Hz; sensing and robot execution latency are outside the analysis.

The 9.1B π0-L result, 3.9 Hz on Thor, is for a hypothetical variant built from a 1.1B SigLIP-Giant encoder, 6.5B Llama2 backbone and 1.5B action expert, not a measured deployed policy Jiang 2026↓How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-PerfWenqi Jiang, Jason Clemons, Karu Sankaralingam et al., 2026. The interactive below uses a separate teaching curve; its 3.0B coordinate is not the paper’s 2.7B baseline.

VLA-Perf defines 10 Hz inference as “acceptable” and 100 Hz as “high-performance” relative to common camera rates; these are the authors’ analysis targets, not universal robot-control requirements Jiang 2026↓How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-PerfWenqi Jiang, Jason Clemons, Karu Sankaralingam et al., 2026. For their on-device π0 setup, the predicted 19 Hz exceeds the first target. They propose smaller models, fewer denoising steps or lower-precision quantization to approach 100 Hz. This does not show that future hardware cannot help.

The baseline π0 vision encoder, VLM and action expert are all memory-bound on the modeled Thor configuration. On the other evaluated GPUs, vision and VLM work are compute-bound while the action expert remains memory-bound. The component-scaling experiment finds approximately linear latency growth with parameter count Jiang 2026↓How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-PerfWenqi Jiang, Jason Clemons, Karu Sankaralingam et al., 2026. These results do not characterize every possible workload on Thor. The study assumes adequate model accuracy and evaluates BF16/FP16 inference; its quantization recommendation is not an FP4/FP8 2 to 4× speedup result or an accuracy-loss validation.

Server placement also depends on the modeled network Jiang 2026↓How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-PerfWenqi Jiang, Jason Clemons, Karu Sankaralingam et al., 2026. For synchronous π0 inference, RTX 4090 over WiFi 6 is 38.7 ms versus Thor’s 52.6 ms, but RTX 4090 over 5G is slower at 55.7 ms. B100 over the same 5G model is 27.8 ms. Table 7 assumes WiFi 6 upload/download bandwidth of 560/800 Mbps and 3.5 ms base latency, versus 80/500 Mbps and 10 ms for 5G. In the cloud examples, B100 with Ethernet 10G plus the fast-cloud link is 23.4 ms, while 4G plus the slow-cloud link is 273.4 ms. The fast/slow cloud links use 10/1 Gbps and 10/100 ms base latency; the 4G hop uses 19/75 Mbps and 25 ms. These are specified link assumptions, not measurements of arbitrary deployed networks.

Three ways to hand off a chunk

Suppose inference takes d milliseconds. The policy has emitted a chunk, the robot is executing it, and a new chunk is on the way. What happens at the boundary is a design decision, and there are three options on the table.

Synchronous execution waits: execute a chosen part of the current chunk, pause while the next one is computed, then resume. Do not equate that executed part with the whole predicted chunk: π0 v4 predicts 50 but executes 16 actions on the 20 Hz UR5e/Franka setups or 25 on the other 50 Hz robots Black 2024↓π0: A Vision-Language-Action Flow Model for General Robot ControlKevin Black, Noah Brown, Danny Driess et al., RSS 2025, 2024. The RTC paper evaluates stopping between chunks as a π0.5 baseline. Its discussion links altered motion dynamics to distribution shift, and its real-world comparison reports faster progress with RTC even after inference pauses are removed from elapsed time Black 2025↓Real-Time Execution of Action Chunking Flow PoliciesKevin Black, Manuel Y. Galliker, Sergey Levine, 2025.

Naive switching never waits: the instant the new chunk arrives, execution jumps to it. There is no dead time, but there is a discontinuity. The new chunk was inferred from an observation d milliseconds old, the world has moved since, and the two chunks disagree about what to do right now. The command steps by the full disagreement in a single control tick: a jerk event the hardware feels.

Temporal ensemblingtemporal ensemblingACT's execution scheme for overlapping action chunks: the policy is queried at every timestep, and the predictions that each in-flight chunk made for the current step are averaged with exponential weights. In the pinned ACT reference implementation, retained predictions are ordered oldest-to-newest and weighted by exp(-0.01 * i), so the oldest retained prediction has the largest weight. This smooths the handoff between chunks at the cost of one inference per step instead of one per chunk., ACT’s 2023 method, queries the policy at every timestep and combines predictions for that same timestep with weights $w_i=\exp(-mi)$. The paper assigns $i=0$ to the oldest retained prediction, so for positive $m$ older predictions receive greater weight; it does not say newer observations dominate Zhao 2023↓Learning Fine-Grained Bimanual Manipulation with Low-Cost HardwareTony Z. Zhao, Vikash Kumar, Sergey Levine et al., RSS 2023. Averaging predictions from different action modes need not produce a valid action. In the RTC paper’s π0.5 real-robot comparison, both temporal-ensembling variants trigger the robot’s protective stop at +100 and +200 ms of injected latency Black 2025↓Real-Time Execution of Action Chunking Flow PoliciesKevin Black, Manuel Y. Galliker, Sergey Levine, 2025.

Real-Time Chunking guides generation of a new action chunk while the current one executes. Its inpainting guidance assigns weight 1 to the first $d$ actions, which will execute during inference; weight 0 to the non-overlapping tail; and decaying intermediate weights to the remaining overlap. These are soft-mask guidance weights, not a claim that transformer attention itself is partially disabled Black 2025↓Real-Time Execution of Action Chunking Flow PoliciesKevin Black, Manuel Y. Galliker, Sergey Levine, 2025. The original method is an inference-time procedure for diffusion/flow policies without retraining. In the paper’s six-task π0.5 real-robot evaluation with five denoising steps, RTC’s aggregate throughput shows no degradation over the tested +0, +100 and +200 ms injected delays. The paper also illustrates precise match-lighting at total inference delay above 300 ms; that is not a claim that every dynamic task succeeds under every such delay Black 2025↓Real-Time Execution of Action Chunking Flow PoliciesKevin Black, Manuel Y. Galliker, Sergey Levine, 2025.

The comparison below is an illustrative hand-off model, not an implementation or measurement of RTC. One delay slider drives three constructed velocity traces. The values use arbitrary velocity units; the 0.30 per-tick threshold is a discontinuity proxy, not physical jerk. Its five-tick linear blend is a teaching assumption, not transformer attention or the paper’s inpainting solver.

Training the tolerance in

π0.7 uses a training-time version of Real-Time Chunkingreal-time chunkingAn inference-time scheme that keeps chunked policies responsive when the model is slower than the control loop. While the robot finishes the current chunk, the policy conditions its next prediction on the actions that will already have executed by the time inference returns, holding that committed prefix fixed and generating only the remainder. Black, Galliker, and Levine introduced it for flow-matching policies, where it tracks moving targets under inference delays that break both naive chunk execution and temporal ensembling.. Its paper states that training simulates delays of 0 to 12 timesteps, corresponding to a maximum inference latency of 240 ms on a 50 Hz robot Ai 2026↓π0.7: a Steerable Generalist Robotic Foundation Model with Emergent CapabilitiesBo Ai, Ali Amin, Raichelle Aniceto et al., 2026. Treat that as a stated training setting, not a measured worst-case latency guarantee or proof of a production deployment. The experiments predict 50 actions and execute either 15 or 25; UR5e robots run at 20 Hz and the other tested robots at 50 Hz. The paper does not assign the execution choices to specific robots Ai 2026↓π0.7: a Steerable Generalist Robotic Foundation Model with Emergent CapabilitiesBo Ai, Ali Amin, Raichelle Aniceto et al., 2026.

Spending the step budget

VLA-Perf’s π0 sweep shows that step count affects expert and total latency by different factors Jiang 2026↓How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-PerfWenqi Jiang, Jason Clemons, Karu Sankaralingam et al., 2026. The following papers report their own step settings. pi0 integrated its flow with 10 Euler steps per chunk Black 2024↓π0: A Vision-Language-Action Flow Model for General Robot ControlKevin Black, Noah Brown, Danny Driess et al., RSS 2025, 2024. pi0.6 cut that to 5, landing at 63 ms per chunk on a single H100 with three cameras Physical Intelligence 2025↓π0.6 Model CardPhysical Intelligence, 2025. pi0.7 stays at 5 Ai 2026↓π0.7: a Steerable Generalist Robotic Foundation Model with Emergent CapabilitiesBo Ai, Ali Amin, Raichelle Aniceto et al., 2026. These three source-specific choices do not establish field-wide nonadoption of one-step distillation or show that it matters chiefly on edge hardware. π0.7’s paper documents its own five-step policy with training-time RTC, not a survey of other laboratories Ai 2026↓π0.7: a Steerable Generalist Robotic Foundation Model with Emergent CapabilitiesBo Ai, Ali Amin, Raichelle Aniceto et al., 2026.