The reliability gap is the difference between the success rates a robot shows in controlled settings and the rates deployment requires. A robot that finishes a task 80% of the time is a demo, one that finishes it 99.9% of the time is a product, and Bessemer calls closing the gap "not a linear problem"Source 1.

A scan, label, apply and manifest machine straddling a parcel conveyor inside Amazon's Swindon fulfilment centre, with label printers mounted above the belt inside a yellow guard frame.
A parcel labelling line at Amazon's Swindon fulfilment centre in 2025. Fixed automation repeats one narrow task and has to finish it nearly every time, the product end of the gap.
Photo: Auledas / Wikimedia Commons. Licence: CC BY 4.0.
21%
95% per-step
end-to-end over 30 steps
74%
99% per-step
end-to-end over 30 steps
97%
99.9% per-step
end-to-end over 30 steps

At 99.9% per step, a 30-step task still fails about once in 34 runs

A task succeeds only when every step does: if each of NN steps succeeds independently with probability pp, the task completes with probability pNp^NSource 30. A result without its horizon cannot be compared, and papers often omit runs, initial conditions and success criteriaSource 9. Try the presets, then drag the per-step slider under "Adjust more" toward 100.

Compounding error
Get each step 95% right, and long jobs mostly fail
How often each step goes right
Adjust more
White trapdoors: the steps · blue marbles: jobs that finish95.0% per step · 30 steps · 21.5% of jobs finish

Illustrative: a simple model, not a measured robot.

Even a robot that rarely slips usually fails a long chore, which is why each step must be almost perfect.
How this was made

This is an illustrative probability model, computed rather than measured on a robot. A job (an episode) is a chain of steps (decisions), and it finishes only if every step goes right. If each step succeeds with the same probability p, independently of the others, a job of n steps finishes with probability p^n. At the current setting that is (0.950)^30 = 21.5%. Real failures are rarely independent, so treat the curve as a teaching model, not a forecast.

The drawing is a marble run of 100 trapdoor gates, ten to a rail; the first 30 are armed as the job’s steps and the rest are pinned shut. A hundred marbles, one job each, roll from the hopper. Each armed gate passes the share p of the marbles that reach it, rounded to whole marbles, so after k gates round(100 p^k) are still rolling, and the finish tray holds the readout’s share as whole jobs out of 100 (now 21). Which marble drops at which gate is a fixed shuffle of the release order; the counts do not depend on it. A run plays in about ten seconds, not to scale. Working backwards, a job that must finish with probability q over n steps needs each step to succeed with probability q^(1/n).

On the reliability-gap calculator a 95.0 percent per-step policy yields 21.5% episode success at 30 steps and only 0.6% at the 100-step far end, with the 50 percent crossing near step 14. In the drawing, 21 of 100 marbles reach the finish tray.

Sampled episode success by episode length
episode lengthepisode success
0 steps100.0%
10 steps59.9%
25 steps27.7%
50 steps7.7%
75 steps2.1%
100 steps0.6%
Here the calculator sets the per-step bar a deployment must clear; the curve is taught in Evaluation crisis.

Real errors are not independent: a mistake carries the robot outside its training distribution, where the next is likelier, so compounding error makes the true curve worse. An imitator with per-step error ε\varepsilon can make up to T2εT^2\varepsilon mistakes over TT stepsSource 10, and smooth deterministic imitators of continuous actions suffer error "exponentially larger, as a function of problem horizon"Source 11. Holding 99% end-to-end on a 50-step task needs about 99.98% per step, which no generalist policy publishes.

Formal methods do not close the last stretch from 99% to 99.9%

Bessemer argues the last stretch needs "fundamentally different approaches", naming tactile sensing, force feedback and better sim-to-real transfer. Lisa Yan of Argus Systems calls closing the gap between 99% and 99.9% "a steep hill climb that takes longer than most people realize"Source 1.

Hamilton-Jacobi reachability faces "exponential computational complexity with respect to the number of state variables"Source 12, and no deployed vision-language-action model publishes formal safety guarantees. Assurance therefore rests on empirical evaluation and runtime monitoring, which makes the reliability gap and the evaluation crisis one problem.

RL from experience lifts single tasks into the 90% range and above

Physical Intelligence reports that π*0.6 with Recap, learning from the robot's own experience, cuts failure rates by 2x or more on hard tasks, with more than 2x throughput on some of the hardest tasksSource 2. Its paper puts every task except diverse laundry in the 90%+ rangeSource 13. RL-100 succeeded in 1,000 of 1,000 episodes across 8 tasks, including 250 consecutive successes on one task, and a juicing robot served mall customers for about seven hours without failureSource 3. Real-Time EXPO-FT lifts four dynamic tasks from 42% to 97%Source 14.

These are mostly per-task specialists, though RL-100 keeps about 90% zero-shot success under environmental and dynamics shiftsSource 3 and RAPolicy raised five jointly trained tasks from 52% to 88%Source 15. Generalist reports a pretraining route: GEN-1 averages 99% on fine-tuned tasks where GEN-0 reached 64%Source 16.

Runtime monitors make failures visible, and human help rescues episodes

Google's ASIMOV-Agentic benchmark tests whether an embodied-reasoning agent respects payload, reach and safety limits and stops near peopleSource 4. DeepMind calls Gemini Robotics ER 2 its safest robotics model on such benchmarksSource 5. FAIL-Detect learns to flag failures "from successful training data alone"Source 17, and Sentinel's two detectors together catch 18% more failures than either aloneSource 18. A monitor catches failures instead of preventing them, so it raises effective reliability without changing the policy's own success rate.

π0.7 accepts step-by-step language coaching that walks it through tasks it has not seenSource 6. HG-DAgger lets a human expert take control to correct compounding errorSource 19, and 1X says NEO owners can schedule a remote expert for chores it does not knowSource 20.

Dated company records report hours and a single site's accuracy

Agility said in September 2026 that Digit 4 has logged more than 65,000 hours of operational time (industrial deployment), with about 98% on-task accuracy across 100,000 totes at GXO's Flowery Branch siteSource 21. Figure's November 2025 report says its BMW Spartanburg deployment logged 1,250+ runtime hours and loaded 90,000+ parts in eleven months, while its 84-second cycle and >99% placement accuracy were targets it did not report as achievedSource 7. Tesla's Q1 and Q2 2026 updates list both Optimus lines as under construction, with first builds going to an "Optimus Academy for training data collection"Source 8 Source 22.

Two silver and black Tesla Optimus humanoid robots standing on round black pedestals in a white-walled showroom.
Two Tesla Optimus humanoids on display in Stockholm in 2024. Tesla's 2026 updates list both of its Optimus production lines as still under construction.
Photo: Ulf Klingström / Wikimedia Commons. Licence: Public domain.
Deployment records
Humanoid work records so far: company numbers, none independently audited
Show records
Adjust more

4 of 4 records

None of these numbers has been checked by an outside auditor, so read them as what companies say, not proven performance.
How this was made

Each bar is the work a company reports for its humanoid robots. The scale grows by ten times at each tick, so 200 hours and 65,000 hours fit on one line.

“Backed by a company document” means the figure traces to a cited company document at that document’s own date; the hours and parts are vendor reports, and no independent audit is claimed. “Demo only” marks a vendor demonstration without a published deployment metric. The records come from each document at its own date, not from one common snapshot. Tesla reports no hours: its lines are under construction, and installed capacity is not output.

Deployment programs with their reported value, what backs each record, and the source behind it. The filter above the chart narrows these rows too.
ProgramValueWhat backs itSource
Agility DigitAgility’s Digit 5 release says Digit 4 has logged more than 65,000 hours of operational time, and about 98% accuracy while on-task over 100,000 totes at GXO’s Flowery Branch site; company-reported, not audited.65,000Operating hoursBacked by a company documentAgility RoboticsSep 15, 2026
Figure 02 at BMW SpartanburgFigure reports runtime and 90,000+ parts loaded at BMW Spartanburg during an eleven-month deployment. The 84-second cycle and >99% accuracy were targets, not achieved KPIs.1,250+Operating hoursBacked by a company documentFigure AINov 19, 2025
Tesla OptimusTesla’s Q2 update still lists its California and Texas Optimus lines as under construction, and says the first builds will collect training data in an “Optimus Academy”; installed capacity is not output.ConstructionProduction statusBacked by a company documentTeslaQ2 2026
Figure Helix 02Package-sorting livestream begun May 13, 2026 as one planned eight-hour shift; Figure’s September post says it ran autonomously for 200 hours. One task at one site, with no published success rate or intervention count.200 hoursAutonomous runDemo onlyFigure AISep 17, 2026

Figure's May 2026 package-sorting livestream sits under "Demo only". Planned as one eight-hour shiftSource 29, it ran for 200 hours by Figure's accountSource 23. Business Insider reported that a stuck robot triggered an automatic resetSource 24. It covered a single task at a single site, with no published success rate.

Showing 99.9% takes about 3,000 straight successes

A testable yardstick would document mean time between failures on real tasks, the human-robot safety assurance method and a protocol for remaining failures. None of the vendor hours above discloses a mean time between failures, an intervention count or general-purpose deployment at scale. With zero failures in nn trials, the 95% upper bound on the failure rate is about 3/n{3/n}Source 25, so showing 99.9% takes about 3,000 straight successes.

Real-robot comparisons are often limited to 10 or 50 trialsSource 26, and TRI's blind, randomized trials show the needed disciplineSource 27. Waymo's crash-rate claim rested on 56.7 million rider-only milesSource 28 (autonomous vehicles). Safety and assurance covers the standards a learned policy cannot be rated under.

Self-check

Your manipulation system sits at 99% success on a 30-step task and a customer needs 99% end-to-end. A vendor offers a runtime monitor that detects failures as they start. Can the monitor get you there?
Read the reasoning
  • Yes, roughly: catching failures before they compound means most detected episodes still complete, so the effective rate climbs toward what the customer needsThis overstates what a monitor does. ASIMOV-Agentic and Gemini Robotics ER 2 raise effective reliability by refusing unsafe calls and triggering safe stops, which turns silent failures into visible ones; a task the system safely aborted still did not complete, and the monitored number is not the policy's own success rate.
  • No, only on-robot experience moves the policy's own number: a monitor converts failures into safe stops and human escalationsMonitoring catches failures instead of preventing them. The methods that move per-step reliability are RL from experience: π*0.6 cuts failure rates by 2x or more on hard tasks and RL-100 reaches 100% across 1,000 episodes, both as per-task specialists.
  • No, so put a human coach on every episode insteadCoaching helps: π0.7 accepts step-by-step language coaching through tasks it has not seen. The number the customer needs is the one with the coach removed, and a coached episode does not show the system reaching 99% on its own.

Monitors make failures visible and coaches rescue episodes; only on-robot experience moves the policy's own per-step number.