Learning methods
How Small Errors Derail Robot Imitation Learning
Prediction errors can accumulate over time and drive an imitation-learning robot outside its training distribution; the article examines mitigations reported in a real-robot chopstick study and a simulated Policy-Calibrated DAgger study.
Robot imitation can break down across a sequence even when the initial problem is a single-step prediction error. A small mistake can lead to failure because prediction errors can accumulate over time, driving the robot outside its training distribution, where covariate shift is associated with poor generalization. The authors of the Policy-Calibrated DAgger preprint explicitly describe this drift as errors build up. The admitted abstracts establish that high-level chain, but they do not provide a state-by-state account of how one wrong action alters each subsequent observation.
A chopstick-manipulation study targeted both parts of the problem. Lacking an accurate model for its fine-manipulation task, the team used model-free imitation learning on a robot equipped with chopsticks. Its first approach applied what the authors call an invariant operator to increase data support at critical grasping steps and reduce single-step prediction errors. Its second added bounded noise and combined parametric and non-parametric methods to generate synthetic corrective labels intended to prevent error accumulation.
Neither approach required an interactive expert or a model, the chopstick study's authors report. On the real robot they built, they observed the agent's success rate increase from 37.3% to 80%, compared with reported human expert performance of 82.6%. This is a result from one chopstick-equipped system, and the admitted abstract does not report a separate success rate for each proposed approach.
Policy-Calibrated DAgger offers a later, different offline mitigation: its authors estimate a diffusion policy's noise from its own predicted action distribution, rather than selecting a noise level through a sweep. They report experiments in a 3D photorealistic simulator and a 2D planar reacher environment, not a physical-robot test in the admitted evidence. Together, the two preprints indicate why correction coverage matters around deviations, while their different settings and limited excerpts do not establish a universal solution to covariate shift.