Simulation to reality
Why can a robot trained in simulation fail in the real world?
A policy trained with simulated observations and physics may face different observations and action results on hardware. Physical trials test whether it transfers to a particular robot and task.
Simulation supplies abundant training data without requiring every trial on a physical robot. It also uses an approximate model. When that model and the hardware differ, behavior that succeeds in simulation may fail on the robot.
Where can the mismatch appear?
Synthetic sensory observations can differ from those measured in the real world. The modeled arm may not precisely match its hardware. A physical controller can respond differently to the same command, while contact and collision can be hard to simulate accurately. The paper describes these gaps in manipulation. They are possible causes to test, not a diagnosis of every failed transfer.
What can researchers change?
One method varies the simulated world during training. In a dynamics randomization study, Peng and colleagues changed properties including mass and friction. Their physical test used a 7-DOF Fetch Robotics arm for a puck-pushing task. Their result is evidence for that pushing task under its test conditions, not a guarantee for other robots or kinds of contact.
The human correction study begins with base policies learned in simulation, then deployed to real robots. Human operators intervene through teleoperation when the robot needs correction. Their intervention data are used to train a residual policy. The authors report 20 trials for each physical robot evaluation and say their method had the best average performance and the best result in all four tasks they tested. They also state that operators decide when to intervene and that the method needs a simulation policy that already performs reasonably well.
Must learning begin in a simulator?
In a separate study, the authors trained online and directly in the real world on four robots, without any simulators. Their system learns a world model from stored real experiences and uses predicted trajectories to train its policy, while gathering further physical experience. The authors note that physical training creates wear on robots that may require human intervention or repair.
These studies answer different questions. Peng's physical test concerns a puck-pushing policy trained with varied simulation dynamics. A simulator score alone cannot establish whether another robot will complete another task; the physical task and observed result set the scope of each finding.