How Robots LearnGuides
A person holds a controller while a robot arm grips a wooden block on a lab workbench; conceptual illustration.
AI-generated conceptual illustration for How Robots Learn; not a depiction of a specific experiment.

How learning works

How do robots learn from human demonstrations?

To learn one task from a person, a robot can record what it observes and what the operator commands. An imitation-learning policy is trained on those examples, then run on the robot to see what it does on its own.

howrobotslearn · Written

Presence first verified on . First publication date unknown.

Imagine guiding a robot arm to move a block into a bin, a task like the one in LeRobot's versioned tutorial. There, a person teleoperates a robot to record motion trajectories. LeRobot's versioned overview describes a demonstration dataset as synchronized camera video and the actions taken. This is the useful distinction: the camera image is part of what the robot observes, while an action is a control choice made at that moment. A guided movement gives the learner examples of both sides of that pairing.

From examples to a policy

LeRobot's tutorial next trains a neural network to imitate the recorded trajectories. The trained neural network is a policy: it controls the robot when it runs on its own. Training on a demonstration is therefore a way to learn a control rule from examples, rather than asking a person to guide every later attempt. The tutorial describes this workflow; it does not establish that any particular dataset will make a robot reliable.

LeRobot first recommends keeping the cameras and grasping behavior consistent while recording an initial task, then adding different grasp locations, techniques and camera positions after the robot grasps reliably. The DAgger study explains a related sequential difficulty: a learner's action affects what it observes next. Because later observations depend on earlier actions, a mistaken action can change what the policy sees next. The paper proposes an iterative learning approach for this setting, subject to its research assumptions. It does not give a general guarantee for every physical robot.

What a separate test can show

LeRobot's base rollout runs the trained policy autonomously without recording, which its tutorial presents as a quick evaluation. This gives the controller a new run beyond the recorded demonstrations. The tutorial documents the procedure, not an independently verified success rate for every setup.

The DROID authors describe a diverse robot manipulation dataset with 76k demonstration trajectories. In their experiments, they train policies across six tasks in four locations to study performance and robustness. Those are the authors' specified experiments, not proof that a trained policy will work in any home or on any robot.

Sources and references