Skills and applications
Retarget the Contacts, Not the Human Hand Motion
A three-stage method reconstructs missing contact forces in simulation, maps touch sequences across hand designs and learns actuation corrections before hardware deployment.
Contact structure provides a morphology-independent target: preserve which finger regions touch which object locations, and in what order, then use simulated physics refinement and residual policy learning to recover forces and adapt the result to the robot’s actuation.
The authors describe three stages. First, a simulated hand refines human motion-capture recordings into physically consistent trajectories with contact and force annotations. Second, a contact-anchored objective transfers the reconstructed touch pattern to a robot hand without tying that objective to a particular hand morphology. Third, a residual policy learns corrections that restore dynamic feasibility under the robot’s actuation constraints. The paper says one shared residual reinforcement-learning policy can perform this repair across diverse demonstrations rather than requiring task-specific physical engineering.
In the authors’ reconstruction evaluations, success increased from 7.3% to 59.3% across 25,454 single-hand trajectories and from 16.0% to 62.4% across 25 dual-hand tasks. They also report transferring one human dataset to four morphologically distinct robot hands, with a gain of 62.4 percentage points.
The hardware evidence is narrower: the authors report executing four contact-rich bimanual tasks on physical hardware with zero real-robot training data. That demonstration shows the resulting policies were deployed beyond simulation, while the paper’s abstract does not provide task-by-task hardware success rates or comparisons. The central lesson is therefore methodological: retaining the structure of touch, then repairing its physical and actuation feasibility, can provide a common bridge from human recordings to robot hands with different designs.