Skills and applications
Touch2Robot brings a robot hand’s predicted touch into human data collection
Touch2Robot visualizes reconstructed contacts for a target robot hand; its authors report higher replay completion, faster collection and better downstream policy results across four physical tasks.
In the authors’ four real-world tasks, showing people reconstructed target-hand contacts improved both real-robot replay completion and the efficiency of collecting successful demonstrations compared with visual-only feedback. This is an author-reported result from the Touch2Robot preprint, not an independent confirmation.
Touch2Robot records human hand motion, tactile-glove measurements and object motion during manipulation. Those recordings guide object-specific reinforcement-learning policies that reproduce the demonstrated object motion while favoring contacts consistent with the recorded human touch. The resulting behaviors are distilled into a unified real-time retargeter, which maps incoming human observations and object geometry to configurations for the robot hand.
While a person collects a demonstration, the system synchronizes its predicted robot-hand configuration with the tracked object pose in simulation. It then reconstructs and displays the robot-object contacts, allowing the demonstrator to adjust later interactions to the target hand rather than relying only on the appearance of the motion.
Across the four physical tasks, the authors report that average real-robot replay completion rose from 37.9% with visual-only feedback to 72.1% with Touch2Robot. Collection time per replay-successful demonstration fell from 58.6s to 18.2s.
The reconstructed contacts reached 44.2% F1 when compared with tactile measurements from the real robot. Policies trained on Touch2Robot demonstrations also improved downstream Diffusion Policy performance by 29.1 percentage points over policies trained with visual-only feedback.
The available evidence is limited to one author preprint and its four reported tasks. The excerpt does not provide task-by-task outcomes, trial counts or uncertainty estimates, so the results support a focused conclusion: contact feedback can improve demonstrations in this tested setup, but broader effectiveness across other tasks and robot hands remains to be established.