Embodied AI models
How ThinkWorld Converts Unlabeled Video Into Action Feedback
The framework grounds motion-token chains with labeled demonstrations, then extracts a training signal by comparing imagined futures with outcomes visible in human and robot videos.
Videos without action labels show how a scene changes, but they do not specify the commands behind the movement. ThinkWorld uses those visible state transitions as supervision within a representation first grounded in labeled robot demonstrations.
The framework expresses each proposed action as a Chain-of-Dynamics, or CoD: a sequence of discrete motion tokens representing intermediate changes from the current world to a future state. After initializing these chains with action-labeled demonstrations, ThinkWorld predicts the future associated with each CoD and compares that prediction with the outcome visible in a video. ThinkWorld refines robot action reasoning from videos without action annotations by reinforcing Chains-of-Dynamics whose predicted futures better match the videos’ observed outcomes.
The authors say this allows human and robot experience to refine the same reasoning space. They report effective human-to-robot transfer and strong generalization to unseen tasks and environments in simulation and on real robots.