UNITAS Joins Robot Motion and Scene Dynamics in a Shared 3D Frame

Robotic manipulation takes place in physical three-dimensional space, while an image is a viewpoint-dependent projection whose pixel distances do not directly encode physical distances. The authors of the UNITAS preprint address that mismatch by placing observations, actions and predicted scene dynamics in a shared metric 3D frame within each interaction.

UNITAS represents motion as 3D point trajectories called action flow and predicts scene change as trajectory-conditioned 3D point displacements called scene flow, with both expressed in a shared metric frame for each interaction. Action flow uses the same representation for human hands and robot grippers across different robot embodiments. The resulting interface supports both direct execution of an action and prediction of how that action will move points in the surrounding scene.

To ground visual tokens in the workspace, the model uses world-aligned 3D positional embeddings, whether or not depth is supplied as an input. A physical-time trajectory tokenizer then compresses each point trajectory into one token anchored at the point’s current 3D position. These components let the model connect where an actor is moving with how the observed world is expected to change.

UNITAS has 1.7B parameters. In evaluations reported by the preprint authors, it produced the best action-conditioned scene prediction on RoboTwin among the compared methods, with displacement errors up to 49% lower than PointWorld. They also report state-of-the-art manipulation success, including 99.8% on LIBERO and an average of 85% across real-world tasks. Because the available evidence is a single author preprint abstract, those benchmark rankings and physical-task results should be treated as the authors’ claims rather than independent confirmation.