Learning methods
How Diffusion Policies Can Preserve Distinct Action Modes
Immiscible Diffusion Policy targets path mixing during training while leaving the policy architecture and inference procedure unchanged.
The preprint starts from a failure case: its authors report that a diffusion policy can collapse to one action mode even when the demonstrated modes are balanced in the dataset and batches have exact symmetry. Their analysis attributes this behavior partly to independent action-noise pairing, which increases mixing and crossing among diffusion paths. The resulting averaged denoising responses can suppress behavior specific to each mode, particularly in dense, low-dimensional robot action spaces.
A diffusion policy can preserve distinct demonstrated action modes by using label-free action-noise assignment during training to keep noise-to-action routes relatively separate. The authors call their add-on Immiscible Diffusion Policy.
The authors evaluated the method on five simulated and two real-world humanoid manipulation tasks using state, RGB and point-cloud observations. They report that the non-dominant mode’s share increased by 6.0x-14.6x across three two-mode tasks. On both four-mode tasks, the method recovered demonstrated modes that were entirely absent from vanilla policy rollouts, while the authors also report strong task performance.
The authors present the reported recovery of demonstrated modes as evidence that label-free action-noise assignment can help preserve distinct action modes during training.