Embodied AI models
FastVLA Adapts the Visual Token Budget for Faster Robot Control
A training-free method ranks image tokens against the robot’s task and changes its compression rate at each control step, with author-reported tests in simulation and on a Franka Research 3.
Vision-language-action models can encode high-resolution, sometimes multi-view images into hundreds of visual tokens at every control step. The FastVLA preprint identifies those tokens as a major source of transformer computation and action-prediction latency.
FastVLA can make robot control more efficient by retaining the visual tokens most relevant to the task at each timestep and varying how many it keeps as attention becomes more or less concentrated. The training-free method scores importance through task-to-vision attention, measures the uniformity of the resulting score distribution with an inverse-Simpson metric, and calibrates its compression rate online using a sliding window. This replaces a fixed token budget that could either preserve unnecessary image information or remove information needed for control.
The authors tested FastVLA with OpenVLA, OpenVLA-OFT, and CogACT in the LIBERO and SIMPLER simulation environments. They report reducing language-model-backbone FLOPs by 35–59% and action latency by 9–32%, equivalent to a 1.09×–1.46× speedup, while maintaining task success. They also describe its efficiency-performance trade-off as comparable to or better than the plug-and-play token compression, pruning, and reuse baselines evaluated in the study.
In experiments on a physical Franka Research 3 robot, the researchers report a 20% reduction in action-prediction latency, or a 1.25× speedup, at comparable success. These author-reported simulation and physical-robot results come from the FastVLA preprint and illustrate the reported efficiency-performance trade-off of task-aware, step-adaptive compression.