Improving Robotic Generalist Policies via Flow Reversal Steering

TL;DR

Flow Reversal Steering (FRS) improves generalist robot policies by reversing actions to find latent noises and mapping them to better action modes.

Problem

Generalist robot policies learn from diverse data but struggle to invoke appropriate actions for challenging new tasks via direct commands. A method to improve suboptimal actions is needed.

Approach

FRS takes suboptimal actions from flow matching generalists, passes them through the policy in reverse to find latent noises, and maps them to nearby good action modes. It converts coarse semantic guidance from humans or VLMs into concrete robot actions.

Results & Contribution

FRS improves zero-shot control in simulation and real manipulation, achieves up to 95% absolute success rate boosts with under a minute of behavioral cloning training, and bootstraps reinforcement learning to improve on tasks where standard RL fails.