Overview

Generative models have shown great promise as trajectory planners, given their affinity to modeling complex distributions and guidable inference process. Previous works have successfully applied these in the context of robotic manipulation but perform poorly when the required solution does not exist as a complete trajectory within the training set. We identify that this is a result of being unable to plan via stitching, and subsequently address the architectural and dataset choices needed to remedy this. On top of this, we propose a novel addition to the training and inference procedures to both stabilize and enhance these capabilities. We demonstrate the efficacy of our approach by generating plans with out of distribution boundary conditions and performing obstacle avoidance on the Franka Panda in simulation and on real hardware. In both of these tasks our method performs significantly better than the baselines and is able to avoid obstacles up to four times as large.

Flow Planner

We propose three improvements to fix stitching. Firstly, we identify that a model architecture with what we call a “local receptive field” is required to avoid exposing global information. Our experiments show that without this, the model is strongly biased towards merely selecting whole trajectories that already exist. A local receptive field instead drives the model to only optimize for local consistency and thus allows for a more flexible global structure.

Secondly, we found that action noise addition was crucial to allowing the model to “jump” between existing clips at points where they overlap. This was uniquely effective when compared to other dataset augmentation schemes, which we hypothesise is due to the breaking of the correlation between the joint states.

Lastly, to both prevent mode collapse, and reduce dynamics inconsistencies, we apply a technique we call trajectory splitting, which differs based on when it is applied. At training time, this consists of randomly mixing half-length trajectories into the training batch. At inference time, this consists of partially renoising the trajectory after the first inference loop, splitting it in two, and then denoising each half separately.

Results

Using the combination of these techniques we create a method called Flow Planner and compare it to previous approaches in two ways. Firstly, we demonstrate its stitching capabilities by successfully producing plans with out of distribution boundary conditions. We define the stitching error as the distance between the last generated state and the boundary condition on each end of the trajectory. Tables 1 and 2 show we are able to produce OOD trajectories more consistently than previous methods and also highlight the importance of each of our modifications.

Secondly, we apply it to an obstacle avoidance problem where no single trajectory is a complete solution, hence also requiring stitching. Flow planner is able to both plan more successfully, and avoid much larger obstacles than previous methods. As shown in the figure below.

We lastly demonstrate our methods generalisability by deploying it onto real hardware.

In conclusion, we present a novel, flow matching based planner for robotic manipulation that, through superior stitching abilities, is able to plan more flexibly, and robustly than previous methods.