Multi-Head Spatial Planning: An Energy-Based Approach

“Is there a better way to decribe a cube than that of its construction?”

László Tóth, Brutalist(2024)

Absract

Error accumulation has long been an issue in the long-horizon planning tasks, such as robot manipulation, auto-driving, etc., and have been actively studied. However, most of the current methods are either domain-specific and cannot be generalized to the out-of-domain tasks, or augment pretrained planners with traditional search algorithms to compensate the inherited error-accumulation issue of the dynamic model.

This project aims to tackle such problem by providing an energy-based perspective. Specifically, we propose a novel training and inference paradigm to capture multiple dependencies including modality, temporal consistency, and goal-achieving within the planning trajectory by modeling multiple energy landscapes.

We plan to show our paradigm’s superiority on spatial planning tasks with incremental complexities, in aspects of less accumulative errors, better performance and superior generalizability, when compared to other baselines.

Please refer to my Notions for more frequent updates(and self-learning)

Proposal on Compositional Spatial Planning(This FYP)

EBM & Contrastive Divergence(Self-use notes)

Algorithms

Training

Sampling

Some resources for learning EBM

EBM Workshops @ICLR2021

Recent Publications on EBMs