Aligning Policies with a Shape Teacher
Patch similarity between the red patch and other patches after the ViT of the policy. Green denotes the second patch.
Description of the method. Per-object PointMAE alignment target computation (left) with the diffusion policy (right).
Success rate over training epochs. Point-MAE alignment (pink) has a faster convergence, both for alignment with a pre-trained policy, and with a from scratch policy.
Abstract
3D-aware representations can improve robotic manipulation by providing geometric information that is robust to changes in viewpoint, object position, and occlusion. However, using 3D inputs at deployment requires additional sensing and processing, making image-based policies more practical. We ask whether an image-based policy can acquire 3D shape information during training while retaining a purely image-based interface at deployment. We propose shape-teacher alignment, a training-time mechanism that uses a pretrained 3D shape representation as an alignment target for an image-based policy encoder. Specifically, we use Point-MAE features from individual objects as targets for corresponding image patches, transferring geometric structure to the policy without changing its deployment inputs or architecture. On LIBERO, shape-teacher alignment produces image representations that are more structured around object shape and corresponding parts, while substantially accelerating policy convergence and improving success rates. For pretrained policies, convergence is accelerated by 4.7x and 5.6x on the Object and Spatial suites, respectively, without requiring the shape teacher or depth input at deployment.
BibTeX
@article{AlignShapeTeacher2026,
title={Aligning policies with a shape teacher},
author={Madelon Bernardy, Mohammad Mahdi Derakhshani, David M Knigge, Riccardo Valperga, Artem Moskalev},
journal={NeurIPS 2nd Embodied Spatial Reasoning workshop},
year={2026},
url={https://miepjemadelon.github.io/aligningpolicieswithashapeteacher/}
}