CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphism

Abstract

While neural representations for static 3D shapes are widely studied, representations for deformable surfaces are limited to be template-dependent or lack efficiency. We introduce Canonical Deformation Coordinate Space (CaDeX), a unified representation of both shape and nonrigid motion. Our key insight is the factorization of the deformation between frames by continuous bijective canonical maps (homeomorphisms) and their inverses that go through a learned canonical shape. Our novel deformation representation and its implementation are simple, efficient, and guarantee cycle consistency, topology preservation, and, if needed, volume conservation. Our modelling of the learned canonical shapes provides a flexible and stable space for shape prior learning. We demonstrate state-of-the-art performance in modelling a wide range of deformable geometries: human bodies, animal bodies, and articulated objects.

Introduction and Method Overview Video

Watch this video here on Bilibili in China

Results Overview Video

Watch this video here on Bilibili in China

Method Overview

We model a deformable object as a canonical shape U (marked in the green box) and the consistent deformation functions (yellow arrows) between deformed shapes.

We factorize the deformation between deformed surfaces as learnable continuous invertible canonical maps (green bidirectional arrows) through a learnable canonical shape. (If the topology is not changed)

Given sparse point cloud observation or depth video observation, our architecture reconstructs the deformed surface for every frame.

Results

Input Observation	Reconstruction	Learned Canonical Shape

@inproceedings{Lei2022CaDeX,
title={CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphism},
author={Lei, Jiahui and Daniilidis, Kostas},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
url={https://cis.upenn.edu/~leijh/projects/cadex},
year={2022}
}