SDM-NET: Deep Generative Network for Structured Deformable Mesh

 

Lin Gao1          Jie Yang1          Tong Wu1          Yu-Jie Yuan1

 

Hongbo Fu2       Yu-Kun Lai3       Hao(Richard) Zhang4

 

 

1Institute of Computing Technology, Chinese Academy of Sciences

 

2City University of Hong Kong       3Cardiff University       4Simon Fraser University

 

 

 

Conditionally Accepted by Siggraph Asia 2019

 

 

 

 

Figure: Our deep generative neural network, SDM-NET, produces structured meshes composed of deformable parts. Part structures and geometries are jointly encoded into a latent space by an autoencoder, enabling quality 3D shape generation. We show shape interpolation results exhibiting flexible structure and fine geometric details. This is achieved by linearly interpolating airplane and chair latent codes and then reconstruction from the in-between codes.

 

 

 

Abstract

 

We introduce SDM-NET, a deep generative neural network which produces structured deformable meshes. Specifically, the network is trained to generate a spatial arrangement of closed, deformable mesh parts, which respect the global part structure of a shape collection, e.g., chairs, airplanes, etc. Our key observation is that while the overall structure of a 3D shape can be complex, the shape can usually be decomposed into a set of parts, each homeomorphic to a box, and the finer-scale geometry of the part can be recovered by deforming the box. The architecture of SDM-NET is that of a two-level variational autoencoder (VAE). At the part level, a PartVAE learns a deformable model of part geometries. At the structural level, we train a Structured Parts VAE (SP-VAE), which jointly learns the part structure of a shape collection and the part geometries, ensuring a coherence between global shape structure and surface details. Through extensive experiments and comparisons with the state-of-the-art deep generative models of shapes, we demonstrate the superiority of SDM-NET in generating meshes with visual quality, flexible topology, and meaningful structures, which benefit shape interpolation and other subsequently modeling tasks.

 

 

 

paper thumbnail

Paper

 

SDM-NET: Deep Generative Network for Structured Deformable Mesh

[ArXiv Preprint]

 

Code

 

Comming Soon

 

Video

 

Comming Soon

 

 

 

 

 

 

 

 

Methodology

Encoding of Shape Part

 

 

Figure: Encoding of each shape part, including its structure and geometry information. The former includes both the support and symmetry information, and the latter describes the deformation of the bounding box, compactly represented as the latent vector of PartVAE.

 

 

 

PartVAE for Encoding Part Geometry

 

 

Figure: Architecture of PartVAE for encoding the geometry details of a part represented as the deformation of the associated template box. V is the number of vertices in the template box. N(0, I) is the Gaussian distribution with 0 mean and identity covariance.

 

 

 

Supporting Structure Analysis

 

 

Figure: Illustration of support and symmetry relations between parts of an airplane model. The support relations, detected by the approach in [Huang et al. 2015b], turn an undirected adjacency graph (Middle) to a directed support graph (Right).

 

 

 

SP-VAE for Structured Deformable Mesh Encoding

 

 

Figure: Architecture of Structured Parts VAE (SP-VAE). The detailed geometry is encoded by PartVAE. The support- and symmetry-induced structure and the associated latent code of PartVAE are encoded by the Structured Parts VAE.

 

 

 

 

 

 

 

Shape Interpolation

 

 

 

Figure: Shape interpolation of cups and chairs with different topologies using our method. Interpolating cars with different geometry using our method. The first and last columns are the input shapes for interpolation.

 

 

 

Shape Generation

 

 

 

Figure: Random generation of monitor and chair shapes using our method, where 990 the training data is from ModelNet [Wu et al. 2015].

 

 

 

Shape Editing

 

 

 

Figure: Our generated Structure Deformable Mesh is directly editable. We show an example editing sequence. (a) is the decoded shape of our method. After applying the deletion operation (b), we obtain a chair without armrests (c). Dragging a leg of the chair makes all four legs longer due to the equal length constraint (d). Finally, we deform the back of the chair to obtain (e).

 

 

 

Additional Materials

Video

 

 

 

 

 

 

BibTex

 

@article {gaosdmnet2019,
    author = {Gao, Lin and Yang, Jie and Wu, Tong and Yuan, Yu-Jie and Fu, Hongbo and Lai, Yu-Kun and Zhang, Hao(Richard)},
    title = {{SDM-NET}: Deep Generative Network for Structured Deformable Mesh},
    journal = {ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH Asia 2019)},
    year = {2019},
    volume = 38,
    pages = {To appear},
    number = 6
}

 

 


Last updated on July, 2019.