SDM-NET: Deep Generative Network for Structured Deformable Mesh


Lin Gao1          Jie Yang1          Tong Wu1          Yu-Jie Yuan1


Hongbo Fu2       Yu-Kun Lai3       Hao(Richard) Zhang4



1Institute of Computing Technology, Chinese Academy of Sciences


2City University of Hong Kong       3Cardiff University       4Simon Fraser University




Accepted by Siggraph Asia 2019





Figure: Our deep generative neural network, SDM-NET, produces structured meshes composed of deformable parts. Part structures and geometries are jointly encoded into a latent space by an autoencoder, enabling quality 3D shape generation. We show shape interpolation results exhibiting flexible structure and fine geometric details. This is achieved by linearly interpolating airplane and chair latent codes and then reconstruction from the in-between codes.






We introduce SDM-NET, a deep generative neural network which produces structured deformable meshes. Specifically, the network is trained to generate a spatial arrangement of closed, deformable mesh parts, which respect the global part structure of a shape collection, e.g., chairs, airplanes, etc. Our key observation is that while the overall structure of a 3D shape can be complex, the shape can usually be decomposed into a set of parts, each homeomorphic to a box, and the finer-scale geometry of the part can be recovered by deforming the box. The architecture of SDM-NET is that of a two-level variational autoencoder (VAE). At the part level, a PartVAE learns a deformable model of part geometries. At the structural level, we train a Structured Parts VAE (SP-VAE), which jointly learns the part structure of a shape collection and the part geometries, ensuring a coherence between global shape structure and surface details. Through extensive experiments and comparisons with the state-of-the-art deep generative models of shapes, we demonstrate the superiority of SDM-NET in generating meshes with visual quality, flexible topology, and meaningful structures, which benefit shape interpolation and other subsequently modeling tasks.




paper thumbnail



SDM-NET: Deep Generative Network for Structured Deformable Mesh

[ArXiv Preprint]


















Encoding of Shape Part



Figure: Encoding of each shape part, including its structure and geometry information. The former includes both the support and symmetry information, and the latter describes the deformation of the bounding box, compactly represented as the latent vector of PartVAE.




PartVAE for Encoding Part Geometry



Figure: Architecture of PartVAE for encoding the geometry details of a part represented as the deformation of the associated template box. V is the number of vertices in the template box. N(0, I) is the Gaussian distribution with 0 mean and identity covariance.




Supporting Structure Analysis



Figure: Illustration of support and symmetry relations between parts of an airplane model. The support relations, detected by the approach in [Huang et al. 2015b], turn an undirected adjacency graph (Middle) to a directed support graph (Right).




SP-VAE for Structured Deformable Mesh Encoding



Figure: Architecture of Structured Parts VAE (SP-VAE). The detailed geometry is encoded by PartVAE. The support- and symmetry-induced structure and the associated latent code of PartVAE are encoded by the Structured Parts VAE.








Shape Interpolation




Figure: Shape interpolation of cups and chairs with different topologies using our method. Interpolating cars with different geometry using our method. The first and last columns are the input shapes for interpolation.




Shape Generation




Figure: Random generation of monitor and chair shapes using our method, where 990 the training data is from ModelNet [Wu et al. 2015].




Shape Editing




Figure: Our generated Structure Deformable Mesh is directly editable. We show an example editing sequence. (a) is the decoded shape of our method. After applying the deletion operation (b), we obtain a chair without armrests (c). Dragging a leg of the chair makes all four legs longer due to the equal length constraint (d). Finally, we deform the back of the chair to obtain (e).




Additional Materials










@article {gaosdmnet2019,
    author = {Gao, Lin and Yang, Jie and Wu, Tong and Yuan, Yu-Jie and Fu, Hongbo and Lai, Yu-Kun and Zhang, Hao(Richard)},
    title = {{SDM-NET}: Deep Generative Network for Structured Deformable Mesh},
    journal = {ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH Asia 2019)},
    year = {2019},
    volume = 38,
    pages = {243:1--243:15},
    number = 6



Last updated on July, 2019.