Accepted by Siggraph Asia 2019
Figure: Our deep generative neural network, SDMNET, produces structured meshes composed of deformable parts. Part structures and geometries are jointly encoded into a latent space by an autoencoder, enabling quality 3D shape generation. We show shape interpolation results exhibiting flexible structure and fine geometric details. This is achieved by linearly interpolating airplane and chair latent codes and then reconstruction from the inbetween codes.
Abstract
We introduce SDMNET, a deep generative neural network which produces structured deformable meshes. Specifically, the network is trained to generate a spatial arrangement of closed, deformable mesh parts, which respect the global part structure of a shape collection, e.g., chairs, airplanes, etc. Our key observation is that while the overall structure of a 3D shape can be complex, the shape can usually be decomposed into a set of parts, each homeomorphic to a box, and the finerscale geometry of the part can be recovered by deforming the box. The architecture of SDMNET is that of a twolevel variational autoencoder (VAE). At the part level, a PartVAE learns a deformable model of part geometries. At the structural level, we train a Structured Parts VAE (SPVAE), which jointly learns the part structure of a shape collection and the part geometries, ensuring a coherence between global shape structure and surface details. Through extensive experiments and comparisons with the stateoftheart deep generative models of shapes, we demonstrate the superiority of SDMNET in generating meshes with visual quality, flexible topology, and meaningful structures, which benefit shape interpolation and other subsequently modeling tasks.
Paper
SDMNET: Deep Generative Network for Structured Deformable Mesh
Code
Data
Methodology
Figure: Encoding of each shape part, including its structure and geometry information. The former includes both the support and symmetry information, and the latter describes the deformation of the bounding box, compactly represented as the latent vector of PartVAE.
Figure: Architecture of PartVAE for encoding the geometry details of a part represented as the deformation of the associated template box. V is the number of vertices in the template box. N(0, I) is the Gaussian distribution with 0 mean and identity covariance.
Figure: Illustration of support and symmetry relations between parts of an airplane model. The support relations, detected by the approach in [Huang et al. 2015b], turn an undirected adjacency graph (Middle) to a directed support graph (Right).
Figure: Architecture of Structured Parts VAE (SPVAE). The detailed geometry is encoded by PartVAE. The support and symmetryinduced structure and the associated latent code of PartVAE are encoded by the Structured Parts VAE.
Shape Interpolation
Figure: Shape interpolation of cups and chairs with different topologies using our method. Interpolating cars with different geometry using our method. The first and last columns are the input shapes for interpolation.
Shape Generation
Figure: Random generation of monitor and chair shapes using our method, where 990 the training data is from ModelNet [Wu et al. 2015].
Shape Editing
Figure: Our generated Structure Deformable Mesh is directly editable. We show an example editing sequence. (a) is the decoded shape of our method. After applying the deletion operation (b), we obtain a chair without armrests (c). Dragging a leg of the chair makes all four legs longer due to the equal length constraint (d). Finally, we deform the back of the chair to obtain (e).
Additional Materials

BibTex
Last updated on July, 2019. 