HSDF (NeurIPS 2022)

HSDF: Hybrid Sign and Distance Field for Modeling Surfaces with Arbitrary Topologies

Li Wang^1,2 Jie Yang^1,2 Weikai Chen³ Xiaoxu Meng³ Bo Yang³ Jintao Li^1,2 Lin Gao^1,2*

Figure 1: We reconstruct three groups of representative objects with both open and closed surfaces using NDF(left) and our proposed HSDF (right). Compared with the SOTA NDF, our method achieves higher surface reconstruction quality and more consistent surface normals. All the results are reconstructed with an equivalent resolution.

Neural implicit function based on signed distance field (SDF) has achieved impressive progress in reconstructing 3D models with high fidelity. However, such approaches can only represent closed surfaces. Recent works based on unsigned distance function (UDF) are proposed to handle both watertight and open surfaces. Nonetheless, as UDF is signless, its direct output is limited to the point cloud, which imposes an additional challenge on extracting high-quality meshes from discrete points. To address this challenge, we present a novel neural implicit representation coded HSDF, which is a hybrid of signed and unsigned distance fields. In particular, HSDF is able to represent arbitrary topologies containing both closed and open surfaces while being compatible with existing iso-surface extraction techniques for easy field-to-mesh conversion. In addition to predicting a UDF, we propose to learn an additional sign field. Unlike traditional SDF, HSDF is able to locate the surface of interest before level surface extraction by generating surface points following NDF. We are then able to obtain open surfaces via an adaptive meshing approach that only instantiates regions containing surfaces into a polygon mesh. We also propose HSDF-Net, a dedicated learning framework that factorizes the learning of HSDF into two easier sub-problems. Experiments and evaluations show that HSDF outperforms the state-of-the-art techniques both qualitatively and quantitatively.

Figure 2: HSDF-Net architecture. The input sparse point cloud is voxelized and encoded in a multi-scale manner into a shape code. Next, the distance predictor takes the shape code and query point p as input to predict an unsigned distance Dis(p). The sign predictor takes the same input and predicts a signed value Sign(p). The field fusion module is proposed to fuse Dis(p) and Sign(p). The rightmost lamp example is reconstructed from test data using the proposed adaptive masked Marching Cubes algorithm.

Figure 3: 2D illustration of sign and distance fusion. Assume the green line is the target surface. Row 1: By multiplying sign and distance point-wise, we can obtain an original HSDF, where some points (e.g. A and B) may be wrongly signed. Row 2: By using gradients of sign function (i.e. yellow vectors) and gradients of distance function (i.e. brown vectors) to optimize the original fused HSDF, HSDF of points A, B can be effectively rectified (i.e. green line).

Figure 4: A 2D illustration of mesh extraction on our fused HSDF. Green contour indicates an open shape in 2D. We push Marching Cubes grid points like A and B to their closest points on the surface, i.e., A′ and B′, respectively, using the gradients of distance function (i.e. black vectors). All the boxes in red enclosing surfaces form a mask for Marching Cubes to extract the complex shapes into meshes.

Figure 5: More results visulaization. The inputs are point clouds and the outputs are reconstructed meshes using HSDF. Dark brown area represents the back-faces. Our method can reconstruct heigh-feidelity meshes with open and closed surfaces. The surface normal is continuous and consistant to the ground truth. Our method also preserves thin structures like the chair legs.