NeRF-Editing:
Geometry Editing of Neural Radiance Fields

 

Yu-Jie Yuan1,2*     Yang-Tian Sun1,2*     Yu-Kun Lai3     Yuewen Ma4     Rongfei Jia4     Lin Gao1,2†

 

* Authors contributed equally
† Corresponding author

 

 

1 Institute of Computing Technology, Chinese Academy of Sciences
2 University of Chinese Academy of Sciences
3 Cardiff University
4Alibaba Group

 

 

 

 

 

 

Figure 1: The pipeline of our NeRF editing framework. The user edits the reconstructed mesh and a continuous deformation field is built to bend the rays accordingly.

 

 

 

Abstract

 

Implicit neural rendering, especially Neural Radiance Field (NeRF), has shown great potential in novel view synthesis of a scene. However, current NeRF-based methods cannot enable users to perform user-controlled shape deformation in the scene. While existing works have proposed some approaches to modify the radiance field according to the user's constraints, the modification is limited to color editing or object translation and rotation. In this paper, we propose a method that allows users to perform controllable shape deformation on the implicit representation of the scene, and synthesizes the novel view images of the edited scene without re-training the network. Specifically, we establish a correspondence between the extracted explicit mesh representation and the implicit neural representation of the target scene. Users can first utilize well-developed mesh-based deformation methods to deform the mesh representation of the scene. Our method then utilizes user edits from the mesh representation to bend the camera rays by introducing a tetrahedra mesh as a proxy, obtaining the rendering results of the edited scene. Extensive experiments demonstrate that our framework can achieve ideal editing results not only on synthetic data, but also on real scenes captured by users.

 

 


 

 

 

Paper

 

[ArXiv Preprint]

[Supplementary Material]

 

Code

 

Comming Soon

 

 

 

 

 

 

 

 

 

 

 

Video

 

 

 

 

 

 

 

BibTex

 

@inproceedings{Yuan22NeRFEditing,
    title={NeRF-Editing: Geometry Editing of Neural Radiance Fields},
    author={Yuan, Yu-Jie and Sun, Yang-Tian and Lai, Yu-Kun and Ma, Yuewen and Jia, Rongfei and Gao, Lin},
    booktitle={Computer Vision and Pattern Recognition (CVPR)},
    year = {2022}
}

 

 


Last updated on March, 2022.