NeRF-based RGB-D Images Generation in Robotics – Experimental Study

Abstract

Multiple learning-based algorithms in robotics require collecting RGB-D images of the scene from various viewpoints. These procedures are time-consuming, so many methods are trained using synthetic images. Recently, a Neural Radiance Fields (NeRF) model of the scene was proposed. Moreover, recent methods show that this model can be trained in minutes. This opens the possible applications in robotics for training the systems to reconstruct scenes, grasp objects or estimate their 3D poses using RGB-D images generated from a small number of input images. In this paper, we verify the quality of RGB-D images generated by the Instant Neural Graphics Primitives implementation of NeRF. We compare the obtained results from the Instant NeRF with the ground-truth RGB-D images obtained from the Kinect Azure and images generated from the point cloud model of the scene. The results show that the difference between generated RGB-D images and ground truth images is small, especially near the object.