On the Importance of the RGB-D Sensor Model in the CNN-based Robotic Perception

Abstract

Mobile and manipulation robots operating indoors use RGB-D cameras as the environment perception sensors. To process data from RGB and depth cameras neural networks are applied. These neural-based systems are trained using synthetic datasets due to the difficulties of obtaining ground truth data on real robots. As a result, the neural model used on the real robot does not produce satisfactory performance due to the differences between the images used during training and the inference. In this paper, we show the importance of depth sensor modeling while training the neural network on a synthetic dataset. We show that the obtained neural model can be used on the real robot and process the data from the real RGB-D camera.