The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Download
Publications Copernicus
Download
Citation
Articles | Volume XLIII-B2-2022
https://doi.org/10.5194/isprs-archives-XLIII-B2-2022-553-2022
https://doi.org/10.5194/isprs-archives-XLIII-B2-2022-553-2022
30 May 2022
 | 30 May 2022

REINFORCEMENT LEARNING FOR AUTONOMOUS 3D DATA RETRIEVAL USING A MOBILE ROBOT

V. V. Kniaz, V. Mizginov, A. Bordodymov, P. Moshkantsev, and D. Novikov

Keywords: Interior orientation, Structure-from-motion, Deep learning, Reinforcement learning

Abstract. 3D data retrieval is required in various fields such as an industrial monitoring, agriculture, and robotics. Recent advances in photogrammetry and computer vision allowed to perform 3D reconstruction using a set of images captured with uncalibrated camera. Such technique is commonly known as Structure-from-Motion. In this paper, we propose a reinforcement learning framework RL3D for online strong camera configuration planning onboard of a mobile robot. The mobile robot consists of a skid-steered wheeled platform, a single-board computer and an industrial camera. Our aim is developing a model that plans a set of robot location that provide a strong camera configuration. We developed an environment simulator to train our RL3D framework. The simulator was implemented using a 3D model of the indoor scene and includes a model of robot’s dynamics. We trained our framework using the simulator and evaluated it using a virtual and real environments. The results of the evaluation are encouraging and demonstrate that the controller model successfully learns simple camera configurations such as a circle around an object.