EVALUATING THE ACCURACY OF 3D OBJECT RECONSTRUCTION FROM THERMAL IMAGES

Abstract. Thermal cameras are increasingly used in many photogrammetric and computer vision tasks. Nowadays it is possible to detect and recognize objects in infrared images, to solve such tasks as pedestrian detection (Huckridge et al., 2016), security applications, and autonomous driving (Wenbin, Li et al., 2017). Nevertheless, some tasks that are easily solved in the visible range data are still challenging to achieve in the infrared range. Reconstruction of a 3D object model from infrared images is challenging due to the low contrast of the original infrared image, noise of the sensor, and the absence of feature points on the image. Nevertheless, thermal cameras have their advantages, which make them popular for solving practical problems. Firstly, thermal cameras can be used in degraded environments (smoke, fog, precipitation, low light conditions). Secondly, infrared images can be fused with color images (Gao et al., 2013) to increases the system’s performance.

This paper is focused on the evaluation of accuracy of 3D object reconstruction from thermal images. The evaluation of the accuracy is threefold. Firstly, we train four stereo matching methods (CAE, LF-Net, SURF, and SIFT) on the MVSIR dataset (Knyaz et al., 2017) and our new ThermalPatches dataset. We used two RTX 2080 Ti GPUs and the PyTorch library for the training. Secondly, we evaluate the matching score for the selected methods. Finally, we perform 3D object reconstruction using the SfM (Remondino et al., 2014) approach and matches for each method. We compare the object space accuracy of the resulting surfaces to the ground-truth 3D models generated with a structured light 3D scanner.