Method for spherical camera to 3D LiDAR calibration and synchronization with example on Insta360 X4 and LiVOX MID 360

Abstract. This paper introduces a method for 360 camera with 3D LiDAR calibration and synchronization with example on Insta360 X4 and LiVOX MID 360. Both data streams (camera and LiDAR) are recorded separately to reach interoperability, robustness, full resolution, and maximal FPS (Frames Per Second). The novelty is based on LED circle strip illuminating timestamp, thus this information is recorded by camera. The timestamp is presented using gray code utilizing individually addressed LED strips. The data signal for LED is prepared by micro-controller. Microcontroller (ESP-8285) communicates using USB. We incorporated ResNet-18 based binary classifier to classify LEDs. Our method can efficiently assign timestamps with a resolution of 100 ms. To fulfill the process of 3D point cloud colorization, we implemented camera to LiDAR calibration procedure requiring manually assigning 5 pairs of image to 3D point correspondences. Our method is independent from hardware, thus it can be used also for affordable one making it more accessible and easy to use.