360° Panorama Stitching Method with Depth Information: Enhancing Image Quality and Stitching Accuracy

Abstract. Panoramic images, prized for their capacity to capture a comprehensive 360-degree field of view, find extensive applications across various domains. However, the challenge arises when attempting to seamlessly stitch together images captured by multiple cameras onto a spherical surface, particularly in cases where the cameras lack concentric alignment. To mitigate this misalignment issue, our paper introduces a novel method that integrates depth information into panoramic image stitching, aiming to enhance registration accuracy and seamline detection. The proposed method encompasses two pivotal steps. Firstly, depth information is employed to rectify the image’s placement on the panoramic sphere, facilitating a two-dimensional registration process. This initial step targets the elimination of misalignment problems between images while preserving image clarity. Subsequently, depth information is seamlessly integrated into the smoothing term of the Markov random field energy function to guide seamline detection. Leveraging depth information aids in circumventing foreground obstacles and directs the search through spatially smooth areas, thereby reducing the likelihood of misalignment issues. Experimental results substantiate the effectiveness of employing depth information for image correction on the spherical surface, especially in scenarios where cameras approximate concentric alignment. Furthermore, the integration of depth information into the stitching network construction markedly diminishes misalignment, leading to a notable improvement in panoramic image quality. This signifies a crucial advancement in the domain of panoramic image stitching from an academic perspective.