Multi-Sensor Spatial Data Fusion for Road Condition Monitoring Digital Twins

Abstract. Pavement Management Systems (PMS) are essential for evaluating and maintaining transportation infrastructure; however, conventional monitoring methods are often labour-intensive, costly, and inaccurate. The growing need for reliable. timely pavement condition data has driven the development of automated, data-driven approaches. This study presents a low-cost and scalable framework for pavement condition monitoring that integrates multimodal sensing with a digital twin (DT) environment. Smartphones equipped with inertial measurement unit (IMU) sensors, GPS, and cameras are used to collect synchronized vibration and visual data during normal driving conditions. Vibration signals are analysed to detect anomalies associated with pavement surface irregularities, while video data are processed using a deep learning-based object detection model to identify surface distress. A late fusion approach combines the outputs from both modalities to improve detection reliability and provide comprehensive condition assessment. The system enables spatial mapping of detected distresses and supports real-time visualization through a web-based DT dashboard. Results demonstrate that multimodal sensing compensates for the limitations of individual sensors, enhancing both detection accuracy and robustness. The proposed framework offers a practical solution for efficient pavement monitoring. It supports data-driven decision-making for proactive infrastructure management, with potential for future expansion through crowdsourced data and additional sensing technologies.