Multimodal Fusion Framework for Urban functional zone change detection using Remote Sensing and Social Sensing Data

Abstract. Change detection reveals the shifts in land-use distribution and composition over time, providing valuable insights for urban management and socioeconomic analysis. Previous studies have focused primarily on single-modal data like high-resolution remote sensing (RS) imagery, overlooking the role of social sensing characteristics in determining urban functional zones (UFZs). In this study, we propose a novel multimodal dual-branch change detection (MDB-CD) deep learning framework, for detecting UFZ changes by combining RS imagery and social sensing data. It includes the RS-branch and the mobile positioning (MP) branch. For the RS-branch, RS imagery provides detailed spatial information about urban transformations; for the MP-branch, MP data as a typical social sensing data captures temporal patterns in human mobility, offering insights into functional shifts in urban spaces. By fusing these two complementary modalities, our approach allows for a more nuanced detection of urban functional zone changes. Empirical results in Shenzhen, China demonstrate that the MDB-CD model significantly outperforms a baseline model trained only on imagery, achieving higher overall accuracy (OA) of 0.858 and Kappa coefficients of 0.818 across change detection. Specifically, the model generates an OA matrix of UFZ change detection transitions between 2017 and 2019, revealing 81 distinct transitions in UFZs. Notably, the integration of MP data proved instrumental in improving the model capturing subtle changes that RS imagery alone could not distinguish. An ablation study further highlights the significant accuracy improvements achieved by integrating RS imagery and MP data, emphasizing the value of a multimodal approach for detecting UFZ changes. This work highlights the value of incorporating social sensing data into urban change detection, offering a robust solution for dynamic urban planning and development.