MULTIMODAL DATA FUSION FOR EFFECTIVE SURVEILLANCE OF CRITICAL INFRASTRUCTURE

Abstract. Monitoring critical infrastructures, especially those that are covering wide-zones, is of fundamental importance and priority for modern surveillance systems. The concurrent exploitation of multisensor systems, can offer additional capabilities, on day and night acquisitions and different environmental/illumination conditions. Towards this direction, we have designed a multi-sensor system based on thermal, shortwave infrared and hyperspectral video sensors. Based on advanced registration, dynamic background modelling and data association techniques, possible moving targets are detected on the thermal and shortwave infrared modalities. In order to avoid the computational intensive co-registration with the hyperspectral video streams, the detected targets are projected through a local coordinate system on the hypercube image plane. The final detected and verified targets are extracted through fusion and data association, based on temporal spectral signatures and target/background statistics. The developed multisensor system for the surveillance of critical infrastructure has been validated for monitoring wide-zones against different conditions showcasing abilities for detecting and tracking moving targets through fog and smoke.