SAR images are different from the optical images in terms of image properties with the values of scattering instead of reflectance. This makes SAR images difficult to apply the traditional object detection methodologies. In recent years, deep learning models are frequently used in segmentation and object detection purposes. In this study, we have investigated the potential of U-Net models for building detection from SAR and optical image fusion. The datasets used are Sentinel 1 SAR and Sentinel-2 multispectral images, provided from ‘SpaceNet 6 Multi Sensor All-Weather Mapping’ challenge. These images cover an area of 120 km² in Rotterdam, the Netherlands. As training datasets 20 pieces of 900 by 900 pixel sized HV polarized and optical image patches have been used together. The calculated loss value is 0.4 and the accuracy is 81%.