The mapping of slum areas is an important task when considering the necessity for an inclusive, safe and resilient cities. While many methods exist in this regard, the use of machine learning and more specifically deep learning has gained traction in recent years. In this paper, we present a systematic comparison of existing deep learning architectures and backbones. The experiments in the paper investigate the question of which architecture and backbone combination and which configuration of dataset preparation is best for use in slum mapping. In another experiment we implemented the trained model to predict slums in existing open data. The experiments in the paper used public open data provided by Helber et al. (2018). Results show that FPN with vgg16 backbone showed the most potential in this particular application. The results of the semantic segmentation also shows promise, although the discrepancy in slum characteristic still hinders a proper generalization of its use.