We propose a pipeline for the detection as well as modeling of individual buildings based on multi-source images. It allows to consistently reconstruct whole buildings at Level of Detail 3 (LoD3): the roof from airborne images and the facades including elements such as windows and doors mainly from terrestrial images. We employ a parametrized top-down model – the “shell model” – with the roof as well as the facades semantically and geometrically integrated. This generative model fosters stability for building detection by enabling the use of multi-source data and offers flexibility in modeling by means of a fully CAD-compatible integration of building components. Experiments performed on imagery from different terrestrial and airborne (Unmanned Aerial Vehicle – UAV) cameras demonstrate the potential of the approach.