In this paper, we present a novel approach for segmenting planar regions in Digital Surface Models (DSMs) by adapting the Segment Anything Model (SAM), an open-source framework. Our approach specifically tailors SAM to recognize planar regions within given building footprints, employing the Low-Rank Adaptation (LoRA) technique. This adaptation benefits from a detailed and realistic synthetic dataset, coupled with a novel labeling strategy for planar regions in our ground truth, enhancing the model’s effectiveness and reproducibility. Unlike traditional plane detection techniques, our method consistently and accurately identifies equivalent planar regions across identical DSM inputs. Following the segmentation phase, we introduced a novel plane fitting algorithm to determine the parameters for each planar region. This enables us to refine the edges of these areas and utilize the resulting plane equations to construct precise, watertight 3D models of buildings. Despite its training on synthetic data, our model exhibits remarkable performance on both synthetic and real-world datasets, exemplified by its application to the Zurich dataset.