Oblique photography technology as an excellent method for 3-D city model construction has brought itself to large-scale recognition and undeniable high social status. Tilt and vertical images with the high overlaps and different visual angles can produce a large number of dense matching point clouds data with spectral information. This paper presents a method of buildings reconstruction with stereo matching dense point clouds from aerial oblique images, which includes segmentation of buildings and reconstruction of building roofs. We summarize the characteristics of stereo matching point clouds from aerial oblique images and outline the problems with existing methods. Then we present the method for segmentation of building roofs, which based on colors and geometrical derivatives such as normal and curvature. Finally, a building reconstruction approach is developed based on the geometrical relationship. The experiment and analysis show that the methods are effective on building reconstruction with stereo matching point clouds from aerial oblique images.