Automatic extraction of building roofs, street and vegetation are a prerequisite for many GIS (Geographic Information System) applications, such as urban planning and 3D building reconstruction. Nowadays with advances in image processing and image matching technique by using feature base and template base image matching technique together dense point clouds are available. Point clouds classification is an important step in automatic features extraction. Therefore, in this study, the classification of point clouds based on features color and shape are implemented. <br><br> We use two images by proper overlap getting by Ultracam-x camera in this study. The images are from Yasouj in IRAN. It is semi-urban area by building with different height. Our goal is classification buildings and vegetation in these points. <br><br> In this article, an algorithm is developed based on the color characteristics of the point’s cloud, using an appropriate DEM (Digital Elevation Model) and points clustering method. So that, firstly, trees and high vegetation are classified by using the point’s color characteristics and vegetation index. Then, bare earth DEM is used to separate ground and non-ground points. <br><br> Non-ground points are then divided into clusters based on height and local neighborhood. One or more clusters are initialized based on the maximum height of the points and then each cluster is extended by applying height and neighborhood constraints. Finally, planar roof segments are extracted from each cluster of points following a region-growing technique.