BUILDING EXTRACTION USING MULTI SENSOR SYSTEMS

In this study, the automatic building extraction is aimed using object-based image analysis method with multi sensor system includes LiDAR, digital camera and GPS/IMU. The image processing techniques, segmentation and classification methods were used for automatic object extraction with defined rule set. The proposed method based on object based classification to overcome the limitation of traditional pixel based classification such as confusion of classes. The generated Digital Surface Model (DSM) from LiDAR point cloud was used to separate building and vegetation classes. The morphologic filters were utilized also optimization of mixed classes. In our proposed approach for building extraction, multi-resolution, contrast-difference and chessboard segmentations were applied. The object-based classification method was preferred in classification process with defined fuzzy rules. First, vegetation and ground classes were generated than building regions were derived with using the results of the classification and segmentation. The data set was obtained from the project of “NABUCCO Gas Pipeline Project”. The data set actually was collected for corridor mapping of pipeline which will link the Eastern border of Turkey, to Baumgarten in Austria via Bulgaria, Romania and Hungary. The study area is a suburban neighborhood located in the city of Sivas, Turkey. The Leica ALS60 LiDAR system, DiMAC, Dalsa Area Bayer RGB Charge Coupled (CCD) Camera and GPS and CUS6 IMU system were used for data collection. The additional data sets were generated with point cloud collected by LiDAR and RGB images from digital camera. The rule sets for automatic building extraction were developed in Definiens e-Cognition Developer 8.64 program system. To evaluate the performance of proposed automatic building extraction approach, reference data set was generated with digitizing of extracted building over the orthoimage. The accuracy assessment was performed with completeness and correctness analyses. Based on the completeness and accuracy analysis, the success rates of 83.08% for completeness and 85.51% for correctness were achieved. * Corresponding author. Email address: ynaci@yildiz.edu.tr


INTRODUCTION
The main aim in object extraction is meaningfully organize, group, and properly represent the points, edges, and area of objects (Vosselman et al., 2004).The different terminologies can be used to describe for object extraction such as detection, extraction and reconstruction.Today, the automatic object extraction has become an important research subject for geosciences and remote sensing.Especially, the automatic extraction of man-made objects provides geoscientific information that helps data management in urban and regional areas.Object extraction is utilized in many applications in remote sensing and GIS.For that reason, researches are focus on this issue especially on natural disasters and crisis management (Cobby, 2001;Steinle, 2001), monitoring environmental and ecological changes (Drake, 2002), urban planning and development (Haala and Brenner 1999;Maas and Vosselman, 1999).
Object extraction is a process to separate different categories of feature classes, such as vegetation, ground and building (Sampath and Shan 2007).In classical approach, pixel-based image processing and classification method have been used for object extraction.The main problem in classical pixel-based methods is the confusion of the extracted classes.Today, classical pixel based methods in object extraction is replaced with object-based classification methods.The automatic building extraction is the popular research topic in recent years (Haala and Brenner, 1999;Mao vd., 2009;Wegner vd., 2011;Benz vd., 2004).In addition the automatic building extraction, road extraction (Heipke, 1996;Haala and Brenner, 1999;Rottensteiner and Clode, 2009), and bridge extraction (Sithole and Vosselman, 2006;Elberink, 2010) are the other popular applications in object extraction.
The use of only spectral value of the pixel in pixel based methods limits the success of object extraction in the areas that have similar spectral information (GAO, 2003).To solve this problem, the most suitable solution is the use of the data from multi sensor systems (Baltsavias 1999;Rottensteiner et. al., 2005;Lafarge et. al., 2008).The multi sensor system includes light detection and ranging (LiDAR) system, digital camera and GPS/IMU positioned on the same platform.LiDAR system has a high potential in automatic object extraction with the data they supplied and automation opportunities.In recent years, the combine use of LiDAR point clouds and images have been used for automatic building and vegetation extraction in research studies (Sithole, 2005Rottensteiner et al., 2007;Elberink, 2010;Elberink and Vosselman, 2011).The height information and intensity value obtained from LiDAR data has been used in filtering, segmentation and classification stage of the automatic object extraction to improve the accuracy of confused class as additional information.
To overcome the limitation of single sensor system, we aimed to propose an approach for automatic building extraction using the data from multi sensor systems.The proposed approach using the LiDAR data and aerial images based on object-based image International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-1/W1, ISPRS Hannover Workshop 2013, 21 -24 May 2013, Hannover, Germany analysis with a multi-sensor system was explained.The targeted vegetation, ground and building class were extracted using the fuzzy classification.

METHODOLOGY
The proposed method in this study based on the rule based object-oriented classification method as an image analysis method for automatic building extraction.First phase of our approach is image segmentation and the second phase is fuzzy logic classification.Figure .1 shows the workflow of the proposed automatic building extraction strategy with multi sensor systems.In order to prevent misclassification, rule sets were developed for automatic building extraction.Within scope of this work, building extraction was performed by detecting of object boundaries.The main problem in building extraction is to distinguish buildings from trees (Rottensteiner et. al, 2004).
In order to determine the object boundaries, the slope image was produced by using slope differences in objects.The slope image was created using Zevenbergen and Throne analysis method (Zevenbergen and Throne, 1987) by using slope differences in objects.This method computes each pixels slope value in Digital Surface Model (DSM).For successful building extraction, segmentation step is important.In this step, image is divided into regions or objects that have common properties (Navulur 2007).This process has an important role for meaningful analysis and correct representation of the objects in the image before classification.
In segmentation process, detection of object boundaries can be performed by using optimum weight values during the process.In this phase, object`s spectral and positional properties were taken into consideration.These values were computed to represent the objects with segments by using the properties such as color, morphology, texture, shape etc.In this study, multi resolution, contrast split and chessboard segmentation methods were used to have correct object segments.Multi resolution segmentation method uses a heterogeneity criterion which includes scale, color, shape, smoothness and compactness parameters.(Benz et.al. 2004).Contrast split segmentation method uses edge ratio of bright and dark pixel value to detect object contours.Chessboard segmentation splits the image in to squares like board of chess to form the regions for the correct class.The targeted classes were generated using the fuzzy based classification to categorize the image in to correct feature classes by using membership values.
The most common approach for determining the accuracy of automatic building extractions is based on comparing the data obtained using the classification results with the reference data.This approach called completeness and correctness analyses (Eq.1 and Eq.2).According to this technique, TP represents true positives, FP and FN are the false positives and false negatives, respectively.TP is the number of pixels classified as building in both data sets, FN is the number of building pixels in the reference data not classified as building in the automatic extracted image and FP is the number of building pixels in the automatic extracted image not classified as building in the reference data (Rutzinger et al., 2009). .Completeness=(TP )⁄(TP+FN) (1) Correctness = TP⁄(TP+FP) (2)

STUDY AREA AND DATA SET
The study area, used for testing our automatic building extraction approach, is a suburban area includes buildings, roads, vegetation, shadows, trees, grass located in the city of Sivas, Turkey.The data set was collected with multi sensor system including the Leica ALS60 LiDAR system, DiMAC, Dalsa Area Bayer RGB Charge Coupled (CCD) Camera and GPS and CUS6 IMU system.The data set was obtained from the project of "NABUCCO Gas Pipeline Project" actually was collected for corridor mapping of pipeline which will link the Eastern border of Turkey, to Baumgarten in Austria via Bulgaria, Romania and Hungary.A gridded DSM with a 0.2-mresolution, intensity image with a ground sampling distance (GSD) of 0.2 m and an orthoimage with a GSD of 0.2 m was produced using the data set from the multi-sensor system (Figure 2).

EXPERIMENT
The automatic building extraction was performed with proposed approach using the data set from multi sensor system.The slope image, band subtraction image and canny image were produced in addition the DSM, intensity image and ortho image, and used for improvement of classification results (Figure 3).4).The contrast split segmentation was performed using the band subtraction image determined threshold value.The vegetation class was generated with defined fuzzy rules using the segmentation results.Then, ground and non-ground class were extracted based on statistical analyses using DSM and slope image with defined fuzzy rules.The initial building class was generated using extracted non-ground class and DSM with defined threshold value for slope.The multi-resolution segmentation was performed using orthoimage and canny image applying the different weights for images.The contrast split segmentation was utilized using the intensity image to improve the accuracy of building class.The objects classified as building that was not building were eliminated with defined rules.The building class was obtained after improvements with morphological operations.Figure 5 shows the extracted building in red after performing defined fuzzy rules with the proposed approach.
Figure 5.The extracted building class The performance evaluation of proposed automatic building extraction approach was performed with the completeness and correctness analyses.The reference data set for this method was generated by digitizing the target buildings over the orthoimage.The achieved results were 83.08% for completeness and 85.51% for correctness.

CONCLUSION
In this study, the workflow for the automatic building extraction is presented using object-based image analysis method including segmentation and classification steps with multi sensor system (LiDAR, digital camera and GPS/IMU).The multi-resolution, contrast-difference and chessboard segmentations were applied for extraction of vegetation, ground and building classes.In our proposed approach for building extraction, the object based classification is preferred with defined fuzzy rules to overcome the limitation of traditional pixel based classification such as confusion of classes.The generated Digital Surface Model (DSM) and intensity image from LiDAR point cloud was used to separate mixed classes.The slope image, band subtraction image and canny image were generated using the data from multi sensor system and used segmentation and classification steps.The final building class was obtained after improvements with morphological operations.The performance evaluation of proposed approach was performed with the completeness and correctness analyses with the reference data generated by digitizing of the target buildings over the ortho-image.The achieved results were 83.08% for completeness and 85.51% for correctness.
Figure 1.The workflow of the proposed automatic building extraction strategy with multi sensor systems

Figure 3 .
Figure 3.The slope image (a), band subtraction image (b) and canny image(c) of study area To perform our approach for building extraction, Definiens e-Cognition Developer 8.64 software packages was used.The segmentation is the first and important step of the building extraction since the determined object primitives in this step used in classification.Different segmentation methods such as; multi resolution, contrast split and chessboard segmentation methods have been used in this study (Figure4).The contrast split segmentation was performed using the band subtraction image determined threshold value.The vegetation class was generated with defined fuzzy rules using the segmentation results.Then, ground and non-ground class were extracted based on statistical analyses using DSM and slope image with defined fuzzy rules.The initial building class was generated using extracted non-ground class and DSM with defined threshold value for slope.The multi-resolution segmentation was performed using orthoimage and canny image applying the different weights for images.The contrast split segmentation was utilized using the intensity image to improve the accuracy of building class.The objects classified as building that was not building were eliminated with defined rules.

Figure 4 .
Figure 4.The results of the contrast split segmentation (a), and multi-resolution segmentation (b)