INTEGRATION OF POINT CLOUDS ORIGINATED FROM LASER SCANER AND PHOTOGRAMMETRIC IMAGES FOR VISUALIZATION OF COMPLEX DETAILS OF HISTORICAL BUILDINGS

Three-dimensional (3D) models of historical buildings are created for documentation and virtual realization of them. Laser scanning and photogrammetry are extensively used to perform for these aims. The selection of the method that will be used in threedimensional modelling study depends on the scale and shape of the object, and also applicability of the method. Laser scanners are high cost instruments. However, the cameras are low cost instruments. The off-the-shelf cameras are used for taking the photogrammetric images. The camera is imaging the object details by carrying on hand while the laser scanner makes ground based measurement. Laser scanner collect high density spatial data in a short time from the measurement area. On the other hand, image based 3D (IB3D) measurement uses images to create 3D point cloud data. The image matching and the creation of the point cloud can be done automatically. Historical buildings include more complex details. Thus, all details cannot be measured by terrestrial laser scanner (TLS) due to the blocking the details with each others. Especially, the artefacts which have complex shapes cannot be measured in full details. They cause occlusion on the point cloud model. However it is possible to record photogrammetric images and creation IB3D point cloud for these areas. Thus the occlusion free 3D model is created by the integration of point clouds originated from the TLS and photogrammetric images. In this study, usability of laser scanning in conjunction with image based modelling for creation occlusion free three-dimensional point cloud model of historical building was evaluated. The IB3D point cloud was created in the areas that could not been measured by TLS. Then laser scanning and IB3D point clouds were integrated in the common coordinate system. The registration point clouds were performed with the iterative closest point (ICP) and georeferencing methods. Accuracy of the registration was evaluated by convergency and its standard deviations for the ICP and residuals on the control points for the georeferencing method.


INTRODUCTION
Documentation of historical buildings are very important for rebuilt and reconstruction of them when they are destroyed.Their reconstruction with the keep their original shapes and size is possible with their three-dimensional (3D) models.The creation of their 3D digital model is effective methods on documentation of cultural heritage.It requires measurement 3D spatial data in a particular space from the whole object surface.Photogrammetry is traditional methods on documentation of historical artefacts (Remondino, 2014;Bosch et al. 2005;Toz and Duran, 2004;Lingua et al., 2003).However the measurement to all details wit particular spaces by the photogrammetry require more time and labour.Thus some characteristic points which represent the shape are measured from the images.On the other hand laser scanning has been extensively used to creating 3D model of cultural heritage for two decades (Altuntas et al., 2014;Akca et al., 2006).Terrestrial laser scanner (TLS) collects high density 3D spatial data from the measurement area in a short time.The historical buildings have complex details.All details cannot be measured by TLS since the TLS makes ground based measurement.On the other hand, 3D virtual modelling and realization needs photogrammetric and laser scanning data (Remondino, 2011;Grussenmeyer et al., 2011).Texture data of photogrammetric images are matched with the laser point cloud for creation of 3D virtual model.In addition different measurement techniques have been used for creation 3D model of historical buildings (El-Hakim et al., 2007;Kulur and Sahin, 2008;Chiabrando and Rinaudo, 2013;Altuntas and Yildiz, 2013).
Image based 3D (IB3D) modelling have been performed automatically by the interest points detected from the images.3D spatial positions and structure from motion are estimated from the matched images in a short time (Chiabrando and Spano, 2013;Grussenmeyer et al., 2012;Barazzetti et al., 2010).The dense IB3D point cloud has been created by matching the image pixels.In this study, IB3D point clouds have been created for the areas that could not be measured by the TLS.Then the point clouds originated from laser scanning and images were integrated by ICP and georeferencing.

Terrestrial laser scanning
TLS measures local coordinates and intensity of the laser light reflected from the measured points.In addition color data can be recorded for the measured points.The laser scanner instruments use direct time-of-flight, indirect time-of-flight and triangulation methods to measure distance from instrument to object point.The triangulation laser scanners have small measurement distance because of the inherent characteristics of the instrument.Their measurement distance is about 8 meters.The measurement distance of laser scanner that uses indirecttime-of-flight technique varies between 100 to 300 meters.Finally the measurement distance of laser scanner that uses direct time-of-flight techniques reaches more than three kilometres.On the reverse of the measurement distances, the accuracy of TLS that uses indirect time-of-flight technique is better than that uses direct time-of-flight technique.The specifications and measurement speed of TLS are getting better by the time.Generally, the measurement accuracies of TLSs are a few millimetres in a 100 meter.And they can collect points more than 500000 in per second.
The registration of laser scanning data has been done by control points (Alba and Scaioni, 2007) or point cloud coordinates and photographs (Dold and Brenner, 2006).The many techniques have been executed to registration of point clouds.The iterative closest point (ICP) is extensively used to registration of point clouds (Zhang, 1994).The registration of point clouds by the ICP have been applied iteratively with the conjugate closest points between them.It has high accuracy result.The accuracy of the ICP is a little affected by the space between the measured scan points (Altuntas, 2014).The main steps on 3D modelling by TLS are below: -Overlap laser scanning -Point cloud registration -Occlusion measurements by other techniques -Mesh creation -Texture mapping and 3D virtual model

Dense image matching
Image based 3D (IB3D) point cloud can be created from the images by dense matching.Hereby high density 3D point positions can be measured with low cost.The procedure needs only camera and software for the processes.Off-the-shelf low cost cameras are used for this aims.Today many, commercial software such as SURE, MicMac, PMVS, Photoscan have been developed for creating IB3D point cloud, dense matching and structure from motion (Remondino et al., 2014).IB3D point cloud generation is attractive method especially for 3D object modelling (Barazzetti et al. 2010(Barazzetti et al. , 2011;;Remondino et al., 2014) and creation of digital elevation model (Rothermel ve Haala, 2011).
Feature based matching with the interest points has been applied for photogrammetric triangulation of multi-image network.Many interest operators such as Förstner, SUSAN and FAST have been used for matching of images in practices (Jazayeri and Fraser, 2010).In addition, invariant image features (keypoints) have been used to match images.Then many keypoint detection metods such as SURF, ASIFT, BRIEF have been developed after introducing the SIFT.The candidate correspondence for each feature points are find after detecting the feature points from the set of images.The epipolar geometry constraints are used to limit the search area and find good matches (Fathi and Brilakis, 2011).The epipolar geometry is used to detect error matched feature points.It has been applied by RANSAC, least median of squares and maximum posteriori sample consensus (Barazetti et al. 2010).3D coordinates of the matched features are then calculated.Two sparse point clouds are created from two consecutive stereo pairs, then, camera motion is estimated by registering point clouds using the quaternion motion estimation (Fathi and Brilakis, 2011).After all, bundle adjustment is applied for the block that contains all images matched.So as to creation IB3D point cloud, each pixel is matched with candidate on the other images by the image match techniques.Then 3D point cloud coordinates which represent the map of the image area is calculated by the photogrammetric evaluation.The scale of the point cloud is performed by ratio of distances or coordinates of control points.More information about dense image matching and creation of IB3D point cloud can be found on Brazetti et al., (2010) and Fathi and Brilakis (2011).

MEASUREMENT
The studies were performed on front facade of the Sultan Selim mosque, Konya, Turkey (Figure 1).Laser scanning was made from 20m away.The space between the measured points is 2 cm on this measurement distance.Because the facade has more complex details, its fully measurement needs laser scanning from many stations.However, all details should be measured to creating the mesh model.The occluded parts were measured by photogrammetric images and point cloud was created from the images.The images were recorded by Nikon P50 camera (pixel array is 2592x1944, pixel size is 2.14 micron).The record of the images was planned as overlap with the point cloud of TLS.
The image pixels (3 micron) corresponds about 1 cm on average imaging distance.The second study was performed on the door that includes more complex shapes.The IB3D point cloud was created from 57 images and 3D positions of 2 million points were calculated (Figure 3).The scale has been executed by four control points.

DATA INTEGRATION
Laser scanning and IB3D point clouds were integrated by the registration with the ICP method.The registration was implemented by PolyWorks software (url-2).The laser scanning data was selected as a reference and the IB3D point cloud was registered relation to reference system.IB3D point cloud of the first study was registered using ICP with 0.003mm mean convergence and 6.4mm standard deviation.The differences between the conjugate points are very low after the registration (Figure 4).Similarly, the IB3D point cloud of the second study was registered to the reference system with mean convergence of 0.059mm and standard deviation of 8.7mm (Figure 5).
Another method for the integration of laser point cloud and IB3D point cloud is registration of each point cloud data into the georeference coordinate system.TLS point cloud and IB3D point cloud has been registered into the geodetic coordinate system by five and four control points respectively.Similarly, IB3D point cloud of the door has been registered by five control points.Thus the TLS point cloud and IB3D point cloud has been integrated by registering into the geodetic coordinate system.On this stage of the processes, we want to compare integration methods of point clouds by ICP and georeferencing.
The TLS point cloud was accepted as a reference and ICP was implemented to the georeferenced IB3D point cloud.The changes on the control points were exploited to evaluate the registration methods (Tablo 1).

CONCLUSION
Laser scanner measures 3D spatial data in short time.Nevertheless, all details cannot be measured by TLS.Thus, unimaged parts of the objects should be measured with the other methods.Dense point cloud creation from the images is the method that can be used as individual or complementary for laser scanning in documentation of cultural heritage.In this study, the surfaces that could not be measured by laser scanner are measured by creating point cloud from dense matched images.Today, many software have been used for dense matching and point cloud creation of the images.PhotoScan is user friendly software.TLS and IB3D point clouds have been integrated by ICP and georeferencing methods.The results of the both methods are satisfactory.According to the results obtained from the comparison, the accuracy of ICP is a little high then the georeferencing.The individual georeferencing of point clouds obtained from TLS and images requires control points that needs more time and labor.

Figure 1 .
Figure 1.Front facade of the Sultan Selim Mosque.Two study were performed for creating point clouds from dense matched images.The first study was executed on the left of the front facade.This part includes two window and mihrab among them.Sixty images were recorded to creation IB3D point cloud of this area, and 46130 points were created by Agisoft PhotoScan software (url-1) from dense matched images.The point cloud model was scaled by four control points (Figure2).The PhotoScan is user friendly and high accuracy software.

Figure 3 .
Figure 3. Dense IB3D point clouds of the door.The second study.

Figure 4 .
Figure 4.The TLS point cloud is reference.The IB3D point cloud of the first study was registered into reference system with ICP.The scale bar shows low divergences.[m]

Table 1 .
Differences on the control points after applying ICP to the georeferenced point cloud.