REALITY CAPTURE METHODS FOR REMOTE INSPECTION OF BUILDING WORK

: Conventional building inspection, which requires in-person visits by a qualified inspector, can be costly, time-consuming, and even pose health and safety risks. The travel restrictions of the global Covid-19 pandemic further highlighted the need for remote inspection methods. Using reality capture techniques to create a digital 3D representation of the site offers promise for remote inspection of building work. This paper aims to assess different reality capture methods and their visualizations for remote inspection of building work through a case study conducted in Melbourne, Australia. The reality capture methods included terrestrial and mobile laser scanning, RGB-D imaging, and aerial photogrammetry using a UAV. Professional building inspectors participated in the evaluation process by conducting remote inspection based on the captured data and comparing it to the on-site experience. The assessment involved different visualizations of the data, including 3D panoramas, point clouds, and triangulated surface meshes. The results indicate that image visualisation and the ability to make measurement on images are desirable by professional inspectors. These findings support the adoption of reality capture techniques for remote inspection of building work, which can enhance the safety and efficiency of the process.


INTRODUCTION
Spatial data play an important role in construction engineering.Various stages in the life of a building, from design to construction, and operation, rely on accurate and reliable spatial data.Point measurements, maps, and 3D models generated before, during, and after a construction project help reduce the time, cost, and environmental footprint of the building (Alizadehsalehi and Yitmen, 2021;Alba et al., 2011).
As an important step in most construction projects, inspection of building works ensures the compliance of the construction activities with the relevant rules and regulations (Victorian Legislation, 2022).Traditionally, building inspection involves an in-person visit to the construction site by a qualified inspector.In-person site visits are, however, costly and time consuming (Law, 2022).Depending on the location of the site and the availability of a qualified inspector, conducting mandatory inspections can result in significant delays in the construction work.In addition, in-person site visits involve various health and safety risks for the inspector (Tekin, 2022).
The global Covid-19 pandemic resulted in long delays in many construction projects due to travel restrictions which hindered in-person inspection (Tekin, 2022).Yet, it also provided opportunities for researchers to investigate the feasibility of remote inspection.Remote inspection of building work is a relatively new concept and concerns the use of technology to capture the required data for inspection with the ultimate goal of eliminating the need for in-person site visits (Tekin, 2022;Victorian Building Authority, 2020).Among different approaches to remote inspection, one of the most promising is to create an accurate digital 3D representation of the site, usually referred to as reality capture, which can be used remotely by an inspector.
Photogrammetry and laser scanning using sensors onboard stationary and mobile platforms are the most common methods for reality capture in various engineering fields (Rao et al., 2022).Recently depth sensing, usually referred to as RGB-D mapping, has become popular especially for reality capture in indoor environments (Khoshelham et al., 2019;Díaz-Vilariño et al., 2022).While several sensor technologies and methodologies are available for reality capture in construction projects, the suitability of these against the requirements of remote inspection has not been studied.This paper aims to provide an assessment of several promising reality capture methods for remote inspection of building work.We present a case study of a construction project in Melbourne Australia, where various types of sensors were used to capture spatial data for remote inspection.We also present preliminary results of the case study which provide useful insights into the strengths and limitations of reality capture technologies for remote inspection of building work.
The paper proceeds with a description of the concept of remote inspection of building work in Section 2. In Section 3 the selected reality capture technologies are described.Section 4 discusses the case study and presents the results.The paper concludes with a summary of the findings in Section 5.

CONVENTIONAL AND REMOTE INSPECTION OF BUILDING WORK
After every construction stage, the compliance of the constructed elements with the relevant regulations must be ensured before the construction work can continue to the next stage.This process is referred to as inspection and typically involves an in-person site visit by a qualified surveyor.The current practice of inspection usually involves a series of walkthroughs, both inside and outside of the building, where a variety of criteria is inspected visually or using measuring tools such as a tape measure.The main objects of inspection are critical components like connections, overhangs, bracings, trusses and lentils (Victorian Building Authority, 2021).A typical inspection takes 30-45 min, which does not include commute time.Scheduling an inspection by a qualified surveyor can delay the construction work, because the next stage may not start until the compliance of the current stage is endorsed by the inspector.
The concept of remote inspection involves the use of data and information of the construction site (acquired by an independent party) such that the inspection tasks can be carried out without requiring the inspector to visit the site in person.Such data can be acquired by using reality capture technologies, such as laser scanning, photogrammetry, and RGB-D mapping, which provide a digital 3D representation of the inspection site.To be successfully utilised for remote inspection, the technology must provide features and support that fit the requirements of the task (Goodhue and Thompson, 1995).We identify the following as the key requirements for the application of reality capture technologies in remote inspection of building work: • Accuracy: building inspection tasks usually include making measurements, for example to establish that the constructed elements have the correct dimensions and there is sufficient clearance between different elements.Therefore, the reality capture data must be of high accuracy to enable making accurate measurements remotely.
• Completeness: inspection checklists often include checking the existence of certain elements.This means that the reality capture data must be highly complete and capture all the constructed elements that require inspection (Tran et al., 2019).
• Speed: the main purpose of remote inspection of building work is to make the process faster so as to minimise delays in the construction work.As such, the reality capture process and the post-processing of the data must be efficient and deliver the final remote inspection-ready product quickly.
• Ease of use: The inspector should be able to use the reality capture data easily and extract all the required information without difficulty.This requires an appropriate visualisation of the reality capture data and the availability of software tools that facilitate interaction with the visualisation.

REALITY CAPTURE TECHNOLOGIES
A variety of sensors and platforms are available for reality capture in construction projects.A common approach is 3D reconstruction using RGB images.A dense 3D point cloud can be generated from multi-view imagery (Westoby et al., 2012), and the process is largely automated thanks to Structure from Motion (SfM) (Furukawa and Ponce, 2012) and dense matching (Hirschmuller, 2015) algorithms.However, due to the complexity of the construction sites (e.g., buildings with multiple rooms and occluded areas) this approach requires a large number of images with sufficient overlap.Stationary imaging using panoramic cameras can simplify the acquisition and reduce the complexity of the processing by taking multiple images from a single station.
In RGB-D mapping an additional depth sensor contributes to the generation of a point cloud.There are multiple technologies for depth sensing available, from stereo vision and structured light to Time-of-Flight cameras (Zollhöfer et al. 2018).This study utilizes the Matterport Pro 2, which relies on structured light (Shults et al., 2019).It projects a pattern of infrared dots onto the scene, and then uses infrared cameras to capture the reflected pattern.By analysing the deformation of the pattern caused by the surface of the subject, the camera is able to calculate depth information.The point clouds from individual depth images are registered to generate a triangulated surface mesh model of the whole site.By integrating the panorama images with the surface mesh model, a 3D panorama is generated for each scan station, which enables making 3D measurements on the image.As sunlight can interfere with the structured light, the primary use of Matterport is indoors.
Laser scanners are another popular choice to measure indoor as well as outdoor environments.Most laser scanners capture 3D point clouds by measuring the Time-of-Flight (ToF) of emitted pulses reflected by the surfaces in the environment (Rao et al., 2022).Terrestrial laser scanners (TLS) are mounted on a tripod and scan the site from a static location.Several scan locations are needed to cover a site.In post-processing, the individual scans are registered into a single point cloud.Measured points do not have inherent colour information, but most laser scanners are equipped with RGB cameras and collect images of the scanned area, which are used to attach colour information to points in post-processing.Mobile Laser Scanners (MLS) allow capturing data while moving through the site.In handheld MLS devices, the captured data are registered using the SLAM algorithm to create a single 3D point cloud of the site.Similar to TLS, MLS are usually equipped with a camera and the recorded video can be used to add colour to the point cloud.
Mobile laser scanning sensors can also be mounted on different platforms like rovers, trolleys or unmanned ground or aerial vehicles (UGV, UAV).UAV-borne laser scanning, in particular, can provide data of locations that are less accessible or are unsafe, e.g., the roof.As construction sites are often characterized with obstacles and uneven surfaces, they are not suitable for trolleys.
Considering the requirements of remote inspection identified in the previous section, we select the following three sensing principles for reality capture in a construction site: TLS, MLS and RGB-D mapping.Data captured using different sensors can also be visualised in different ways.To identify the best visualisation of reality capture data for remote inspection of building work, we create several visualisations, namely 3D panorama, point cloud, and triangulated surface mesh model.The suitability of each method is assessed in terms of accessibility of the information required for inspection, interactability for navigation and measurement, and the overall ease of use.

CASE STUDY
A case study was conducted at building sites in Melbourne, Australia, at three different construction stages: slab, framing and roof.All three involved a single-storey residential house with an area of about 250 m 2 .
The inspection of the slab involves examining the early stage of construction right before the pouring of concrete.The building in this case study had an above-ground waffle slab foundation, utilizing polystyrene waffle pods.Further inspections are conducted after the timber framing of the building and after the completion of the roof.The building in this case study features a so-called Hip roof design.

Data collection
Data was captured by using three different sensors, as shown in Figure 1.The Faro Focus S TLS was used for stationary laser scanning, Emesent Hovermap ST was used as a handheld mobile laser scanner as well as onboard a UAV, and Matterport Pro 2 was used RGB-D mapping.The data collection of the slab and of the framing was similar, and involved stationary laser scanning, handheld laser scanning and RGB-D mapping with Matterport.For the roof, the Hovermap was mounted on a UAV to capture lidar data and imagery of the roof.The weather condition at the site was characterized by cloud cover throughout the day.The concrete floor had some residual water due to the earlier rainfall and the surface of the wooden framework of the building was still wet.To cover the whole building from the inside as well as from the outside, a total of 13 scans were acquired using the Faro Focus S. Each scan took approximately nine minutes, resulting in a total scan time of two hours for the whole site.Scanning the site with the MLS Hovermap was considerably quicker and took roughly 30 minutes.
As the Matterport sensor has a short measuring range of about 5 metres, a total of 33 scans were required to cover the entire site.Each scan took approximately 30 seconds.As the registration process failed several times due to the challenging environment, several scans on multiple locations had to be repeated.In total, mapping with Matterport took roughly 2.5 hours.The data collection times are given in Table 1.The data processing included point cloud registration and colourisation on a laptop computer with an Intel Core i7 CPU but without a GPU.
Table 2 provides an assessment of the three sensors in terms of accuracy, completeness of the data, speed of data acquisition, and susceptibility to environmental conditions including daylight and rain.TLS provides the most accurate data.The individual TLS point clouds were registered in FaroScene, which resulted in a mean point error of 5.5 mm.However, the TLS is slow and the data may be incomplete because of the stationary nature of the data acquisition.Additionally, the site of the case study had large reflective water puddles on the ground which led to gaps in the TLS point cloud.Data acquisition by MLS is faster and the data are more complete since the sensor is mobile and can scan an object from all sides.The processing of the MLS data took significantly more time (20 hours -see Table 1) than the other sensors presumably due to the complexity of colourization with the recorded video.The resulting 3D point cloud is also less accurate as compared to the one created from TLS data.As MLS uses SLAM to map the environment it requires distinctive features in the point cloud.The wooden frame of the building exhibits many similar patterns of beams, which can lead to a mismatched point cloud.Therefore, special caution is required when scanning the framing with MLS.
RGB-D mapping provides moderate accuracy, completeness, and high speed in ideal environmental conditions.However, the case study showed that RGB-D data are highly affected by daylight and rain.The infrared structured light sensor suffers from sunlight and other strong artificial light sources, even on a cloudy day.Wet surfaces and water puddles cause severe data gaps, as they reflect light like a mirror.The complex and very repetitive pattern of the wooden frame can also be an aggravating factor.This led to several registration errors of the RGB-D data.
Another important aspect is the user friendliness of the presented sensors.Here, the Matterport stands out because it is very easy to use.There are no different scan modes to choose from, and the user only has to start a scan and later move the scanner to the next position.The same ease of use applies to the processing of the data, which does not require any input from the user, as there are no options available to adjust the processing.For data captured by each sensor, different visualisations of the construction site were created as shown in Figure 2. As can be seen in Figure 2 (c), the 3D mesh model generated from RGB-D data is highly incomplete, which indicates the significant impact of environmental conditions (daylight and rain) on the performance of Matterport sensor.

Sensor
For the roof, in addition to the lidar data we captured a photogrammetric block of images with the Zenmuse P1 camera onboard the DJI Matrice 300 UAV.In total 246 images were captured with flying height varying from 10 m to 30 m.The images were used to create a 3D model using the software Agisoft Metashape.After aligning the images, a point cloud was created and then triangulated to get a textured mesh model.By positioning a measurement tape on the site, the whole model was scaled.The photogrammetric mesh model was used to create an orthomosaic of the roof, as shown in Figure 3.

Remote inspection
To evaluate the feasibility of the created models for inspection purpose, the different visualisations of the data were presented to experienced building inspectors to perform inspections like they do it on site.
The Matterport data was visualized within the web based Matterport viewer which works very fluent as long as the internet connection is stable.The navigation is similar to Google Street View and enables a fluid motion between panorama-views.Inside the model it is possible to conduct freehand measurements to get the distance between two points.The accuracy of this distance is based on the accuracy of the triangulated surface mesh model.
For the visualization of the TLS and MLS data, the software Autodesk ReCap was used.As a desktop software it relies on the computing power of the local machine and the processing can be computationally expensive.In this case study, the loading time between different panorama-views amounts to four to five seconds.ReCap provides different measurement options like freehand, orthogonal, surface or pipe radius measurements, shown in Figure 4. Furthermore, single points or groups of points can be marked and attached with a note, which can be useful for marking compliance issues.To evaluate the accuracy of manual distance measurements inside the three different models, we used known reference lengths determined by measuring tape.Distance measurements exhibit a root mean square error of 0.5 cm inside the TLS point cloud, 1.6 cm inside the MLS point cloud and 8.9 cm inside the Matterport model.Two qualified building inspectors were asked to assess the suitability of the different visualisations of the data in terms of accessibility of the information required for inspection, interactability for navigation and measurement, and the overall ease of use.The assessment involved rating each visualisation on a scale of 1 to 10.The result of the assessment by the building inspectors is shown in Table 3.As it can be seen, the building inspectors preferred the visual data of panorama images over point cloud or mesh visualizations.The panorama image visualisation felt the closest to their on-site perception and the most reliable compared the point cloud and the mesh which exhibit data gaps.Several points of the inspection checklist are related to small objects like braces and screws, which have sizes smaller than 1 cm.While the MLS point cloud has the highest completeness, it does not achieve this level of accuracy, which is why the MLS point cloud was rated low in terms of accessibility by the two inspectors.Even though the TLS point cloud has millimetre accuracy, the detection of fine structures in photos was preferred by the inspectors.With a point density of e.g. 2 mm, one 5 mm screw might be represented with only 4-5 points, which is not enough for measurements.Also, points fall on random places on objects, rather than on their edges, which is another issue and a reason why a very high point density is required to capture objects and enable accurate measurements.
Using panorama images as a primary visualization, the 3D information of point clouds and mesh surfaces can be used in the background to enable 3D measurements.As inspection is a critical process that ensures the structural integrity and safety of the building (Tekin, 2022), digital measurements should be handled with caution with having their respective error range in mind.Table 3. Assessment of visualisation methods by the inspectors in terms of accessibility, interactability, and overall ease of use on a scale of 1-10 (10 is best).
Many TLS scanners like the Faro Focus S provide panorama images.But the number of stations and thus also the number of panorama-views in this field study was lower than that of the RGB-D sensor.This was a crucial point for the inspectors to lean more towards RGB-D data.In future studies the number of TLS scans could be raised which means that the point density of individual scans could be reduced.
The inspectors made Matterport the inspection technology of their choice.This was also based on the interactability, or more precisely the user-friendly interface and navigation which enabled them to do the most fluent and confident inspection.This proves that the way of visualisation and the smoothness of the workflow of an inspection is of great importance.In contrast, navigation and orientation in 3D point clouds in Recap was intimidating for the inspectors.Another plus is the ability to share the Matterport scan online with other people who don't need a local program with local computing power or an account.However, it must be considered that this data is not the property of the user, but of the company providing the server.This could conflict with privacy concerns.
The inspectors also pointed out, that an experienced building inspector can do a very fast on-site inspection.The navigation with zooming and panning on the computer screen can not reach the same speed.As the Matterport exhibits blind spots right above and below it, especially the inspection of the ceiling is challenging and requires finding the right position with the right viewing angle.
The view from the ground is limited and compliance issues could be occluded and overseen.The roof inspection with an UAV was of high value for the inspectors, as they could assess areas which were not accessible for them before.One important part of the roof inspection covers the gutters and the drainage.Therefore, the textured surface mesh can deliver useful information.However, the some fine structures were represented poorly in the mesh model.The inspectors preferred the high resolution orthomosaic, as it shows the most details and was very-user friendly.By opening the orthomosaic in a GIS software, it is also possible to conduct distance measurements.

CONCLUSIONS
This paper evaluates the feasibility of reality capture techniques for remote inspection of building work.We have performed a case study where professional building inspectors performed a remote inspection based on different reality capture methods, namely terrestrial and mobile laser scanning, RGB-D and aerial imagery.An assessment of these methods in terms of accuracy, completeness, speed, and susceptibility to environmental conditions is provided.Furthermore, different visualisations of the data, such as 3D panoramas, point clouds, and triangulated surface meshes, were evaluated for their effectiveness in remote inspections.Based on an assessment by two qualified inspectors, the 3D panorama stands out as the preferred visualisation of reality capture data for remote inspection.The 3D panorama images were found to be the most user-friendly visualisation and the closest to the on-site experience of the inspectors.
The results of the case study provided useful insights into the short and long-term facilitation of remote inspection techniques and highlight multiple benefits of remote inspection, including cost and time savings, enhanced safety, and improved efficiency.In the short term, UAV-borne imagery and lidar data have a significant potential to enable remote inspection of inaccessible parts of the building, such as the roof, to reduce the amount of unnoticed compliance issues.However, the application of reality capture technologies for remote inspection of building work requires a trained assistant to operate the reality capture sensors to collect data.In the long term, reality capture technologies could be operated from remote or even be fully automized by using autonomous drones for example.Machine learning can assist or automate the data processing to extract inspection related information and identify critical compliance issues.
As more and more industry players realise the potential of the spatial data for the inspection of building work, new technologies, sensors, and software applications are also being developed.For example, HoloBuilder by Faro is a construction progress tracking platform and can be examined in future research.Other directions for future research include the development of new visualisation methods, and evaluation of new sensors such as the Matterport Pro3 which uses lidar instead of infrared structured light to minimise its susceptibility to daylight.

Figure 2 .Figure 3 .
Figure 2. Different visualisations of the data captured of the construction site: (a) point cloud of slab, captured by Faro Focus S, (b) point cloud of frame, captured Faro Focus S, (c) textured mesh model of frame, generated from Matterport RGB-D data, (c) 3D panorama of frame, generated from Matterport RGB-D data.

Figure 4 .
Figure 4. Measurement of the size of window frames in the TLS point cloud.

Table 1 .
Data collection times for stationary and mobile laser scanning as well as RGB-D mapping.

Table 2 .
Assessment of reality capture sensors in terms of accuracy, completeness, speed, and susceptibility to environmental conditions (daylight and rain/water).