FINDING THE NEXT-BEST SCANNER POSITION FOR AS-BUILT MODELING OF PIPING SYSTEMS

Renovation of plant equipment of petroleum refineries or chemical factories have recently been frequent, and the demand for 3D asbuilt modelling of piping systems is increasing rapidly. Terrestrial laser scanners are used very often in the measurement for as-built modelling. However, the tangled structures of the piping systems results in complex occluded areas, and these areas must be captured from different scanner positions. For efficient and exhaustive measurement of the piping system, the scanner should be placed at optimum positions where the occluded parts of the piping system are captured as much as possible in less scans. However, this “nextbest” scanner positions are usually determined by experienced operators, and there is no guarantee that these positions fulfil the optimum condition. Therefore, this paper proposes a computer-aided method of the optimal sequential view planning for object recognition in plant piping systems using a terrestrial laser scanner. In the method, a sequence of next-best positions of a terrestrial laser scanner specialized for as-built modelling of piping systems can be found without any a priori information of piping objects. Different from the conventional approaches for the next-best-view (NBV) problem, in the proposed method, piping objects in the measured point clouds are recognized right after an every scan, local occluded spaces occupied by the unseen piping systems are then estimated, and the best scanner position can be found so as to minimize these local occluded spaces. The simulation results show that our proposed method outperforms a conventional approach in recognition accuracy, efficiency and computational time.


INTRODUCTION
Renovations in plant equipment of petroleum refineries or chemical factories have recently been frequent, and 3D as-built modelling of piping systems of the plants based on terrestrial laser scanning is strongly expected to make renovation process more efficient.To this end, many researches have been intensively studies to recognize piping objects such as straight pipes, elbows, junctions and valves from point clouds captured from terrestrial laser scanners (Vosselman et al., 2004), (Rabbani et al., 2004), (Rabbani et al., 2006), (Belton et al., 2006), (Bey et al., 2011), (Masuda et al., 2012), (Kawashima et al, 2013).When capturing the plant objects using a terrestrial laser scanner, several scans must be taken from different scanner positions to ensure a full coverage of the object surfaces and to avoid the occlusions.As shown in Figure1(a), piping systems of plants are complex and tangled in structure, and this results in many occluded space where background piping objects are hidden by the front-side objects in a scan, and the objects contained in the occluded space cannot be measured and modelled.To solve the occlusion, as shown in Figure 1(b), the scanner has to be repositioned at the different place from the former one where the occluded space can be captured as large as possible while keeping the number of scans small.However, this "next-best" scanner position is usually determined by experienced scanner operator in practice, and there is no guarantee that the sequence of scanner positions is the best one for capturing the unseen piping objects as much as possible in less scans.* Corresponding author: Satoshi Kanai (kanai@ssi.ist.hokudai.ac.jp) The problem to find an optimum sequence of sensor arrangements for measuring a given object is known as "next best view"(NBV) problem, and many solutions have been proposed mainly in robotics so far, for example (Dornhege and Kleiner, 2013) and (Potthast and Sukhatme 2014).An excellent review of the NBV solutions is already presented (Scott and Roth, 2003).On the other hand, in photogrammetry community, little study tacked this problem where the terrestrial laser scanner should be placed sequentially in large indoor/outdoor environment.A few exceptions include (Munkelt et al., 2006) and (Soudarissanane et al., 2011).In these conventional studies, the best scanner position can be found where the volume of the occluded unseen space is minimized.However, this strategy is not necessarily effective and efficient to solve the NBV problem in the measurement of piping objects of plants.
To solve the issue, the purpose of this study is to propose a new computer-aided method to find a sequence of next-best scanner positions specialized for as-built modelling of piping systems without any a priori information of the piping objects.Different from the former approaches, in this study, as shown in Figure 1(c), the best scanner position is found in the way that the method tries to minimize the spaces which are occluded and unseen but any piping object is likely to exist.To this end, part of the piping objects are sequentially recognized from its point clouds right after each scan using our object recognition method (Kawashima, Kanai and Date, 2014), and this recognition result is used as a clue to estimate the potential spaces which are likely to contain piping objects in the occluded spaces.And finally, a best scanner position is found so as to minimize only the potential spaces where the piping objects is likely to exist.The effectiveness of the proposed method was verified by the computer simulation.And the results showed that our proposed estimation method of the scanner positions outperforms one of the recent solutions for NBV problems (Potthast and Sukhatme 2014) which has similar problem settings to ours in the recognition accuracy, efficiency and the computation turnaround time.

Functional Requirements
In our study, the next-best position of the terrestrial laser scanner specialized for as-built modelling of large-scale complex piping systems of the plant which contain straight pipes, junctions and elbows needs to be estimated.Therefore, the estimation method should fulfil the following functional requirements; -The scanner positions can be estimated without any a priori knowledge of the geometry of the piping system to be measured, because as-built or even as-planned CAD models of the plants are generally unable to use.-The scanner positions are not limited to the outer peripheral space of the piping system but to the inside of it, because the visibility of the piping objects from the interior of the system sometimes becomes richer than that from the outer peripheral space and it enables effective reduction of the occlusion.
-A scanner must be positioned so that it can measure the potential space where the occluded part of the piping objects is most likely to exist as much as possible.-A scanner device itself must be stably-positioned without any collision with the piping objects.

Related Work and the Issues
Many algorithms for the next-best view (NBV) or view planning problems have been proposed mainly in robotics and image processing fields so far.And most of them concern with the user of range sensors.(Scott and Roth, 2003) presents an informative and exhaustive review of the existing algorithms to solve the NBV problems.These algorithms are classified into model-based and non-modelbased ones.However, the model-based algorithms such as (Scott, 2009) require an a priori object model at some level of fidelity and does not agree with our requirements.
Studies of non-model-based algorithms for NBV have been intensively proposed for reverse engineering using range sensors with multi-DOF positioning platforms.For examples, in (Karaszwwski et al., 2012), the range sensor is fixed at the 6DOF robot hand and can be moved on the outer periphery around a small object.The next best sensor position is selected as the one among the periphery from which the sparse point density area on the object can be maximally captured.Similar studies are done by (Papadopoulos-Orfanos et al., 1997), (Reed et al., 1999), (Khalfaoui et al., 2012) where the completion of the 3D reconstruction of a small object is focused in NBV.(Pito, 1999) takes into account the scan overlap identification and tolerance constraints of the choice of the NBV in the similar context.(Munkelt et al., 2006) also proposes a voxel-space-based NBV algorithm where the point clouds with better quality are achieved using rough a-priori information of the object and better chosen viewpoints with respect to object surface normals.However, in these studies, the object to be measured is a single one with simple geometry and its size is comparable to the scanner's measuring area.In this sense, the setting of the scanners and their purposes of the optimization in NBV are different from the ones of the large-scale measurement using terrestrial laser scanners.
The NBV problem for the large-scale complex indoor objects using the range sensors have also been studied in (Klein et al., 2001), (Dornhege and Kleiner, 2013), (Potthast and Sukhatme 2014).The mobile robots with manipulators which can control the position and orientations of the range sensor go driving inside or outside of the indoor objects, and the NBV problems are solved so as to maximize the visible space by resolving unseen occluded volumes.Their strategy for the NBV is to find the best scanner position where the volume of the occluded unseen space is minimized.However, this strategy is not necessarily effective and efficient to solve NBV problem for the environment of plant piping system.(Soudarissanane et al., 2011) recently proposes an algorithm to determine optimum viewpoints in a large scale indoor scene based on terrestrial laser scanner capabilities.An optimum scanner arrangements for a large room with complex walls is obtained under an incidence and range constrains of the scanner device.Unfortunately, their method can work only on a 2dimensional map of the large scene, and the 2D map must be provided as a priori knowledge of the environment.This assumption is not practical for the view planning of 3dimensional plant piping system without any a-priori information.(Dan et al., 2013) proposes a practical algorithm for scan planning of outdoor buildings using a terrestrial laser scanner.
The algorithm is reduced to a kind of mixed linear integer programming problem, and efficiently works in 3D.Unfortunately, their algorithm requires a priori rough knowledge of the buildings, and the small limited number of candidates for the best scanner position need to be pre-specified inside the buildings.
On the other hand, in conventional photogrammetry, the imaging network design using passive sensors and the optimum sequential view planning using multiple images from a single camera have been investigated.The typical examples include (Farashidi, et al., 2009), (Hosseininaveh A. et al., 2014).The view planning algorithms using multiple images provide promising results of the reliable 3D acquisitions, and they substantially solve the similar problems to NBV.Some of them tackled the view planning for large-scale buildings or statues (Alsadik et al., 2012) (Won et al., 2013).However, the main focus of their view planning algorithms is to improve the image coverage and reconstruction accuracy, and they cannot be directly applied to solve the NBV problem of the measurement of complex piping objects using a terrestrial laser scanner.
To wrap up, to the best of our knowledge, there is no research which meets the requirements of section 2.1 and which gives a solution of next-best-view problem for terrestrial laser scanning of 3-dimenntional, complex and large-scale plant environments.

Overview
Figure 2 shows the overview processing flow of the proposed estimation method for next-best scanner position.In this study, the laser scanning operation is simulated in the computer instead of the real scan, and the scanned point clouds are generated directly from a reference CAD model so as to make the validation easy.We assumed that the approximated maximum size of the piping system to be measured (width, depth and height) is known.
The outline of the scanner position estimation consists of the following steps: (1) Voxel attribute assignment: Right after the laser scan of the environment, the spatial occupancy of each voxel is classified based on the ray-tracing between a given scanner position and every point in the scanned point clouds.
(2) Estimation of scanner position candidates: The candidates for the next place to put a laser scanner device are estimated in the voxel space.
(3) Recognition of piping objects: From the aggregated scanned point clouds, the piping objects are recognized using our automatic recognition algorithm.
(4) Estimation of the voxels to be measured and their priority scores: A limited number of the unseen voxels which are likely to exist on some piping objects are picked up based on the recognition results of (3), and their priority scores for the next measurement are evaluated.
(5) Estimation of the optimum next scanner position: By estimating the observation probability between the scanner position candidates and the priority scores, the optimum next scanner position is determined.
The details of them are described in the following subsections.

Voxel Space Decomposition and Space Classification
In our method, the best scanner position is determined based on the voxelization to encode the special occupancy in the measured space.As shown in Figure 2, a space which is all-enclosing the piping system is initially decomposed into a set of small voxels.
Every voxel  has its spatial attribute () to encode the space occupancy which means; -() = unknown : the voxel  has not been measured by the scanner yet, -() = free : the voxel  has been measured already, but there is no scanned point and no object in , and -() = occupied: the voxel  has been measured, and there is at least one scanned point in .
In the beginning of the estimation, all of the voxels are initialized as () = unknown.

Executing a Laser Scan and Voxel Attribute Assignment
One scanning operation using a terrestrial laser scanner is executed in a specified scanner position.Only in the first scan, the scanner position is determined manually, but after the second scan, the best scanner position can be determined in fullyautomatic way by using the proposed method.In this paper, we assume that the next scan is done by fully respecting this nextbest scanner position.Right after the point clouds are captured from a scan, using the combination of a given scanner position   and every measured point   in the point clouds of this scan, a set of scanned voxels   each of which is pierced by the line     is extracted using the ray tracing operation in the voxel space.And, for every voxel  ∈   on the line     , the spatial attribute of  is changed to () = free if any measured point   is not included in .On the contrary, the attribute is changed to () = occupied when  includes   .By repeating this operation for all measured point   in the point clouds, the spatial attributes of all voxels () are updated for this scan.
To simplify the validation of the proposed system, in this paper, instead of executing the real scanning operations, the scanning operation of the terrestrial laser scanner is simulated in the computer, and the measured point clouds are given directly by sampling the points on surfaces in a 3D CAD model of a given plant piping system.

Estimation of Scanner Position Candidates
From the (simulated) measured point clouds and the current space attributes of the voxels, the possible candidates for the scanner position for the next scan are estimated in the voxel space.
The candidates fulfil the functional requirement that the scanner must be stably-positioned without any interference with the piping system.Therefore, the height and radius (ℎ  ,   ) of a bounding cylinder   enclosing an assumed terrestrial scanner device shape is specified by the user in advance.As shown in Figure 3, first, the normal vector of each measured point in the point cloud is estimated using PCA, and the set of points whose normal orient vertically-upward is extracted.Then, the points close to each other among them are aggregated into one horizontal point cluster using Euclidian clustering.This point cluster can be considered as a "floor point cluster", and the space attribute of the voxel  including the floor point is changed to () = floor .Then, a set of scanned voxels   is picked up which place on the voxels on the floor (() = floor), enclose the bounding cylinder of the scanner   and do not include any occupied voxel ( () = occupied).And a voxel   in   which is located at the horizontal level of ℎ  and satisfies (  ) = free is finally selected as a possible candidate scanner position for the next scan.By collecting all of the voxel   among the voxel space, a set of scanner position candidate voxels   can be generated.

Recognition of Piping Objects from Point Clouds
The portions of straight pipes, elbows and junctions are recognized in a fully automatic way from the union of the measured point clouds from the aggregation of the previous scans using the object recognition method proposed by the authors (Kawashima, Kanai and Date, 2014).Figure 4 shows an example of the recognition of the piping objects of an oil rig.Based on the recognition results, the appropriate cylinders and torus surfaces are fit to the measured point clouds placed on the piping system.

Estimation of the Voxels to be measured and their Priority Scores
The feature of the proposed method is that the object recognition results are used as a clue to estimate the space of remaining piping system in the currently occluded area among the unknown voxel space (() = unknown).In our estimation, only a limited set of unknown voxels which are likely to exist on the surface of The bounding cylinder of the scanner C scan Moreover, the priority score for the next scan which represents the existence probability of the occluded piping system is given to each unknown voxels to be measured.The score is designed so as to become high in a voxel where the remaining part of the piping system is most likely to exist.As shown in Figure 5, the occlusion of the pipe in the laser scan is roughly classified into two typical types.In the first type, the back side of the cylindrical pipe is hidden by the scanned front side, and the remaining part is most likely to exist on the back side of the cylindrical pipe.While in the second type, a portion of a pipe is hidden with the other pipes, and remaining part is most likely to exist on the cylinders placed on the extended portion of the recognized pipe.Therefore, only the unknown voxels which locate on this back side or on the cylinders placed on the extended portion of the recognized pipe are selected as the spaces to be measured in the next scan.
According to the consideration above, the priority score () for an unknown voxel () is evaluated as follows.As shown in Figure 5(a), in the first type of occlusion, if a circular section of the pipe has been partially measured and the covering angle  on the circle of the current measured point cloud is still small, the remaining cylindrical part must be captured in the next scan to ensure the stable cylinder fitting in the piping system recognition.However, if the covering angle  is large enough, the remaining part does not need to be measured anymore in the next scan.By reflecting the observations, the priority score in the first type   () ∈ [0,1] is given as Eq.( 1).
where,   is a critical covering angle enabling the stable cylinder fit to the scanned points on pipes.If the angle exceeds   , the measured points already covers on the pipe surface large enough for the stable and accurate cylinder fit, and the remaining backside part does not need to be measured anymore in the next scan.  denotes the minimum critical covering angle the cylinder fit. ∈ [0,1] is a weight for prioritizing the score   () when determining a final score of the voxel ().And even if the angle exceeds  , the remaining backside still need to be measured for the stable cylinder recognition in the next scan.
On the other hand, as shown in Figure 5(b), in the second type of occlusion, if all of the section of the pipe has not been measured yet, the pipes are most likely to exist in the coaxially extended portions of the pipes which have been already scanned and recognized.We assume that this possibility gradually decreases as the distance  from the end face of the pipe increases.So the priority score in the second type   () ∈ [0,1] is given as Eq.( 2).
where,   denotes the user-specified critical extension length, and  ∈ [0,1] is a weight for prioritizing the score   ().
Finally, all unknown voxel  to be measured is inserted to the set   , and a priority score () is finally assigned to  as () = {   () ,   ()}.

Estimation of the Optimum Scanner Position
Using the set of unknown voxels to be measured   , the priority score () ( ∈   ) and a set of scanner position candidate voxels   , the optimum scanner position for the next scan is determined.The estimation process is our original extended version of the conventional solution for the next-best view problem (Potthast and Sukhatme 2014).We evaluate how many voxels to be measured can be observed from a scanner position candidate as an expectation value of their priority scores.The detail evaluation process is described in the following.As shown in Figure 6, first, the observation probability (  |  ) of a voxel to be measured   (∈   ) from a voxel of the scanner position candidate   (∈   ) is evaluated as Eq.(3).
where  is a straight line connecting the centroids of the voxels   and   , and   is a set of voxels which intersects with .(  ) gives a probability whether a laser beam of the scanner can pass through the intermediate voxel   (∈   ) of the line  , and is defined as Eq.( 4).

𝑝(𝑣
where q gives a decrement parameter indicating how much the observation probability of a laser beam degrades in an unknown voxel depending on the average density of the occupied voxel in the space.By summarizing the values of (  |  ) among all voxels to be measured, the expectation of the priority score of the voxels to be measured (  ) from a scanner position voxel   can be estimated as Eq.( 5) Finally, by evaluating the expectation (  ) , the optimum scanner position for the next scan   is determined as Eq.( 6).
where [] gives a centroid position of a voxel .

SIMULATION RESULTS
In this study, the scanning operation was simulated by sampling the point on the surfaces in a reference CAD model of a piping system shown in Figure 7 which consists of many straight pipes, elbows, junctions and some tanks.And we assumed a standard scan condition (pitch angle increment, etc.) and the scanner dimension of FARO-Focus3D in the sampling.The scanning condition in the simulation is summarized in Table 1.And the parameters used in the next-best scanner position estimations are summarized in Table 2.
As a baseline for comparison in recognition efficiency and accuracy with our proposed method, we also implemented a previous optimum scanner position estimation method proposed by (Potthast and Sukhatme, 2014) where the best scanner position can be found so that the volume of all unknown voxels (e.g.() = unknown) is tried to be minimized as a whole.Their method can be easily implemented in the way that   is composed of all unknown voxels, (  ) = 1.0 ( ∀  ∈   ) and Eq.( 4) is rewritten to Eq.( 7).where we specified  0 = 0.4 and   = 0.1[] in the simulation.

𝑝(𝑣
An identical scanner position of the first scan is specified at the center of the measuring space and is given to the both methods. Figure 8 shows the aggregated scanned point clouds, (  ) distributions and the next best scanner positions after every scan using our proposed method and the previous method (Potthast et al., 2014).As shown in this figure, in the previous method, the Table 1.The scanning condition in the simulation Table 2.The parameter settings in the estimation scanner tends to be placed as far away from the pipes to be measured as possible so as to capture the unknown voxels as much as possible in the next scan.Because the observation probability of a voxel close to the tangled pipes significantly drops in this method, some positions were concentrated to a similar location (lowest right corner), and it eventually caused the inefficient scan operations for recognizing the piping objects.
On the other hand, shown in Figure 8, in the proposed method, the scanner positions relatively diverged into different locations in the measuring space so that the occluded space occupied by the remaining unseen piping objects is minimized.Figure 9 compares the change in the recognition rate of the piping objects in our recognition system (Kawashima et al., 2014) when using the scanner positions derived from both methods.It was clear that our method achieved the better recognition rate than that of the previous one at the same number of scans.While the recognition rates of all elements reached around 100% even at 6 or 7th scan in our method, they did not improve efficiently even after 10th scan in the previous method.
Figure 10 compares the recognition results of piping objects after the 7-th scan.It is obvious that many portions of pipes are still not yet scanned and recognized in the previous method.On the contrary, in our proposed method, almost all objects of the piping system are already recognized and modelled.Finally, table 3 compares the averaged computational time for estimating the next-best scanner position.In the proposed method, the estimation of the optimum scanner position in each scan finished within 5min.On the contrary, because a larger number of unknown voxels must be the checked for evaluating the expectation of the observation probability, the conventional method approximately took 1.4 hours for finding the optimum 85.1 4.0 scanner position in each scan.Therefore, it is difficult to use the previous optimum scanner position estimation method in the practical scanning operations for large-scale environment.
The above simulation results clearly show that our proposed method outperforms conventional approaches in the recognition accuracy, efficiency and the computational time.

CONCLUSION AND FUTURE WORK
A new computer-aided method to find an optimum sequence of next-best positions of a terrestrial laser scanner specialized for as-built modelling of piping systems was proposed.Different from the conventional approaches, in the proposed method, piping objects in the measured point clouds are recognized right after an every scan, local occluded spaces occupied by the unmeasured piping systems are then predicted, and a best scanner position for the next scan is selected so as to minimize these local occluded spaces.From the comparison of the simulation result with that of the previous similar method, it was shown that the proposed scanner position estimation method outperforms the conventional one in the recognition efficiency, accuracy and the computation turnaround time.To our knowledge, our method is the first study which gives a solution of next-best-view problem for terrestrial laser scanning of 3-dimenntional, complex and large-scale plant environments.
As the further work, the proposed method should be applied to the real scanning operations for the validation.The scanner position estimation which ensures some amount of overlap between the scanned point clouds should be considered.The constraints of the incidence angle and measuring range similar to (Soudarissanane, S. et al., 2011) should also be considered as well as the spatial coverage.The efficient spatial management using octree should be used instead of voxels for the measurement of larger-scale plant environments.The dependency of the performance on the initial scanner position selection should also be investigated experimentally.

Figure 1 .
Figure 1.The strategy for the best 2 nd scanner position for the measurement of piping systems

Figure 2 .
Figure 2. Overview of the proposed estimation of next-best scanner position for the measurement of piping systems

Figure 4 .Figure 3 .
Figure 4. Automatic recognition results of piping objects from point clouds of an oil rig(Kawashima et al., 2014)

FigureFigure 6 .
Figure 5.The voxels to be measured and their priority score evaluation

Figure 7 .
Figure 7.The CAD model of a piping system for the simulation

Figure 9 .
Figure 9.The change in the recognition rate of piping objects