Filament Extraction in 3D Printing of Shotcrete Walls from Terrestrial Laser Scanner Data

: This paper presents a method for filament extraction as a step in Shotcrete 3D printing (SC3DP) for quality control using Terrestrial Laser Scanning (TLS) after the printing process. The proposed approach involves comparing a Point Cloud (PC) generated from TLS data with the original design model using Cloud-to-Model (C2M) distance to obtain a coloured deviation map. This deviation map is then rasterized into an image with the same size as the object, with each pixel's colour representing the C2M distance. The method incorporates denoising techniques, such as bilateral filtering, and applies Canny edge detection to identify the filaments’ contour. Morphological operations are used to extract only the horizontal edges relevant to the planned printing path. To connect isolated edges, an algorithm based on the M-estimator sample consensus (MSAC) algorithm is employed to fit lines accurately. The proposed method achieves an average precision score of 76% in detecting printing filaments of a shotcrete wall. The results demonstrate a reliable quality control capability and the potential for early identification of manufacturing-related issues.


INTRODUCTION
To meet the demand for sustainable and efficient design and construction in the built environment, the architecture, engineering, and construction (AEC) industry is constantly evolving.One of the most exciting breakthroughs in this digital transformation is 3D printing (Wolfs et al., 2021).Over the past 15 years, large-scale additive manufacturing technologies have been developed specifically for the construction industry (Xu et al., 2020).However, the Shotcrete 3D printing (SC3DP) method sets itself apart from traditional concrete 3D printing by spraying the material under pressure rather than extruding it in strands (Hack and Kloft, 2020).
A critical step in the 3D printing process is quality control, and integrating automated quality control into the production cycle can enhance productivity (Slepicka et al., 2022).As 3D concrete printing (3DCP) enables rapid construction, maintaining high standards becomes even more important.This necessitates the development of innovative approaches and concepts to replace time-consuming manual quality control processes.Additionally, methods need to be developed to assess the structural characteristics of 3D-printed objects, ensuring that design requirements are met and any potential flaws are identified (Mechtcherine et al., 2022).
The filament plays a critical role in the 3D printing process as its shape can be influenced by various factors including material mix, nozzle distance and diameter, extrusion rate, pumping pressure, robot speed, and airflow pressure, affect its shape (Kloft et al., 2020).In a study by Slepicka et al. (2022), different types of quality inspections were categorized and their application and importance were discussed based on the stage of the printing process.The authors identified filament extraction as a part of feature-wise inspection, which focuses on examining specific features of the printed object.
The term "filament" is adapted from the study conducted by Slepicka et al. (2022).It specifically denotes a single strip of concrete layer produced through one-line of planned printing path (refer to Figure 1a).However, the reason for adopting the term "filament" in this paper is to eliminate any confusion with the term "layer."In Figure 1(c), the term "layer" refers to the additional concrete material added after the printed core.(Slepicka et al., 2022).
Figure 1a illustrates the intended printing path, while Figure 1b showcases the anticipated filament geometry.In contrast, Figure 1c displays the actual state of the filaments after printing a simple concrete wall with dimensions represented by 430x1685 mm (width, and height) respectively.To evaluate the quality of filament, a comparison was made with manually segmented counterparts.Analysing the filament geometry allows for the application of various inspection methods, providing valuable insights into the different parameters involved in path planning.Furthermore, an additional concrete layer can be applied over the printed core to address any inconsistencies between the as-printed object and the asdesigned model, ensuring the desired finished surface is achieved.
This paper focuses on post-printing quality control for filament extraction using Terrestrial Laser Scanning (TLS).Section 2 provides an overview of the current state of the art in filament extraction in additive manufacturing (AM).In Section 3, we present our proposed methodology for filament extraction.The algorithm's results are discussed in Section 4, followed by a comprehensive analysis of the findings in Section 5. Finally, in Section 6, we summarize the paper and outline its key contributions.

RELATED WORK
Quality inspection (QI) plays a vital role in ensuring the geometry and surface quality of objects produced in various manufacturing processes.In the context of 3D printing, several studies have investigated the impact of extrusion printing parameters on filament characteristics.Wolfs et al. (2021) examined tearing and buckling of filaments.Tearing occurs when the layer width decreases in the direction of the nozzle, while buckling happens when the filament width increases in the same direction as the nozzle.Quah et al. (2023) conducted a comprehensive survey on technical parameters and their effects on filaments, categorizing them into process, environmental, and material parameters.Davtalab et al. (2020) employed image processing techniques for filaments detection in 3D printing extrusion.Their method involved edge detection, orientation estimation using the Hough transform, and defect detection based on unexpected orientations.The algorithm showed promising results.However, the algorithm's robustness is uncertain as the images were limited to number of filaments and was tested on clean extrusion 3d printing technique rather than shotcrete.
On the other hand, Senthilnathan and Raphael (2022) proposed a computer vision-based methodology for quantifying textural variations on the surface of concrete 3D-printed filaments.
Their approach involved texture analysis using a 2D camera, enabling continuous monitoring and identification of quality issues.Nevertheless, their texture analysis is based on already manual segmented filaments.Rill-García et al. ( 2022) utilized a camera-based sensor and deep learning architecture to automatically segment filaments for printed objects.The segmentation was based on interlayer line orientation, thickness, and geometrical anomalies.The authors achieved a high F1 score for segmentation accuracy, indicating the effectiveness of deep learning techniques.However, the authors utilize isolated images, each displaying only a small portion of the object.This approach hinders the analysis of the entire filament segment as a whole and may result in issues with overlapping areas between images.Additionally, the method was tested on extrusion 3d printing rather than shotcrete.Wi et al. (2020) employed a structured light scanner (SLS) to assess the printing quality of 3D-printed clay objects.Their method measured various parameters to evaluate printing quality, but it did not provide an automatic inspection approach.Mendřický and Keller (2023) validated a method for quality control in 3D printing using SLS.Their approach involved comparing scans of printed wall segments at different time intervals and analysing deviations from a reference model.While the study did not offer an automatic inspection method, it highlighted time-dependent shrinkage as an important factor affecting accuracy.Furthermore, assessing the quality of the filaments using a digital calliper necessitates expertise.
In the analysis of filaments in 3D printing, various conclusions can be drawn.Excessive velocity occurs when the extrusion nozzle's velocity exceeds the extrusion rate, leading to discontinuous filaments or longitudinal tearing.Over-pressing happens when the pressure from the extruded filament surpasses the strength of the previous one, resulting in staggered filament patterns and a loss of control over filament width and thickness.Flow-out occurs when the material's yield stress is insufficient to support its own weight, causing a poor material deposition as the distance between the printed piece and the extrusion nozzle increases (Rill-García et al., 2022).
Extrudability is an important property for 3D-printing concrete, referring to its ability to pass through a rigid nozzle with high shear and maintain a liquid behaviour.Pumpability, on the other hand, is the concrete mix's ability to be pumped through pipes and flow under pressure without significant changes in its initial properties.Buildability relates to the concrete filament's capacity to withstand the weight of successive filaments without deformations.Inadequate yield stress in the bottom filaments of 3D-printed concrete elements can lead to a reduction in filament thickness or flow away under the weight of accumulated filaments (Senthilnathan and Raphael, 2022).
Overall, these studies demonstrate the importance of quality control and inspection in the 3D printing process to ensure that the final product meets the desired specifications.Computer vision techniques have the potential to automate the QI process, reducing the need for skilled workers and saving time and money.Despite these contributions, there is still a need for more research, particularly in the areas of filament extraction and the utilization of TLS sensors in 3D printing, specifically for SC3DP, which is the primary focus of this study.
We have chosen to use TLS since our goal is to capture the complete object and accurately represent the entire filament and its boundaries simultaneously.While it is also possible to achieve this using single, overlapping images and employing a Structure-from-Motion (SfM) process, including dense image matching and potentially orthoprojection, it would require additional processing time.This could potentially interfere with our objective of achieving near-real-time processing.Moreover, we have a pre-established setup that enables direct coregistration of the point cloud and the model, which is illustrated in our previous paper (Mawas et al., 2022).

METHODOLOGY
Methodology (Figure 2): TLS data capture for Point Cloud (PC); C2M distance for deviation map; aliasing effect considered; bilateral filter for noise reduction; Canny edge detection and horizontal edge extraction; RANSAC-based algorithm for line detection in sliding windows.To employ the C2M algorithm between the as-printed specimen (point cloud) and the as-designed model, an alignment method is required.For detailed information on our registration method, please refer to our previous paper (Mawas et al., 2022).For rasterization, an important requirement is to ensure that the point density is sufficient to satisfy the sampling theorem.Therefore, in order to generate an image of the area of interest from the point cloud, it is crucial to consider the aliasing effect and preserve edge details, particularly the contours of the filaments.Also, the generated image size is equal to the wall dimension to avoid any distortion.Additionally, the pixel values are derived from the C2M distance results, and if any data is missing due to rasterization, denoising using an interpolator such as nearest neighbour or linear interpolator might be necessary.
The bilateral filter is applied to the image afterwards to reduce noise while preserving edge features.It considers the spatial proximity and similarity in photometric values of pixels to determine their closeness.Afterwards, Canny edge detection is employed to identify the edges in the image.The bilateral filter has two fundamental parameters: the range parameter and the spatial parameter.Increasing the range parameter makes the bilateral filter behave more like a Gaussian blur, while increasing the spatial parameter results in smoother and more prominent features (Tomasi and Manduchi, 1998).
Figure 3 illustrates the effect of the bilateral filter on the Canny edge detection results.Figure 3a shows the edges without the bilateral filter, while Figure 3b shows the obtained edges after applying the bilateral filter.It is important to note that the path planning for layers in shotcrete 3D printing is only horizontal (see Figure 1).However, Canny's edge detection captures not only horizontal edges but also isolated islands within the filaments (see Figure 3b).These topological islands arise from the nonsmoothness of filaments in shotcrete 3D printing and also from the C2M distance colour map.As a result, certain regions within a filament may not be homogeneous.Therefore, minimizing the effect of edges between filaments is desirable.To extract only the horizontal edges, a morphological operation involving erosion followed by dilation is applied using a horizontal kernel of size 1x3 with all elements set to one.Nevertheless, the edges obtained through Canny edge detection algorithm appear isolated and unconnected.To overcome this limitation and go beyond detecting only straight lines with the Hough transform, we propose utilizing the RANSAC-based algorithm for polynomial fitting.Unlike the Hough transform, RANSAC can fit any desired polynomials of varying degrees, enabling the detection of shapes beyond straight lines.This flexibility allows for future adaptability in different path planning scenarios.
In our specific case of path planning, which involves exclusively straight lines, we utilize the M-estimator sample consensus (MSAC) algorithm (Torr and Zisserman, 2000) for connecting the pixels along the filament's edge.MSAC is an extension of the RANSAC algorithm that introduces robust M-estimation techniques.Unlike treating all inliers equally, MSAC assigns weights to the inliers based on their residuals, which improves the accuracy of model estimation by reducing the impact of outliers.Through the line-fitting process, we identify the inlier points extracted from the Canny edges' algorithm.This enables us to obtain edges that precisely follow the contour of the filament, surpassing the limitations of a simple straight-line representation.

RESULTS
The line detection algorithm is applied to every patch through a sliding window approach across the entire image.
The window size should match the defined layer height for the planned printing path to ensure that the pixels grouped in each window belong to a single filament's edge with a width that matches the wall width.Figure 4 presents the results of our methodology.Figure 4a displays the extracted horizontal lines obtained from the Canny filter.In Figure 4b, the inlier points from the line detection using the MSAC algorithm are depicted.Finally, Figure 4c shows the extracted edges of the filaments after connecting the points that belong to one fitted line.
The results of comparing the segmented filaments from the images with the manually labelled data are presented in Figure 5. Figure 5a shows the filaments labelled manually, while Figure 5b displays the filaments extracted by our proposed algorithm.The manual labelling process involved creating a polygon for each filament with a unique label using CVAT. 1 From the manual labelling, we obtained a total of 29 filaments, while our method detected 28 filaments.To assign the correct labels to the detected filaments, computed the Intersection over Union (IoU) score between the different layers.The IoU score measures the degree of overlap between two layers, with a score of 1 indicating a perfect match.1. IoU score between the different layers.

Label
Table 1 presents the IoU scores between the algorithm's detected filaments and the manually labelled ones.We observe some critical matches, indicated by lower IoU scores.To determine the correct matches, we select the highest IoU score among the matches.For example, manually labelled filament 3 has a higher IoU score with algorithm filament 3 compared to algorithm filament 4, so the match with the highest score is considered.Also, from the table, we found that filament 4 from the algorithm is an outlier since it has no high score.Also, there are two missing matches for the manual filaments 18 and 25.
The precision, recall, and F1-score for the different filaments were computed to further analyse the algorithm's performance, as shown in Figure 6.TN, TP, FP, and FN represent true negative, true positive, false positive, and false

DISCUSSION
The method aims to detect filaments for a whole component printed object using the 3D shotcrete method to perform post-print quality inspection.The studied wall has a width and height of 430x1685 mm.The results achieve a maximum precision score of 92%.However, it is evident from the data that when one filament is missed by the algorithm, the accuracy of neighbouring filaments is dramatically affected.This limitation arises from the use of a fixed sliding window height.In future research, it is crucial to adapt the sliding window based on the structure of the filament.Also, the issue with the fixed-height sliding window becomes particularly evident at the beginning and end of the filament, where the material bulking effect is most pronounced (refer to Figure 6 and Figure 7).One approach to achieve this goal is by taking vertical profiles from different parts of the image.The peaks and troughs of the profiles represent the edges of the filaments.Thus, the sliding height and width can be adaptively determined based on the distances between one vertical line and its neighbour.The height can be defined as the distance between the trough and its next trough from the vertical profile.However, it is important to note that the used holistic extraction of one filament will affect the result of the neighbouring extraction, as illustrated.If these effected filaments are not eliminated, the mean precision score reach up to 76%.
One challenging situation for the algorithm is when the layers are not continuous, as shown in Figure 7.The best solution to overcome this phenomenon might be to use a deep neural network model for filament detection.However, it is worth mentioning that even for humans, detecting this phenomenon is a challenging task as well.Overall, the current method demonstrates promising results in filament detection, but further refinements and adaptations are necessary to improve its robustness and generalization to different filament structures and non-continuous filament patterns.Incorporating advanced techniques like deep learning could potentially enhance the algorithm's performance in handling complex scenarios.
Figure 1.(a): Planned printing path of the simple straight wall, (b): Expected print result, (c) Extraction of the geometry of filaments and workflow quality inspection from printed core to finished surface, (a) &(b) adapted from

Figure 2 .
Figure 2. Flowchart of Our Methodology

Figure 4 .Figure 5 .
Figure 4. Results of our Methodology.(a) Extracted horizontal lines from the Canny filter.(b) Inlier points from line detection using the MSAC algorithm.(c) Extracted edges for every detected filament.

Figure 6 .Figure 7 .
Figure 6.The results score for the different layers between the proposed algorithm and manual labelling 1 https://www.cvat.ai,AccessedMay 29, 2023.negative,respectively.Precision measures the proportion of correctly predicted filament (true positives) out of all the detected filaments by the algorithm (true positives + false positives).It indicates how well the algorithm captures the true filaments and avoids falsely labelling non-filaments.Recall, also known as sensitivity or true positive rate, measures the proportion of correctly predicted filaments (true positives) out of all the filaments in the manual labelling (true positives + false negatives).It indicates the algorithm's ability to detect all the filaments present in the manual labelling.F1-score is the harmonic mean of precision and recall, providing a single measure that combines both metrics.It gives equal importance to both precision and recall.A higher F1-score indicates a better balance between precision and recall.In this case, precision is particularly relevant because the filaments in the image are thin but have the same width as the entire image.A high precision score would indicate that the algorithm effectively extracts the true positive filaments without falsely labelling other parts of the image as filaments.