Mobile LiDAR-based Real-time Identification of Transmission Lines

This paper proposes a method for identifying 3D point cloud of transmission line acquired by light detection and ranging (LiDAR) real-time mobile scanning. Since the single frame of point cloud obtained by LiDAR is sparse, the method employs a sliding spatial window strategy with Kalman filtering for dynamic point cloud registration. Then, a 3D point cloud deep learning neural network that utilizes uniform sampling and local feature aggregation (LFA) is designed specifically for transmission line objects. The network handles the problem of long-span objects and a large amount of point cloud. Finally, the instantiated transmission line objects are extracted from the top-down projection of the semantically segmented 3D point cloud by fast Euclidean clustering algorithm. Experiments demonstrate that the method achieves a classification accuracy of 94.7% and a mean intersection over union of 81.6% on 3D point cloud datasets of transmission line obtained from LiDAR mobile scanning, validating its ability to achieve real-time identification and distance measurement of transmission line objects.


Introduction
Contacting energized transmission line may cause fatal electrocutions or equipment damage, especially for cranes and lifting equipment (Matikainen et al., 2016).Real-time measurement of the distance between construction personnel or equipment and transmission line, with timely alarms issued when the distance falls below a safety threshold, can enhance construction safety.However, transmission line is typically erected in complex outdoor environments with factors such as trees, vegetation, and varied topography (Li et al., 2021).Additionally, transmission line is often composed of hollow pylons and slender power lines (Ma et al., 2021).Therefore, realtime distance monitoring of transmission line is challenging.
Video surveillance-based methods are comparatively widespread using various cameras.Monocular (Hui et al., 2019) or binocular (Huang et al., 2020) cameras were typically employed in these methods to capture visual images of the target scene.However, due to the limitations of camera imaging principles, changes in lighting conditions significantly impact detection results.Moreover, the length of power lines between pylons is typically very large.Long-term similar power lines images can easily lead to mismatches between views (Zhang et al., 2020).
Light detection and ranging (LiDAR) scanning technology provides a 3D measurement approach different from visual images, with strong adaptability to lighting conditions.In recent years, it has been widely researched and used in the field of transmission line.However, automated identification of transmission line from LiDAR 3D point cloud faces numerous problems.Initially, model fitting based on features of spatial topology were involved in segmenting transmission line (Guan et al., 2016;Lin and Zhang, 2016;Ortega et al., 2019;Zhang et al., 2019).These methods first utilized LiDAR point cloud to reconstruct the 3D scene, locating the positions of pylons based on features such as height and point density.The 3D point cloud was then projected onto a 2D plane, where the power lines approximated 2D catenary lines.Finally, algorithms such as feature clustering, Hough transform, or catenary line fitting were employed for power line extraction and fitting.However, these methods directly reduce the original 3D point cloud data to 2D space for analysis, leading to the information loss of one dimension.And they cannot address the impact of mutual occlusion in the projection direction.Therefore, these methods exhibit poor versatility and are more suitable for environments with flat terrain and fewer trees.
The research of deep learning technology brings a new turn to identification of transmission line from 3D point cloud.A point cloud convolution module based on directional constraints was proposed (Wen et al., 2020).This module effectively segments point cloud by extracting 3D local features from the 2D projection of the point cloud.Building on this, the method further designed a multi-scale convolutional neural network to combine with up and down sampling techniques.The PointNet++ algorithm (Qi et al., 2017) was improved (Huang et al., 2023), addressing feature such as large target structures and sparseness in transmission line point cloud.The enhanced algorithm has a larger receptive field, a finer network, and uses the focal loss function (Lin et al., 2017) to balance loss calculation results, thereby improving accuracy.However, most deep learning-based point cloud identification algorithms for transmission line applied currently rely on offline data (Chen et al., 2022).They cannot meet real-time distance measurement requirements.
We propose an effective measurement approach by combining simultaneous localization and lapping (SLAM) and deep learning for real-time transmission line detection and measurement.By exploiting the spatial topological features of transmission line, we construct a semantic segmentation model suitable for transmission line and training the model with a large amount of outdoor transmission line point cloud data.For transmission line, a single-frame point cloud is too sparse and often incomplete.It will impact the effect of deep learning network for objects identification and distance measurement.To address this issue, we design a local point cloud map dynamic updating method based on SLAM.Furthermore, to meet the real-time requirements of encrypted point cloud, we optimize the structure of the deep learning model to enhance detection speed and accuracy.

Methodology
The proposed method aims to extract the point cloud belonging to transmission line from point cloud data stream of LiDAR mobile scanning, in a robust and real-time efficient manner.This method innovatively combines SLAM and deep learning model to comprehensively process point cloud.An overview of the proposed methodology is shown in Figure 1.

Dynamic Registration and Mapping
The structure of transmission line is long and slender.It causes the point cloud of transmission line easy to be missing.We propose a point cloud enhancement method based on sliding spatial windows registration.This method can generate a point cloud of the local area around observation points, constraining the field of view, and increasing the number of scanned points on the target structures of transmission line, thus mitigating the occurrence of missing data.

Error state Kalman Filter for Registration:
To obtain the optimal registration parameters between each frame of point cloud data from LiDAR real-time scanning, a fast yet erroraccumulation-negligible registration algorithm is required.Drawing inspiration from the SLAM algorithm FAST-LIO2 (Xu et al., 2022), we integrate inertial measurement unit (IMU) and LiDAR sensor using an error-state Kalman filtering (ESKF) algorithm (Roumeliotis et al., 1999) to establish a motion model of the mobile scanning in a large-scale outdoor scene.
Since the Lie group formed by rotation matrices and displacements is not closed for addition but closed for multiplication, we define the operators ⊕ and ☉ .where ⊕ represents the exponential mapping from Lie algebras to Lie groups.And ☉ then ⊕ the inverse process of, represents the logarithmic mapping from Lie group to Lie algebra.
First set the status [ , , ] represents the position of the platform, [ , , ] the speed of the platform, and [ , , , ] w x y z q q q q q = represents the attitude of the platform in the form of quaternion.Combined with IMU data, we establish the state equation of the motion model: Where, [ , ] x z w y y z w x w x y z q q q q q q q q q q q q R q q q q q q q q q q q q q q q q q q q q q q q q q (3) In addition, the subscripts in the text k all refer to the k iteration.Different from the ordinary Kalman filter that focuses on the state quantity, the error-state Kalman filter models the state error of the motion process.Establish the error-state equation and observation equation of the current frame: Among them, x , x and x δ represent the true value, estimated value and the error between the true value and estimated value of the current state respectively.We believe that position, velocity and IMU data are directly observed, so the observation function h is a unit function.
establish the maximum posterior estimation model: ( ) Introducing the Kalman gain K and δ can be solved: Next, we update the estimated value , put the new estimated value 1 ˆk x + back into equation ( 6) for calculation, and loop iterate until the change is less than the convergence threshold σ , i.e. .After that, the globally optimal registration parameter x of this frame and its corresponding covariance matrix P can be obtained: Finally, calculating the registration transformation matrix by x to project the current frame point cloud to achieve registration between the current and the existing point cloud.

Identification of Transmission Lines
This paper builds a lightweight deep neural network model suitable for large-scale 3D point cloud segmentation, specifically tailored for the target point cloud types in transmission line, utilizing the uniform down-sampling (UDS) and local feature aggregation (LFA) modules.Figure 3 illustrates the encoderdecoder design of the deep neural network architecture proposed in this study.In the encoder, LFA is applied before each layer to propagate points to surrounding points, preventing information loss.Subsequently, UDS is employed for point cloud downsampling to reduce computational complexity.Between corresponding encoding and decoding layers, multi-layer perceptron (MLP) layers with shared parameters are utilized for point feature mapping.Both encoder and decoder consist of 5 layers to learn features from more points.Finally, Dropout layers and fully connected layers are utilized to prevent model overfitting and predict semantic labels for each point.Transmission line facilities typically span large distances.If the model's receptive field is too small, it may weaken the connection between features, leading to reduced recognition accuracy.To prevent the loss of important point information during UDS, the deep network adopts the LFA module to expand the receptive field of each point before down-sampling.This allows each point to contain information from surrounding points, reducing the probability of losing important point data.
Each point feature learning layer of the neural network model utilizes the LFA module for local spatial encoding.The LFA module explicitly encodes the 3D coordinates of the input point cloud to enhance the association of spatial geometric features, enabling the network to better learn the spatial geometry from the relative positions and distance information of each point.Moreover, an attention mechanism is employed to automatically learn and aggregate useful information from the nearby point feature set.By treating the learned attention scores as a flexible mask that automatically selects important features, the resulting features are the weighted sum of these nearby feature points, enhancing the aggregation capability of the nearby point feature set.This approach enables the network model to fully leverage nearby point information, expand the propagation range of point cloud features, broaden the local feature receptive field, and facilitate better learning of large-scale point cloud structures such as transmission line.

Instantiation Segmentation by Euclidean Clustering
Deep learning network has the inherent limitations in point cloud identification, especially in large-scale scenarios requiring precise distance measurements.This paper designs a postprocessing method based on the fast-Euclidean clustering algorithm to mitigate these challenges.The proposed method aims to address the noise introduced by radar hardware and misidentified point cloud, thereby enhancing the overall accuracy and robustness of the system.Moreover, the method incorporates instance segmentation specifically tailored for transmission line point cloud, identifying independent objects and further improving its robustness in complex scenarios.
Due to the numerous and expansive characteristics of transmission line point cloud, a fast-Euclidean clustering algorithm has been designed.We also consider the spatial distribution features unique to the point cloud.The algorithm's detailed procedure is as follows: (1) Overhead projection: Projecting the 3D point cloud results of semantic segmentation onto the xy -plane.
(2) Randomized Initialization and Labelling: Selecting a random initial point 0 a and identifying nearby points within a distance threshold d .Both the nearby points and the initial point 0 a are labelled as label 0.
(3) Sequential Point Labelling: Choosing another random initial point 1 a and finding nearby points within the distance threshold d in the remaining point cloud.If none of the nearby points have been labelled, both the nearby points and the initial point 1 a are labelled as label 1.If some nearby points have been labelled, both the nearby points and the initial point 1 a are labelled with the same label as the previously labelled point.In case there are multiple labelled points nearby, the label assigned to the nearby points and the initial point 1 a is the same as the label of the point with the smallest label among them.
(4) Iterative Clustering: Iteratively executing Steps (2) and ( 3) until all points in the point cloud have been assigned labels.Points sharing the same label are considered part of the same point cloud cluster.
For each segmented point cloud cluster, those that do not conform to the topological structure of power lines or pylons are removed based on their point quantity and spatial distribution.Finally, we integrating positional and motion information from consecutive frames to calculate the closest distance between each point cloud cluster in the scene.This approach contributes to the refinement and robustness of the proposed method for the recognition and distance measurement of transmission line objects in challenging scenarios.

Experimental Dataset and Environment
The experiment utilizes LiDAR 3D point cloud data from three different voltage-level transmission line in Nanjing.These data are collected using handheld scanning with the Livox AVIA LiDAR system by DJI.The total length of the three transmission lines is 4.9 km, involving voltage types of 110 kV, 220 kV, and 500 kV, with a total point cloud count of approximately 31.8 million.Among these, Figure 5A shows dataset A from openfield transmission line, Figure 5B presents dataset B from transmission line within complex scenes, and Figure 5C is dataset C from transmission line within longer scenes.
Our experimental platform is equipped with an Intel Core i7-9700 @ 3.00GHz CPU, an NVIDIA GeForce GTX 1660 TI GPU, and 16 GB of RAM.During training, we opt to allocate 80% of each dataset for training purposes, reserving the remaining 20% for testing.The distribution of various labelled points within the dataset is illustrated in Figure 6.

Qualitative and Quantitative Analysis
Figure 7 shows the identification results of proposed method on three datasets.
Figure 7. Identification results of our method.In order to evaluate the identification results and ranging accuracy of power lines and pylons more accurately, the experiment uses overall accuracy (OA) and mean intersection over union (mIoU) as the evaluation criteria for recognition accuracy.At the same time, the mean absolute value error (MAE) and the frame rate per second (FPS) are used to quantitatively analyse the ranging accuracy and speed.The results of the quantitative analysis are shown in Table 1.Our method can obtain an overall accuracy of more than 90 % on three datasets, and the processing efficiency of 3.2 Hz can also meet most mobile measurement needs.
Table 1.Quantitative analysis results on different datasets.

Comparisons
In order to fully verify the reliability and advancement of the method we propose, the experimental evaluation further compared the positioning accuracy and identification accuracy of this method with the current advanced PointNet algorithm (Qi et al., 2017), PointNet++ algorithm (Qi et al., 2017) and KPConv algorithm (Thomas et al., 2019).As shown in Table 2, the OA of our algorithm is 94.7% and the mIoU is 81.6%, both of which are better than other methods.This further demonstrates the algorithm's advancement in recognition accuracy.At the same time, the accuracy of recognition also determines the accuracy of transmission line distance measurement to a large extent.The MAE between the object distance measured through the segmentation result of the methodology we propose, and the true value is 0.33m, which is smaller than other algorithms.It shows that this algorithm can meet the safety measurement needs of transmission line construction sites.In addition, considering that PointNet++ and KPConv algorithm consume huge memory resources during sampling, the algorithm we propose uses UDS processing method, so it has certain advantages in processing efficiency.

Conclusion
This paper proposes a method for identification of transmission line based on 3D point cloud scanned by LiDAR in real-time.We present a novel design of a sliding window based on Kalman filtering strategy for point cloud densification, aiming to collect comprehensive and dense point cloud of transmission line.Furthermore, a semantic segmentation neural network, utilizing 3D point cloud as input, has been tailored for power lines.This network meets real-time requirements and exhibits excellent robustness across various transmission line.The experimental results affirm the method's ability to rapidly and accurately identification and measurement in transmission line operational environments.Ongoing research efforts will focus on further subdividing and modelling 3D point cloud of transmission line.
Additionally, the investigation of real-time identification in adverse weather conditions for transmission line remains a crucial aspect of our future research agenda.

Figure 1 .
Figure 1.An overview of the proposed methodology.2.Methodology

=
provided by the IMU, and { } R q is the transformation from quaternion to rotation matrix.

(
k J and k H are the Jacobian matrices corresponding to the Taylor expansion of the state equation and the observation equation at.0 k x δ = .In addition, the error-state parameter x δ and error-observation parameter υ satisfy Gaussian distribution.Their corresponding covariance matrices are P and respectively Q .Let ˆk Mapping: To address the incremental expansion challenge in 3D point cloud mapping, this study employs a sliding spatial window-based strategy to dynamically manage local maps, removing point cloud data distant from the area of interest to enhance data utilization efficiency.Figure 2 illustrates the schematic diagram of dynamic local map updates, incorporating two key distance threshold parameters L and l , where L l > .These thresholds represent the range threshold of the interest region and the maximum scanning radius of the LiDAR.To meet the real-time distance measurement requirements for power lines, a distance threshold L for the region of interest is established.Only objects within a plane radius of L from the scanning centre are considered objects of interest for measurement.As depicted in Figure2, the points C and ' Crepresent the previous and current scanning centre points, while the orange and blue cycle regions illustrate the dynamic update of the local point cloud set before and after updating.The centre positioning point plays a role like that of keyframes in the field of image and video processing.However, the centre positioning point retains the coordinates of the centre position, while 3D points around this centre are incrementally added to increase the map density.When the green circular region is inscribed with the blue circular region, that is, the detection boundary touches the boundary of the region of interest, the region of interest update is triggered.The new region of interest is a circular region with a radius L cantered on the point o where the radar is currently located.

Figure 2 .
Figure 2. Schematic diagram of dynamic update of local point cloud collection.Then, the radius of the new positioning point is used to select the incremental mapping of the local 3D point cloud.As the measurement centre moves and triggers the boundary of the centre point for scanning, the local point cloud set is clipped according to a fixed radius constraint, avoiding the continuous incremental data increase of traditional SLAM and the computational burden of frame-by-frame point cloud boundary clipping.This local point cloud registration and enhancement method ensures an adequate number of scanning points for slender structural objects and provides support for the real-time performance of subsequent identification and measurement algorithms.

Figure 3 .
Figure 3. Structure of the classification neural network.(N, D): Dimensions of points and features; FC: Full convolution; LFA: Local feature aggregation; UDS: Uniform down-sampling; MLP: Multi-layer perception; US: Up sampling; DP: Dropout) Classical deep learning methods for point cloud segmentation typically rely on farthest point sampling(Qi et al., 2017)  or inverse density importance sampling(Groh et al., 2018) to mitigate the computational impact of point cloud quantity.However, their time complexity is 2 ( ) O n and ( ) O n , respectively, resulting in increased computational complexity with growing point cloud quantity.In contrast, the UDS strategy divides the space into N cubes with volume x y z ∆ × ∆ × ∆ and selects at most one points from each cubes, with computational complexity dependent solely on the down-sampling cube number N .Additionally, it can homogenize the input point cloud density to enhance network robustness.Therefore, the proposed network model in this paper utilizes global UDS in all 5 encoding layers to progressively reduce the point cloud quantity, achieving sparsity on the original point cloud and enhancing computational efficiency.

Figure 4 .
Figure 4. LiDAR point cloud data collected in the experiment.

Figure 6 .
Figure 6.Statistics of the number of each label in dataset.

Figure 5 .
Figure 5. LiDAR point cloud data collected in the experiment.
the TP means number of true-positive points, FP means number of false-positive points, FN means number of false- negative points, M means total number of points, n is the number of frames, i D and ˆi D represent the actual and predicted distance between LiDAR and transmission line objects.

Figure 8
Figure 8 shows the comparison of results between the algorithm proposed in this paper and other advanced algorithm when conducting experiments on the same dataset.In the comparative experiment, dataset A and dataset C are used as the training set, and dataset B is used as the test set.

Figure 8 .
Figure 8.Comparison of recognition results of different algorithms.

Table 2 .
From Figure6, we can find other algorithms have obvious misrecognition of local point cloud.Both the PointNet and PointNet++ algorithms incorrectly label some power lines and pylons points with each other; while the KPConv algorithm identifies some road points as other points, and some points on the top of the pylons are labelled for power line.Due to the limited receptive field of points, these algorithms ignore the large-scale correlations between 3D point cloud features, resulting in severe limitations in feature diffusion and the inability to establish connections between local features.In contrast, our algorithm effectively aggregates point cloud features of transmission line through multi-layer LFA modules, thereby enhancing the correlation between neighbourhoods.In addition, the algorithm also re-segments the data segmented by the deep learning network based on the spatial distribution relationship and topological structure of the transmission line, further reducing the misrecognition rate of point cloud.The quantitative analysis results of several comparative methods are shown in Table2.Quantitative analysis results of different algorithms.