POINT CLOUD REGISTRATION USING LASER DATA FROM AN ORANGE ORCHARD

: Mobile laser systems have been used for close-range applications, such as agricultural management, due to the high penetrability of the laser beam enabling a unique 3D representation of the plant structure. However, some challenges remain in generating 3D mapping in agricultural environments. For instance, orange orchards are composed of trees with dense canopies which the laser scanner cannot penetrate for stem mapping. Therefore, most of the laser point clouds are pulses reflected from leaves and ground. This paper analyses approaches to detect keypoints, extract features and register the point clouds acquired with a laser scanner in an orange orchard. Three keypoints extraction methods (Uniform Sampling - US, 3D Harris and 3D Intrinsic Shape Signatures – 3D ISS), SHOT feature extraction and global registration methods (ICP and CPD) were evaluated. The results showed that improvements are still necessary for point cloud registration for orange orchards. In feature-based matching methods, an approach to filter the mismatches is needed to improve the estimation of the translation and rotation parameters. In the global registration methods, initial values of the trajectory are needed for ICP. CPD achieved good results for five sequence scans without initial values.


INTRODUCTION
Mobile mapping technologies (MMT) are a promising tool for agricultural management and smart farming.Mobile mapping data can directly contribute to autonomous farming operations, such as machine auto-driving, seeding, harvesting, plant health and growth monitoring.Notably, laser scanning mobile mapping platforms have been widely applied for the three-dimensional analysis of crops (Escolà et al., 2017, Zhang et al., 2020, Mahmud et al., 2021) since the good penetration of the laser beam enables a unique 3D representation of the plant structure (Wehr and Lohr, 1999).However, incorporating MMT in everyday farming operations with the required accuracy is proving to be challenging.One of the main issues is the features' similarity, which directly affects keypoint detection, point cloud registration and platform positioning estimation.Here, we discuss the challenges of keypoint detection and matching in laser point clouds acquired by mobile mapping platforms at homogeneous agricultural environments, such as orange fields, with a focus on point cloud registration.
Laser scanning mobile mapping platforms acquire laser data from different positions and orientations while the platform moves.The alignment of these individual scans to a common coordinate system is required to obtain a complete representation of the scene.This process is well-known as point cloud registration.Consecutive point clouds can be registered by estimating the parameters of a rigid body transformation (rotations and translations) between clouds and then applying the transformation to the scans (Vosselman and Maas, 2014).The estimation of the rotation and translation parameters between two or more laser point clouds needs corresponding points, which can be measured manually or automatically.Manual tie point selection can be quite efficient for small datasets.However, it becomes laborious for large data, complex scenes, and real-time applications, emphasising the need for an automated detection algorithm.
A keypoint should have repeatability and distinctiveness (Aldoma et al., 2012).The first refers to the capability of a detector to consistently extract the same keypoints in sequences of point cloud acquisitions.The latter is the ability to detect points that are easily described and matched.Challenges for automated feature detection in point clouds among others are homogeneity, occlusions in complex scenes, noise, and variable point densities caused by different scanning distances (Yang et al., 2018).Some works have presented strategies to cope with these problems by locating distinctive and robust 3D keypoints, extracting their features, and performing 3D matching for point cloud registration.
Urban and indoor environments contain distinctive features that can be used in point cloud registration.These features may include buildings, corners, road networks, and transport facilities, which can provide points, planes, and surface features (Cheng et al., 2018).Arastounia and Lichti (2021) presented an approach for identification and point cloud registration using poles.Forest and agricultural environments are more challenging for point cloud registration due to the complexity of the field features.In these environments, laser data can be noisy, and a few features are available for matching, which has motivated the development of recent solutions for point cloud registration using forest and agricultural features.Liang and Hyyppä (2013) proposed a method for point cloud registration using tree stem locations as registration features.The method was evaluated in a point cloud obtained with a static terrestrial laser scanner in managed boreal forests.Kukko et al. (2017) presented a method for point cloud matching also using stem and ground observations for mobile mapping systems.Yuan et al. (2022) proposed a variant ICP (Iterative Closest Point) for the registration of point clouds collected in a peach field.
However, tree stems are not always available in some agricultural environments, such as orange fields.Mature orange trees have a dense canopy covering the whole tree (see Fig. 2 and Fig. 3).Points reflected from the leaves populate a large portion of the data but are not suitable for solving an accurate registration with the existing methods.Therefore, a feasibility study of keypoints detection and matching in such a challenging dataset for mobile mapping laser point cloud registration is still needed.In this work, Uniform Sampling (US) (Aldoma et al., 2012), 3D Harris (Sipiran and Bustos, 2011) and 3D ISS (Intrinsic Shape Signatures) (Zhong, 2009) keypoint detectors were investigated to evaluate the performance in an orange orchard dataset.The laser point clouds were collected by an Ouster OS0-128 laser scanning mounted in a backpack mobile mapping system.

BACKGROUND
2.1 Solid-state LiDAR sensor: Ouster laser scanner Solid-state sensor technology made feasible the introduction of laser scanning systems with only a single laser unit generating lightweight and compact systems, which have been currently employed for backpack mobile mapping systems (Castanheiro et al., 2022).The Ouster system is composed of vertical-cavity surface-emitting lasers (VCSEL) and a CMOS sensor with single-photon avalanche diodes (SPADs) to detect the returned pulse (Pacala, 2018).VCSEL is a technology for laser beam emission, which is a small generating compact sensor.SPAD is a solid-state photodetector that generates binary data when a returned pulse is detected.SPAD is incorporated into a silicon CMOS sensor, replacing many analogue components.The Ouster laser scanner is also composed of micro-optics that direct the laser light.Castanheiro et al. (2022) presented an overview of Ouster laser scanning systems.Fig. 1 shows the optical module of Ouster laser scanners, in which (a) illustrates the internal architecture and (b) the set of micro-optics.

Point cloud registration methods
Point cloud registration can be performed by feature-based or global methods.In this work we evaluated the performance of the global ICP method and CPD (Coherent Point Drift), and featurebased approaches combining 3D keypoint detectors Uniform Sampling, 3D Harris and 3D ISS with the 3D feature extractor SHOT, aiming registration of consecutive point clouds acquired at orange fields.
The most popular method of point cloud registration is the ICP, which has shown great potential for outdoor environments because it does not require keypoints and feature extraction (Besl and McKay, 1992).Another method is CPD (Myronenko and Song, 2010), in which rigid, affine, and non-rigid transformation point set registrations are supported.Implementation for most of these methods can be found in PCL (Point Cloud Library) (Rusu and Cousins, 2011), which has open-source codes to handle point cloud datasets.Furthermore, some of them were also implemented in MATLAB.
Feature-based registration requires 3D keypoints extraction, 3D feature extraction, keypoint matching and the estimation of rigid body transformation.The 3D keypoint detector methods tested in this work (US, 3D Harris and 3D ISS) are briefly presented as follows.US keypoint detector builds a 3D voxel grid in the point cloud and takes the average point inside the voxel as keypoint.
The resulting point cloud is downsampled in a uniform way.The 3D Harris detector is an extension of the 2D Harris corner detector (Harris and Stephens, 1988).The keypoints are detected by taking the normals to the input point cloud.Points within a certain neighbourhood are selected, and then a covariance matrix of the normal is calculated.A value is calculated for each neighbourhood and its normal, based on the determinant and trace of the covariance matrix, similar to the 2D Harris operator.Finally, a local maximum suppression step is applied, and the resulting points are the keypoints for the input point cloud.3D ISS keypoint detector uses the magnitude of the smallest eigenvalue and the ratio between two successive eigenvalues.
After the 3D keypoints extraction, 3D feature extraction can be performed by applying different methods, such as Point Feature Histogram (PFH) (Rusu et al., 2008), fast PFH (FPFH) (Rusu et al., 2009) and Signature of Histogram of Orientations (SHOT) (Tombari et al., 2010).We evaluated SHOT in this work with the keypoint detectors.SHOT combined signatures and histogram features.The 3D descriptor is based on obtaining a local reference frame using the eigenvalue decomposition around a point.A spherical grid with an origin at this point divides the neighbourhood.Then, a weighted histogram of normal is determined in each grid, introducing geometric information concerning the location of the points.
Most of the studies that evaluated 3D keypoint detectors and feature extraction have used benchmark point clouds, such as those from Stanford 3D Scanning repository (Stanford Bunny, the Happy Buddha, the Dragon, and the Armadillo), that do not represent real natural world objects (Tombari et al., 2013, Stancelova et al., 2020).Only recently, some works addressed the challenge of 3D point detection and feature extraction in mobile mapping point clouds acquired in indoor and outdoor environments.For instance, Ghorbani et al. (2022) evaluated the use of 3D keypoints detectors (i.e., 3D SIFT and 3D ISS) and SHOT method to register sequential indoor and outdoor point clouds.In conclusion, the authors highlighted as the main challenges for point cloud registration the quality, quantity, and spatial distribution of the extracted keypoints, and the matching of features extracted from homogeneous point clouds.This is because point cloud registration requires the estimation of the rigid body transformation parameters (Equation 1) using matching points found based on extracted features.The matching points can be obtained by comparing the resulting features based on a matching metric, such as Euclidean distance.Therefore, the features extracted from homogeneous point clouds, such as forest and agricultural environments, can be similar due to repetitive patterns, leading to false-positive matches and inaccurate estimation of 3D rigid body transformation.

Backpack mobile mapping system
The backpack mobile mapping system (Fig. 2 a) is composed of Ouster OS0-128 (Pacala, 2020) laser scanning system, a Ricoh Theta S dual-fisheye camera, Ublox GNSS receiver, a computer, a WiFi router and a 6S LiPo battery for power supply (Fig. 2 b).
Regarding the aim of this work, only Ouster data was considered.The OS0-128 VCSEL simultaneously emits 128 pulses arranged in four vertical columns at an azimuthal FoV of 22° (Fig. 2 c).
The laser scanner's vertical and horizontal FoVs are 90° and 360°, respectively, and can be classified as a wide FoV sensor.
Table 1 shows the technical specifications of the OS0-128 laser scanner system.The Ouster laser scanners have an internal IMU with three-axis gyroscopes and three-axis accelerometers.The internal Ouster IMU (InvenSense ICM-20948) data is recorded at 100 Hz frequency.A Dell OptiPlex 3070 computer was used to collect the Ouster data.

Test area and data acquisition
The experimental data were collected in an orange orchard (Fig.

Data processing
All experiments were performed with algorithms implemented in C++ in Visual Studio using PCL functions and MATLAB.The US, 3D Harris and 3D ISS keypoint detectors were implemented in C++ using PCL (Rusu and Cousins, 2011).These three methods use only 3D point coordinates.There are other methods available in PCL that use laser intensity stored in the point cloud data, but their use was left for future work.
First, the keypoints were detected in five sample point clouds with each one of the selected methods.The repeatability of the keypoint detection was evaluated as presented by Salti et al. (2011).A keypoint detected in a point cloud is transformed with three (3) rotations and three (3) translations to another point cloud, based on the initial alignment.A transformed keypoint is considered repeatable if the 3D distance to the nearest neighbour is less than a threshold.The threshold was set at 20 cm, based on the angular resolution and the average distance from the trees to the Ouster sensor.After that, the matched keypoints were used to estimate the transformation matrix by the Least Square Method (LSM).This result was compared with the transformation matrix estimated with the point cloud registration tool in CloudCompare ( 2016), used as a reference.
The SHOT feature extractor was used with the keypoints that presented the best results.Then, the 3D matching was performed by calculating the Euclidean distance.The estimated transformation matrix was compared with the reference, as mentioned above.Aiming to evaluate the true matches, the 3D distances between the matched keypoints were calculated and compared to the specified threshold, as mentioned before.Then, the rigid body parameters were again estimated by LSM.
The point cloud registrations with ICP and CPD were performed in MATLAB.Ouster system provides different vertical and horizontal resolutions for the resulting point clouds, as shown in Fig. 4 a.Therefore, the laser point clouds were downsampled, aiming to uniformalise the point cloud and improve the performance of the point cloud registration with ICP and CPD.
In this case, no initial values were used.The CPD approach was applied considering 3D rigid body transformation.The transformation matrix obtained with each method was compared with the reference transformation matrix.The point clouds were registered with the first scan as the origin.

Keypoints assessment
The parameters in the keypoint extractor algorithm were set aiming to extract approximately 100 points.On average, 137, 65 and 164 keypoints were obtained with US, 3D Harris and 3D ISS, respectively.Table 2 shows the total of keypoints detected in each point cloud with US, 3D Harris and 3D ISS methods and the keypoints repeatability of each point cloud evaluated in comparison to the first scan.The matched points with the 3D Harris keypoints were almost the same in all stations in the trajectory, but only a single point was matched in the last station.Therefore, the transformation parameters were not estimated for this sample since it did not have enough matched points to compute a rigid body transformation (Table 2).The largest number of matched points was obtained with 3D ISS method.

Scan
The rigid body transformation parameters estimated with the matched keypoints by LSM were compared with the parameters estimated with registration with CloudCompare.Table 4, Table 5, and Table 6 show the mean, standard deviation and root mean square error (RMSE) of discrepancies between the reference parameters and those estimated with UniformSampling, 3D Harris and 3D ISS keypoints, respectively.This statistical analysis for parameters estimated using 3D Harris keypoints was performed for the first five point clouds (1 to 5).RMSEs of the translations were approximately 7 cm for each 3D keypoint method, which is similar to the laser accuracy.The angular precision was an average of 0.7° and 0.5° for ω and φ, respectively, and slightly higher for κ.

SHOT method assessment
The SHOT method was used to extract features for the keypoints detected by Uniform Sampling and 3D ISS for the five scans with the selected parameters.RMSE in translations were larger than 1 meter and the angular errors were higher than 10° for both experiments.These errors are the result of many mismatches.The matching technique implemented in PCL compares a keypoint detected in a point cloud with all the keypoints detected in another point cloud (Brute-Force matcher).This can lead to incorrect matches in a homogenous environment, such as orange orchards.Fig. 6 shows the result of the point cloud registration obtained with the combination (a) SHOT+US and (b) SHOT+3D ISS keypoints.The green lines represent the matched pairs, and they evidence the incorrect matches.Therefore, it is necessary to eliminate false matchings before estimating the transformation parameters.In this paper, we filtered the mismatches by calculating the 3D distances between the matched points and compared them with the threshold.Table 7 shows the total matches obtained with SHOT and the number of correct matches based on the 3D distances and the threshold.Some methods, such as RANSAC, can be used to identify mismatches combined with additional strategies, such as the reduction of the search space.These techniques are fundamental for feature-based matching methods in homogenous areas, as it was observed in this paper since better results were obtained when filtering the outliers by distance.

Global method registration assessment
The ICP technique had good performance for the registration of the two first point clouds in the sequence, which were acquired with 1-second interval (spatial distance of 1.5 m).However, the registration of the third and subsequential scans with the first point cloud did not present good results with the ICP, requiring reasonable initial values for the transformation parameters.The initial values of the trajectory can be obtained from the IMU data, an alternative which will be studied in future work to reduce motion deformations.On the other hand, the registration with CPD presented better results in the registration trials, and without needing for initial values for the trajectory.

CONCLUSION
This work presents the assessment of feature-based approaches combining the 3D keypoint detectors Uniform Sampling, 3D Harris and 3D ISS with the 3D feature extractor SHOT, and global methods (ICP and CPD) for point cloud registration.First, the keypoint detection methods were analysed, with 3D ISS and US presenting better results.After that, the SHOT feature-based matching was evaluated considering both 3D ISS and US keypoints.Most of the laser data contain points over leaves and ground, making it difficult to extract stable features since most of these 3D feature extraction methods are developed for indoor areas abundant with corners and planes.The orange fields do not exhibit these types of objects (Fig. 4 b).In addition, orange orchards present an environment of globally repetitive patterns and locally fractal features, leading to a challenging matching problem.The excessive number of mismatches affected the estimation of the translation and rotation parameters, requiring an approach to filter the outliers.In this work, the matches were filtered based on 3D distances, which improved the results.
Global methods, which do not use feature extraction, were also assessed.For better performance of these approaches, the point clouds need to be resampled to produce a regular pattern.In addition, the results can be improved using initial values, which can be obtained from the internal IMU sensor of the OS0-128 scanner.The CPD outperformed ICP and did not require initial values for the transformation.
For future work, other feature-based methods will be tested for this environment.Furthermore, an additional strategy will be used to filter the mismatches and reduce search space.
3 a).The backpack platform was carried by an operator while walking in the orange field (Fig.2 a).The laser data was acquired with a 20 Hz scan frequency and a horizontal resolution of 1024 ticks (number of steps in a single sensor turn).A trajectory of approximately 220 m totalling 7977 scans was recorded (Fig.3b).In this work, five (5) first 360° scan point clouds (named here as frames), with 1 fps, were selected with the aim of evaluating the efficiency of the selected 3D descriptors to generate a local map.Local maps are one of the steps of SLAM (simultaneous localisation and mapping), which is used to generate a global map of the entire environment.Fig. 3 a shows the test field, while Fig. 3 b shows the test field location and the trajectory (in yellow).Fig. 4 a shows examples of (a) top and (b) vertical views of a 360° laser frame point cloud collected by the Ouster laser scanner in a top view.In Fig. 4 b, it is possible to visualise the oranges trees mapped in a 360° laser frame.Only the canopy, leaves and ground are mapped in the point cloud.The canopy is very dense, and the laser scanner penetration (only dual-mode echo detection) is not enough to detect the stems.

Fig. 5
Fig. 5 shows the distribution of the keypoints detected in the first scan with (a) US, (b) 3D Harris, and (c) 3D ISS methods.Repeatability and spatial distribution of the keypoints are important factors for accurate point cloud registration.Between the 3D keypoint methods tested, 3D Harris method presented the lowest performance in terms of repeatability and spatial distribution, detecting fewer keypoints when compared with the
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-1/W1-2023 12th International Symposium on Mobile Mapping Technology (MMT 2023), 24-26 May 2023, Padua, Italy ()  ()  () are the rotation matrices from the reference system of point cloud i to point cloud j, and (TX, TY, TZ) are the translations in the respective axis.

Table 4 .
Statistical of discrepancies of the transformation parameters estimated with US keypoints.

Table 5 .
Statistical of discrepancies of the transformation parameters estimated with 3D Harris keypoints.

Table 6 .
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-1/W1-2023 12th International Symposium on Mobile Mapping Technology (MMT 2023), 24-26 May 2023, Padua, Italy Statistical discrepancies of the transformation parameters estimated with 3D ISS keypoints.

Table 7 .
Table8shows the RMSE of the estimated parameters, which were significantly improved.The RMSE of discrepancies were lower with 3D ISS keypoints.Total of matches obtained with SHOT and keypoints (US and 3D ISS) and the correct matches based on the 3D distances between points.

Table 8 .
RMSE of discrepancies of the transformation parameters estimated with the filtered SHOT matched keypoints.

Table 9 .
RMSE of discrepancies of the transformation parameters estimated with ICP and CPD.