Autonomous robotic drone system for mapping forest interiors

During the last decade, the use of drones in forest monitoring and remote sensing has become highly popular. While most of the monitoring tasks take place in high altitudes and open air, in the last few years drones have also gained interest in under-canopy data collection. However, flying under the forest canopy is a complex task since the drone can not use Global Navigation Satellite Systems (GNSS) for positioning and it has to continually avoid obstacles, such as trees, branches, and rocks, on its path. For that reason, drone-based data collection under the forest canopy is still mainly based on manual control by human pilots. Autonomous flying in GNSS-denied obstacle-rich environment has been an actively researched topic in the field of robotics during the last years and various open-sourced methods have been published in the literature. However, most of the research is done purely from the point-of-view of robotics and only a few studies have been published in the boundary of forest sciences and robotics aiming to take steps towards autonomous forest data collection. In this study, a prototype of an autonomous under-canopy drone is developed and implemented utilizing state-of-the-art open-source methods. The prototype is utilizing the EGO-Planner-v2 trajectory planner for autonomous obstacle avoidance and VINS-Fusion for Visual-inertial-odometry based GNSS-free pose estimation. The flying performance of the prototype is evaluated by performing multiple test flights with real hardware in two different boreal forest test plots with medium and difficult densities. Furthermore, the first results of the forest data collecting performance are obtained by post-processing the data collected with a low-cost stereo camera during one test flight to a 3D point cloud and by performing diameter breast at height (DBH) estimation. In the medium-density forest, all seven test flights were successful, but in the difficult test forest, one of eight test flights failed. The RMSE of the DBH estimation was 3.86 cm (12.98 %).


Introduction
In recent years, the autonomous flying of drones has been an actively studied topic in both commercial and academic communities.Most autopilots can fly autonomously in open areas where GNSS is available.However, inside dense forest environments, the localization of the drone cannot rely on GNSS because of problems in accessing signals, and the drone also must perform autonomous path planning to avoid obstacles in the path.New solutions are required to overcome these challenges.On the other hand, a drone flying inside the forest provides a completely new perspective on mapping the internal environment of the forest and many other applications.
Recent advances in robotics and the development of small embedded computers have opened new possibilities also for autonomous flying in GNSS-denied cluttered environments.During the last few years, several open-sourced solutions for inside forest flying have been proposed in the literature.Most of the solutions use either Lidar (Liu et al., 2022;Ren et al., 2022;Liu et al., 2024;Ahmad et al., 2022) or camera (Zhou et al., 2022;Campos-Macías et al., 2021) to create a 3D grid map for collisionfree path trajectory planning and track the position of the drone in local coordinate system.Some of these methods support also autonomous navigation for swarms of drones (Zhou et al., 2022;Ahmad et al., 2022).Some of the proposed methods replace the map-based trajectory planning with a neural network that directly maps the images from onboard depth camera to collisionfree trajectories (Loquercio et al., 2021;Nguyen et al., 2023).
However, autonomous flying inside the forests is still mainly researched from the point of view of robotics.Liu et al. (2022) calculated the number of discovered trees during the flight path, but typically the works concentrate solely on autonomous flying.On the other side, the forest research utilizing under-canopy drones has been mainly based on manual flying by human pilots (Kuželka and Surovỳ, 2018;Krisanski et al., 2020;Hyyppä et al., 2020aHyyppä et al., , 2021;;Chisholm et al., 2013Chisholm et al., , 2021;;Wang et al., 2021;Tavi, 2023).Shimabuku et al. (2023) used semi-automated commercial drone that was able to avoid collisions with tree-trunks but the flight path had to be manually corrected after the the avoidance.Recently, Liang et al. (2024) performed DBH and stem curve estimation in subtropical urban forest with LiDAR data collected with an autonomous under canopy drone.
The objectives of this study were to implement a prototype of a drone capable of autonomously flying inside the forest, to evaluate its reliability in navigating boreal forests, and to test its performance in mapping forest parameters.A prototype of an autonomously flying drone was developed using state-of-theart open-source algorithms for GNSS-free pose estimation and autonomous trajectory planning.The performance and suitability of the prototype for flying inside boreal forests were evaluated through multiple flight tests.In addition, the image data captured during one test flight by a low-cost onboard stereo camera, utilized for navigation, was post-processed into a 3D point cloud.The potential of the system in the mapping of Boreal forest interiors was then demonstrated by performing DBH estimation on the processed point cloud.

Autonomous navigation
An open-source algorithm for autonomous trajectory planning, EGO-Planner-v2 (Zhou et al., 2022), served as the foundation for the implemented prototype.While the original EGO-Planner-v2 employs a swarm-based approach, this study implemented a single-drone method, and swarm flight related parts of the algorithm were omitted.The mapping module of the EGO-Planner-v2 relies on probabilistic occupancy grid maps (Moravec and Elfes, 1985), with depth images from the stereo camera serving as input.Furthermore, the mapping module adopts the fixed-sized circular buffers proposed by Usenko et al. (2017) for maintaining the local map.The map height is limited by user-defined virtual floor and ceiling which can be used to restrict the flying altitude.EGO-Planner-v2 employs a trajectory representation known as MINCO (minimum control) (Wang et al., 2022) which represents the trajectories as piece-wise polynomial splines.The initial path given as an input for the optimization is obtained with A*.For pose estimation, a visualinertial-odometry method (VIO) based on a stereo camera and an IMU is utilized, employing the VINS-Fusion algorithm (Qin et al., 2018(Qin et al., , 2019)).For the tracking of the planned trajectory, an open-source tracking controller "px4ctrl" (ZJU FAST Lab Team, 2022) included in the source code of Zhejiang University's drone building course was used.The tracking controller outputs low-level attitude and thrust commands that are forwarded to the PX4 flight controller software (PX4 Community, 2023).All the algorithms run on top of the Robot Operating System (ROS) (Quigley et al., 2009) which handles the communication between the modules.
The first version of the prototype was implemented and evaluated in the study by Karjalainen et al. (2023).This study presents an enhanced version with improved hardware and flight duration and our first results from the forest characteristics estimation with the data recorded during the flight tests.

Hardware
The prototype used a Jetson Orin NX (Santa Clara, California, USA) an onboard computer.The PX4-compliant flight controller unit employed was the Holybro Kakute H7 (Hongkong, China), which also integrated the IMU.Navigation algorithms operated on the Robot Operating System (ROS), with communication facilitated by the Mavros protocol between the onboard computer and the autopilot.For imaging, an Intel RealSense D435 (Intel Realsense Development Team, 2023; Keselman et al., 2017) camera was employed.This camera provided both depth images for 3D mapping and grayscale stereo images for Visual-Inertial Odometry (VIO).The image resolution of the depth images was 1280 × 720 pixels.The grayscale stereo images were recorded in 1280 × 720 resolution for the postprocessing and downsampled to 640 × 360 for VINS-Fusion to ensure the real-time pose estimation.The laser emitter of the D435 was turned off since the infrared points moving with the camera would have caused problems for the VIO, and the camera was used as a passive stereo camera instead.The extrinsic parameters and the time shift between the camera and the autopilot IMU were calculated with an open-source camera calibration toolbox, kalibr (Furgale et al., 2012(Furgale et al., , 2013;;Maye et al., 2013) by using Aprilgrid (Olson, 2011) as a calibration target.
The computer, camera, and flight controller unit were mounted on a 330 mm drone frame.Each propeller had two pieces of 6 cm long blades.The measured weight of the drone was 791 g without batteries and 1153 g with batteries, including motors and the onboard computer.The used drone hardware is presented in Figure 1.

Test sites and flight tests
The performance of the system was evaluated in two Boreal forest areas located in Evo,Finland (61.19°N,25.11°E).The flight paths traversed well-documented forest sample plots that have been widely used in forest research (e.g., Liang et al., 2018Liang et al., , 2019;;Wang et al., 2019;Hyyppä et al., 2020a,c).In April 2020, the sample plots have been surveyed using terrestrial laser scanning (TLS) with the Leica RTC360 to obtain precise measurements of tree height, DBH, and tree density.The plots are classified into three complexity categories: "easy", "medium", and "difficult".The classification is based on tree density and the amount of understory vegetation in the plot.This study used two of the forest plots classified as "medium", and "difficult" (Figure 2).
The tree density of the medium plot was 650 trees/ha with a mean DBH of 27 cm.The tree density of the difficult plot was 2000 trees/ha with a mean DBH of 17 cm.Notably, only trees with a DBH greater than 5 cm were included in these statistics.The most common tree species in both plots was Norway spruce (Picea abies (L.) H.Karst.).
The performance of obstacle avoidance was tested by performing seven test flights in the medium forest and eight test flights in the difficult forest.The flight distances varied between 34 m and 42 m.In both test forests, either the takeoff location or the takeoff heading was changed after five test flights.The flight was considered successful if the drone managed to avoid collision and fly to the predefined goal.If the given goal was inside an obstacle, the flight was considered successful if the drone managed to fly next to the goal.
In this study, the forest data collection is demonstrated by performing the DBH estimation to the dataset collected from the first test flight in the "medium" forest.

Post-processing and DBH estimation
The DBH estimation was performed on point clouds that were created photogrammetrically from the onboard stereo camera data in the post-processing with Agisoft Metashape (Metashape Development Team, 2021).Every sixth recorded image pair was selected for the processing and stereo baseline of 5 cm was added.For the post-processing, circular coded targets provided by Metashape were placed within the forest plot and total of 15  were used as Ground Control Points (GCPs) during the point cloud processing.The relative distances between the control points were measured with the Leica Nova TS60 total station with a precision of approximately 2 mm. Figure 3 presents a frame captured during a flight test with the onboard camera and shows the targets placed in the forest.Figure 4 visualizes the GCPs, the stereo images and the reference trees during the point cloud processing in Metashape.
The trees were detected and DBHs were estimated with an automatic stem diameter algorithm (Hyyppä et al., 2020b,a).The ground truth DBH values for the reference trees were calculated with the same automatic stem diameter algorithm from the high-quality TLS point cloud.
The Accuracy of the DBH estimation was evaluated by calculating differences between the ground truth DBH values and the estimated DBH values calculated from drone-collected point cloud data.To estimate the accuracy of the DBH estimation, we calculated several metrics including Root Mean Square Error (RMSE), Relative RMSE (RMSE%), Bias, Relative Bias (Bias%), and Standard Deviation.The RMSE represents the square root of the mean of the squared variances between the estimated and reference DBH values.Meanwhile, bias signifies the average disparity between the estimated and reference DBH values.Additionally, we evaluated the completeness of stem detection, where we compared the total number of reference trees found from our drone-collected point cloud data to the total number of reference trees within our point cloud boundaries.We did not account for the influence of tree growth occurring between the drone-collected point cloud data and the reference measurements obtained via TLS.Additionally, some reference trees have fallen between the measurements as a result of the storm and have not been taken in account in the estimations.The equations used in the calculation of the performance metrics were where n represents the total number of observations.
where D BH reference represents the mean of the reference DBH values and SD stands for the Standard Deviation.

Completeness =
Reference trees found Total number of reference trees × 100% (6)

Results
The success rate of the autonomous flying in the medium forest was 100% when all of the seven flights were successful.Of the seven flights, five were performed without emergency stops.The flight success rate in the difficult forest was 87.5 % when one of the eight flight tests failed due to collision with a small dry branch.Photogrammetric processing results are introduced in Table 1.
All the images was aligned correctly resulting of number of tie points of 197132, reprojection error of 0.39 pixels and highly dense point cloud of 3.58 points/cm 2 .Figure 6 shows the postprocessed flight trajectory, point cloud boundaries, reference trees and found trees with stem detection algorithm.The total number of reference trees inside the point cloud boundary completely was 29, of which 23 was found from drone-collected point cloud.Thus, calculated completeness was 79.31 %.The stem detection algorithm failed to detect one large tree and two small trees (see Figure 4 ref9 and ref17) located near the drone's trajectory, despite these trees being well-reconstructed in the point cloud.The three additional trees that were not found were located at a greater distance from the trajectory and exhibited poor reconstruction quality in the point cloud.Figure 7 illustrates estimated DBH RMSE errors with respect to distance to trees.From that we can see that there is no correlation between distance of the trees and DBH errors.Even though, there is more smaller error trees closer to the trajectory than further away.The overall RMSE of the DBH estimation was 3.86 cm (12.98%), bias of 1.40 cm (4.71%) and standard deviation 3.60 cm was achieved.In these preliminary results, the performance of the autonomous flying was good in the medium-density forest.In the difficult forest, the prototype had difficulties sensing dry and needleless spruce branches, even though the flight success rate was still relatively high.The DBH estimation accuracy from data of the first test flight was promising when a high number of Ground Control Points were used during the point cloud postprocessing, but more analysis is needed to evaluate the real potential in the DBH estimation.
The future work includes a more comprehensive analysis based on the data recorded during the flight tests.The analysis includes both estimating the accuracy and reliability of VIO localization and performing a more extensive evaluation of DBH estimation potential.The DBH estimation accuracy will be further analyzed by comparing the estimate accuracies between the test flights and examining the effects of removing the GCPs or reducing the number of used GCPs in the post-processing.After the more comprehensive analysis, the results will also be compared against the drone-based DBH estimation methods presented in the literature.
The potential future work includes also further development and enhancements of the prototype.
Besides the point cloud based forest inventory attribute estimation, potential future applications of the system in forest research include image data collection of understory vegetation biodiversity research and detection of damages in the bark of the trees due to insect pests, to mention but a few.

Acknowledgements
This research was funded by the Academy of Finland within projects "Learning techniques for autonomous drone based hyperspectral analysis of forest vegetation" (decision no.

Figure 1 .
Figure 1.The drone hardware used in the flight tests.
(a) The medium test forest.(b) The difficult test forest.

Figure 2 .
Figure 2. The view from the first takeoff locations of the both test forest plots.

Figure 3 .
Figure 3. Ground control points in medium difficulty forest.The image is taken during the flight with the Intel Realsense D435 onboard camera.

Figure 4 .
Figure 4. Ground control points, stereo images and reference trees visualized during the point cloud processing in the Agisoft Metashape (Metashape Development Team, 2021).

Figure 5 .
Figure 5.The 3D grid map used for the obstacle avoidance and the online estimated flight path by VIO during the first test flight seen from above.