AUTONOMOUS UNMANNED HELICOPTER SYSTEM FOR REMOTE SENSING MISSIONS IN UNKNOWN ENVIRONMENTS

: This paper presents the design of an autonomous unmanned helicopter system for low-altitude remote sensing. The proposed concepts and methods are generic and not limited to a speciﬁc helicopter. The development was driven by the need for a dependable, modular, and affordable system with sufﬁcient payload capacity suitable for both research and real-world deployment. The helicopter can be safely operated without a backup pilot in a contained area beyond visual range. This enables data collection in inaccessible or dangerous areas. Thanks to its terrain following and obstacle avoidance capability, the system does not require a priori information about terrain elevation and obstacles. Missions are speciﬁed in state diagrams and ﬂight plans. We present performance characteristics of our system and show results of its deployment in real-world scenarios. We have successfully completed several dozen infrastructure inspection missions and crop monitoring missions facilitating plant phenomics studies.


INTRODUCTION
There is a growing interest in deploying unmanned aircraft for vegetation monitoring, photogrammetric survey, and infrastructure inspection (Berni et al., 2009, Eisenbeiss et al., 2005, Montambault et al., 2010).These applications are remote sensing tasks as quantitative or qualitative information about an object is acquired without making physical contact with the object.Unmanned aircraft are attractive for data acquisition because they enable sensing with high spatial and spectral resolution any time weather permits for relatively low cost.High spatial resolution can be achieved even with low-resolution sensors by capturing data close to objects of interest.The feasibility of using small unmanned aircraft for low-altitude remote sensing has been demonstrated (Nebiker et al., 2008, Berni et al., 2009), though challenges remain in widespread adoption (Hardin and Jensen, 2011).This paper focuses on the automation of the data acquisition part of a low-altitude remote sensing task.
Helicopters are well-suited for quasi-static positioning of sensors in 3D space and moving sensors along 3D paths with high precision.Even in windy conditions, helicopters enable precise flight control and operations in cluttered environments because of their maneuverability and ability to fly at any low speed.Precise and repeatable collection of sensor data is achieved through automatic flight control.From an operational point of view, helicopters only require a small area for takeoff and landing and are relatively easy to transport thanks to their compact size.Flying unmanned helicopters, however, requires a skilled pilot, especially for flights at greater distances and close to obstacles.Within visual range (VR), flights are usually conducted as RC (radiocontrolled) flights.Manual flights Beyond Visual Range (BVR) are possible in First Person View (FPV) mode with video goggles but situational awareness is challenging and it requires a reliable communication link.From a safety point of view, helicopters are also problematic.Larger helicopters especially can cause significant damage when they fail.On the other hand, larger platforms are often required for carrying heavy and bulky sensor packages or for covering large areas.
We aim to overcome the problems of deploying unmanned helicopters for remote sensing applications by developing dependable autonomous helicopter systems which do not require expert users.Given there is no interaction required for specifying a task during flight, truly autonomous helicopters also do not require communication links.For BVR flights at low altitude in unknown environments, i.e. without a priori information about terrain elevation and obstacles, the following three capabilities are essential for an autonomous helicopter: (1) ground detection and terrain following; (2) obstacle detection and avoidance; and (3) stable, effective control.Autonomous flights with unmanned helicopters close to ground and obstacles have been successfully demonstrated by (Scherer et al., 2008) and (Tsenkov et al., 2008).However, it is unclear if the helicopters can safely be operated beyond visual range without a backup pilot.Moreover, the proposed methods are computationally expensive and have been demonstrated with heavy, high-end components.One of our objectives has been to maximize application payload capacity.We developed computational efficient methods for flights in unknown environments which only require a lightweight COTS 2D LIDAR and no separate on-board computer.A detailed description of the methods can be found in (Merz and Kendoul, 2011).
Our goal has been to develop an autonomous helicopter system with: (1) high dependability; (2) sufficient payload capacity and flight endurance; (3) good usability; and (4) sufficient flight performance in terms of control errors and average cruise speed.For regular deployment in real-world scenarios, we consider dependability more important than flight performance.Dependability becomes especially important for BVR operations without a backup pilot.We adopted the fundamental concepts of dependability (dependability of a system is the ability to deliver a service International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXVIII-1/C22, 2011 ISPRS Zurich 2011 Workshop, 14-16 September 2011, Zurich, Switzerland that can justifiably be trusted) as described in (Avizienis et al., 2000).Dependability has the following attributes: availability, reliability, safety, confidentiality, integrity, and maintainability.
In our design, we have considered all of the attributes above.
There are several means to attain dependability: fault prevention, fault tolerance, fault removal, and fault forecasting.There is little literature on the design of dependable autonomous unmanned helicopter systems.Most research related to dependability of such systems focuses on achieving reliability through fault tolerance.
Methods for fault tolerant flight control and guidance systems can be found in (Ducard, 2009).Implementing fault tolerance requires additional computational power (e.g.diverse implementations of filters and controllers) or additional hardware (e.g.redundant actuators or flight computers) which effectively reduces payload capacity.On the other hand, fault prevention, removal and forecasting alone reduces the risk of failures significantly.We focus on failures which are likely to occur and which have catastrophic consequences.For attaining dependability we put significant effort in the design, selection, manufacturing, testing, and maintenance of critical components.We attain fault tolerance through system monitors (error detection) which invoke system reconfigurations or control mode changes (system recovery).
In the following section, we present a decomposition of a proven autonomous helicopter system with a description of its components.Thereafter, we propose flight services which are useful for remote sensing applications.Flight services are based on the described components.Section 3 presents performance characteristics of our helicopter system and shows results of its deployment for real-world remote sensing missions.

SYSTEM COMPONENTS AND FLIGHT SERVICES
Larger autonomous helicopters are complex systems.To tackle the problem of complexity we decompose a system in hardware and software components which can be individually tested against a specification (Figure 2).For modelling system behavior and seamless integration of software components considering realtime constraints we use the ESM software framework (Merz et al., 2006).To minimize risks, critical components have to pass ground tests before they are flight tested and all glitches observed on the ground and during flight are thoroughly analyzed.
The unmanned aircraft system (UAS) we propose consists of an autonomous helicopter and a portable ground station (Figure 1).Communication with the helicopter is realized through a spread spectrum wireless modem with encryption.The ground station offers a convenient user interface to the aerial robot.It reduces risks of failures during BVR operations and simplifies flight testing.However, it is not an essential component for operating the helicopter.The RC transmitter is used for manual flights.In BVR mode it is switched off.
The ground station has a touchscreen with a user configurable graphical layout and a backup user interface including a flight termination switch.The touchscreen is used for (1) selecting flight plans; (2) starting and aborting mission plan execution; and (3) monitoring and setting of on-board system parameters.The backup interface permits to send commands to the helicopter in case of a failure of the graphical interface.The ground station stores flight plans, records data received from the helicopter during flight, and archives flight data after landing.Flight plans are uploaded to the flight computer through the wireless data link.
For wireless communication, we use a packet based transmission method with constant transmission rate, fixed packet size, and minimal data buffering to avoid latencies.The autonomous helicopter is based on a COTS RC helicopter (Vario Benzin Trainer).The main criteria for choosing the RC helicopter have been: (1) proven reliability with a specified payload capacity of at least 5kg; (2) easy handling; and (3) quick availability of spare parts.Modifications to the helicopter have been held to a minimum to avoid compromising its proven reliability.The available payload capacity is divided into internal and external payload.The internal payload includes all application independent components added to an RC helicopter which are integrated with the base airframe.The external payload consists of an extended undercarriage and a vibration isolated carrier board holding the remaining components.The extended undercarriage is attached to the original undercarriage of the RC helicopter.Without the external payload the helicopter is a fully functional RC helicopter.The GNC (Guidance, Navigation, and Control) system required for autonomous flight and mission specific components are part of the external payload.We have several modified RC helicopters of the same type and several different external payloads which can be attached to any of the helicopters.Table 1 contains technical specifications of the key components of our helicopter system.
The main internal components we add to a COTS RC helicopter are an interface to the RC components (receiver, swashplate mixer, attitude stabilizer, servos), a system monitor, an electrical power system, and a flight termination system.In addition, we replaced the original fuel tank with a larger tank and integrated sensors for fuel level, engine temperature, and engine RPM.The system monitor monitors, among others, fuel level, voltages, engine temperature, RC link, and a signal from the flight computer indicating its correct operation (heartbeat).A strobe light indicates critical problems such as loss of RC link by flashing with high frequency and warnings such as low fuel or low battery by flashing with low frequency.
Monitoring information is also sent to the flight computer and the flight termination system.In VR mode, the flight termina- The on-board electrical power system is designed for a wide range of supply voltages.During flight, a single LiPo battery powers the RC components and the GNC system.On the ground the helicopter is powered externally.Having to replace only one battery enables quick turn around times.The power system also includes a low self-discharge backup battery for supplying the RC components in case of a power failure.

GNC System
The GNC system consists of a navigation computer, a flight computer, and the following sensors: an Attitude and Heading Reference System (AHRS), a GPS receiver, a barometric pressure sensor, and optionally a 2D LIDAR for obstacle detection.The LIDAR has a 270 • scan range and is mounted vertically pointing forward, i.e. the scanning plane is parallel to the yaw axis and the center beam parallel to the longitudinal axis of the helicopter.All components are part of the external payload and mounted to the same vibration isolated carrier board.The GPS antenna is placed on the tail boom.The pressure sensor is mounted underneath the carrier board in a box with openings away from strong air flow.Although the AHRS is surrounded by ferrous objects and electronic components the magnetometer could be calibrated for accurate heading estimates.The greatest disturbance is the engine.Calibration was performed with running engine.
Computation is split across two computers because of limited computing power and for modularity.The navigation part is implemented on the navigation computer, the guidance and control part on the flight computer.Control inputs are sent to the RC system through a serial link.The execution of different software components and their data flow is managed by two coupled, deterministic state machines, one for each computer.Both state machines are implemented in ESM.Functions are implemented in C. All computers run Linux with a real-time kernel patch and the ESM run-time environment.
The system includes a simulator component enabling hardware in the loop (HIL) simulations in real-time.HIL simulation is a valuable tool for testing the interaction of system components.
Another ground test applicable to portable helicopters is running the control system in the hovering state, moving the helicopter, and observing the servos.Both computers contain a solid state drive based data logger.The data loggers record sensor data and control inputs at a high rate and all important state transitions.This allows for accurate analysis of flights during development and in case of problems.Flight and state machine data is also sent through the telemetry link to the ground where the data is visualized and recorded.
The navigation computer calculates helicopter state estimates using extended Kalman filters and processes LIDAR range data to estimate height above ground and distance to frontal obstacles.
The data is sent to the flight computer at a high rate.Flight control is based on cascaded, decoupled SISO PID controllers for attitude angles, velocities, and positions using state estimates.
The flight computer also includes a PI controller combined with a feedforward controller using collective pitch input for maintaining constant main rotor speed.In our experience, properly tuned PID controllers suffice for controlling unmanned helicopters for remote sensing applications.The basic flight modes are hover, yaw, climb, descent, and cruise.During forward flight the helicopter controls its height either based on barometric pressure or height above ground which is estimated from LIDAR readings.The latter results in a terrain following behavior.The helicopter follows a vertical or horizontal straight line path during climb, descent, and cruise.
We added two special flight modes -pirouette descent and waggle cruise -which increase the field of view of the 2D LIDAR.
The pirouette descent mode enables descents in unknown environments without colliding with obstacles or terrain.It creates a spinning LIDAR with a cylindrical field of view by rotating the helicopter around its yaw axis while descending vertically.During a climb, the helicopter does not perform pirouettes as we assume there are no obstacles above the helicopter.The waggle cruise mode is used to detect obstacles during horizontal path following.It performs a horizontal sweep while flying forward, allowing to scan a corridor-shaped space.

Application Payload
The helicopter can carry a combination of a variety of sensors typically used for remote sensing.All sensors are mounted to the same vibration isolated carrier board which also holds the GNC system.This has two advantages: there is no need for additional vibration isolation and it is possible to use the helicopter state estimates for determining the pose of a sensor.Sensors can be mounted in any orientation.Sensors for vegetation monitoring are typically mounted vertically pointing down, sensors for structure inspections are typically mounted horizontally pointing forward.The undercarriage can be further extended to accommodate larger sensors.
To maximize sensor payload capacity and flight endurance we do not use gimbal mechanisms.For our applications, this has not been a severe limitation.We can tolerate small changes in sensing direction and we only require still images taken with short exposure times.Our applications also do not require interactive control of sensors and transmission of sensor data.Sensors are controlled by the flight computer as defined in a mission plan and data is recorded on board the helicopter using flash memory.Sensor data is saved synchronously with helicopter state estimates.Table 1 contains details about the sensors we have been using and the accuracy of pose estimates.Depending on the application, geo-referencing of data is possible based on pose estimates only or a combination of pose estimates and ground control points.
The digital cameras support raw image capture and manual exposure and focus control.The time it takes from sending a shutter command from the flight computer until a camera captures an image has been calibrated.

Flight Services
We developed systems for delivering the following flight services: (1) take off; (2) hover; (3) heading change; (4) waypoint flight; (5) waypoint flight with obstacle avoidance; (6) waypoint flight with air traffic control interaction; (7) return to base; (8) closerange inspection; and (9) landing assistance.Note that a service delivered by a system is its perceived behavior which may differ from its intended function (Avizienis et al., 2000).The term 'flight service' refers to a description of an intended function of the helicopter system.
In the following, we provide a brief description of flight services useful for remote sensing applications.Before requesting a flight service except for service (1), the helicopter must hover.After delivery of a correct flight service, the helicopter will hover.The take off service includes warming up the engine and spooling up the rotor.When ready for take off, the helicopter performs a vertical climb to a height well outside the ground effect zone.The climb finishes in hover.The hover service is based on the hover mode (Section 2.1).The heading change service is used for pointing the helicopter in a specified direction during hover.Position changes are achieved through three different waypoint flight services: Service ( 4) is used for flights in obstacle free space.While flying to a given waypoint, the helicopter accurately tracks vertical and horizontal straight line paths.Service ( 5) enables operations in airspace shared with other aircraft.To ensure separation between the helicopter and other aircraft, the helicopter interacts with the Automated Dynamic Airspace Controller (ADAC) system developed by Boeing Research and Technology USA (Clothier et al., 2011).Service (6) allows to fly safely to a specified destination in areas likely to contain obstacles.The implemented LI-DAR based avoidance strategy does not require maps and despite being reactive, guides the helicopter in most cases to the specified destination.The return to base service is utilized to fly the helicopter on the shortest path to a pre-defined location in obstacle free space avoiding pre-defined no-fly-zones.The close-range inspection service enables positioning the helicopter at a specified height at a given distance from a structure.The helicopter approaches the structure defined by geographic coordinates from a given approach point until the distance measured by the LIDAR reaches the specified distance.Flight service ( 9) is requested to assist the user in landing the helicopter.The user only commands vertical velocity.Landing has not been fully automated yet as the helicopter currently can not reliably estimate distance to a given touch down point and horizontal velocity before touch down.
For flight services (2-5) and ( 8) the user can choose between pressure based height control or terrain following for low-altitude flights.Services with position or direction changes include acceleration and deceleration stages accounting for dynamical constraints of the helicopter.A requested service will be rejected or a provided service cancelled if given parameters or circumstances do not allow a safe flight.In addition to the flight services, there is a helicopter shutdown service which is available when the aircraft is on the ground.
Missions are specified in ESM state diagrams and flight plans.ESM state diagrams are converted in state machine code with tools of the ESM framework.State machines request flight services and other services based on events from timers, services, and system monitors.A flight plan is a sequence of 3D waypoints with cruise speeds and optional parameters (e.g. for terrain following).Flight plans are uploaded from the ground station to the helicopter either before take off or while hovering.Mission plans defined in ESM are part of a complex state machine.However, thanks to the decomposition properties of ESM, the user does not have to understand details of subsystems to model behaviors.The state diagram shown in Figure 3 models a simple remote sensing mission where the helicopter is commanded to take off, fly to waypoints defined in a flight plan, take images at each waypoint, and return to base.Mission and flight plans can be generated manually or by a mission planner.

SYSTEM PERFORMANCE AND DEPLOYMENT
In this section, we first present performance characteristics of our helicopter system.All flights have been conducted in accordance with the Australian civil aviation safety regulations.Our system is typically operational within 15 minutes after arriving at a take off location.a with damage to the helicopter affecting its suitability for safe flight Table 2: Flight statistics of four helicopters.
Table 2 shows data from the log books of the four helicopters we have been using.There were very few flights with damage to the aircraft which demonstrates the dependability of the system.Two helicopters were damaged beyond repair.The first helicopter crashed during a BVR flight and the second during a manual flight.to 2m/s.The terrain following began after a descent during which height control was changed from pressure based control to LI-DAR based control.Figure 5 shows a typical obstacle avoidance scenario.During a mission, the state machine requested the goaloriented obstacle avoidance service described in Section 2.3 for a flight from waypoint 1 to waypoint 2 with 1m/s.The system successfully detected the tree and 'tracked' its outline at a safe distance until it intersected with the original flight path.

Infrastructure Inspection
The helicopter system provides infrastructure inspection services by combining the flight services presented in Section 2.3.We have successfully completed 37 inspection missions with frontal and vertical image capture and recorded more than 14 hours of autonomous flight time.All flights were performed without providing a priori information about the environment and two missions were flown in BVR mode without a backup pilot.At the end of the Smart Skies project (Clothier et al., 2011), we demonstrated the execution of a BVR inspection task with air traffic control interaction.The task included a 1.4km flight to an inspection area, the inspection of a windmill, and the return to base.Before take off, a flight plan was uploaded defining a descent point in the inspection area, the approach point, and the position of the windmill.During the flight to and from the inspection area, the helicopter interacted with the air traffic control system mentioned in Section 2.3 to ensure separation from other aircraft.The cruise speed was set to 5m/s.Results of one of the inspection missions are shown in Figure 6.The helicopter hovered approximately 10m in front of the windmill to take images.The inspection image shows sufficient detail for analysis.During the return flight, the air traffic control system sent a flight plan to the helicopter to avoid an oncoming aircraft.A detailed description of the mission can be found in (Merz and Kendoul, 2011).

Plant Phenomics
Plant phenomics is the application of new technologies to the measurement of plant phenotype, i.e. the physical characteristics of plants, particularly when grown in mono-cultures as a crop.The meaning of 'phenomics' derives from the word 'genomics' which was coined during the 1990s to describe the study of the genetic composition of a species which was being revealed by new DNA detection and measurement technologies.Hence, highthroughput image analysis of crop growth is considered to be a 'phenomics' technology.Previous research has demonstrated the value of UAS-gathered imagery to interpret spectral reflectance characteristics of crops and trees at spatial resolutions of ca.20 cm (Berni et al., 2009) and to consequently monitor contrasting irrigation treatments, for example.Applications in plant phenomics aim to compare the growth and development of different plant varieties or cultivars, and the causes of such genetic differences tend to be more subtle than those between agronomic treatments.
To resolve individual leaves of wheat (1 to 2cm) using the digital cameras listed in Table 1, missions have been flown at 15 to 30m, cf. the 150m of Berni et al.The flights were scheduled to cover entire fields of approximately 1 to 1.5ha in size (Figure 7) and obtained measurements across multiple experiments planted within each field.Typically, the flights were flown at two elevations: at 30m to obtain high-density, high-resolution images, and at 60m to capture entire experiments in single images.We have successfully completed 40 missions and recorded more than 16 hours of autonomous flight.
Figure 8 shows a processed image which illustrates variation in leaf area 'cover' between plots (2 x 7m dimension) that is associated with different wheat varieties being grown adjacently in the same field experiment.An NIR filter facilitates the identification of vegetative material.Additional image processing protocols are being developed to allow determination of seasonal changes and variety differences in the brightness of the leaf color (i.e.chlorophyll content), the rate of leaf senescence late in the season, and the relative leaf temperature (using a thermal sensor).These types of images are being used as part of a project to identify wheat varieties with improved adaptation to higher temperature conditions as may be experienced if expected changes in climate come about.

CONCLUSIONS
We presented the design of an unmanned helicopter system capable of performing remote sensing missions in unknown environments autonomously beyond visual range.Missions are specified in state diagrams and flight plans.This approach is flexible and enables definition of complex missions.Our focus has been on system dependability which is a key requirement for realworld deployment.We use a component based approach to tackle the problem of complexity of such a system.The concepts and methods we propose are applicable to any larger unmanned helicopter.The design has been validated through the development and deployment of the CSIRO autonomous helicopter.The helicopter has been successfully deployed for autonomous capturing of images for infrastructure inspections and plant phenomics studies.We have not encountered problems with motion blur and the recorded images show objects of interest with sufficient detail.To decide if a system is suitable for a particular remote sensing task, we estimate the accuracy at which sensors can be positioned in 3D space as well as the accuracy of pose estimates during image capture.
Future work is directed towards further increase of dependability and the level of autonomy by including failure modes dealing with GPS problems, an autonomous landing service, and a mission planner.

Figure 2 :
Figure 2: Components of the helicopter system.Components of a typical COTS RC helicopter are printed on blue background.Software components are printed in italics.

Figure 3 :
Figure 3: Example of an ESM state diagram defining a simple mission plan.

Figure 4 :
Figure 4: Helicopter height during a terrain following flight.

Figure 6 :
Figure 6: Helicopter trajectory of a BVR windmill inspection mission and zoomed inspection image taken by the helicopter.

Figure 7 :Figure 8 :
Figure 7: Flight path and height of a typical plant phenomics mission.

Table 1 :
Technical specifications of key system components.tionsystem kills the engine and commands full collective pitch in case of RC link loss, low voltage, or miscellaneous fatal errors.For BVR flights, a loss of the RC link is ignored as long as the heartbeat signal is active.The flight termination is activated when the heartbeat is inactive and the RC link is lost or in case of low voltage or fatal errors.In addition, it can be activated by the flight computer.

Table 3
lists the empirical error bounds for different basic flight modes.The errors bounds are estimated from recorded flight data of several flights in winds up to 10m/s from different directions by comparing desired values with state estimates.Velocities are with respect to the earth fixed frame.The horizontal position error of the cruise mode refers to the error when reaching a waypoint.The listed control errors do not include state estimation errors and we assume that state estimation errors do not significantly affect control errors.As our focus has been on dependability rather than control performance, there is certainly room for improvement; especially for flights at higher airspeed and in turbulent conditions.For our applications, though, the achieved control accuracy has been sufficient.In calm conditions, the pitch angle of the carrier board is less than 2 • at 3m/s forward speed.Results of a terrain following flight are shown in Figures 4. Thereference height was set to 5m above ground and the cruise speed hover 5m/s cruise 1m/s climb/desc.yaw 95 percentile of absolute error h=height error, r=horizontal position error, c=cross track error, ψ=heading error, v=horizontal/vertical velocity error a

Table 3 :
Empirical control error bounds a of basic flight modes.