AUTOMATED DATA PRODUCTION FOR A NOVEL AIRBORNE MULTIANGLE SPECTROPOLARIMETRIC IMAGER ( AIRMSPI )

A novel polarimetric imaging technique making use of rapid retardance modulation has been developed by JPL as a part of NASA’s Instrument Incubator Program. It has been built into the Airborne Multiangle SpectroPolarimetric Imager (AirMSPI) under NASA’s Airborne Instrument Technology Transition Program, and is aimed primarily at remote sensing of the amounts and microphysical properties of aerosols and clouds. AirMSPI includes an 8-band (355, 380, 445, 470, 555, 660, 865, 935 nm) pushbroom camera that measures polarization in a subset of the bands (470, 660, and 865 nm). The camera is mounted on a gimbal and acquires imagery in a configurable set of along-track viewing angles ranging between +67° and –67° relative to nadir. As a result, near simultaneous multi-angle, multi-spectral, and polarimetric measurements of the targeted areas at a spatial resolution ranging from 7 m to 20 m (depending on the viewing angle) can be derived. An automated data production system is being built to support high data acquisition rate in concert with co-registration and orthorectified mapping requirements. To date, a number of successful engineering checkout flights were conducted in October 2010, August-September 2011, and January 2012. Data products resulting from these flights will be presented.


INTRODUCTION
The AirMSPI instrument is part of the development towards the next generation of the moderate resolution multi-angle, multispectral instruments pioneered by the Multiangle Imaging SpectroRadiometer (MISR) (Diner, 1998a).MISR with its nine view angles and four spectral bands per angle has been collecting global aerosol, cloud, and surface bidirectional reflectance data since 2000 on board NASA's Terra satellite.Advanced technology developments under the Multiangle SpecroPolarimetric Imager (MSPI) program include broader spectral coverage and addition of high accuracy polarization channels utilizing dual photoelastic-modulators (PEMs) (Diner, 2010).AirMSPI is our first airborne prototype, and includes an expanded set of spectral bands as well as an upgraded mode of acquiring multi-angle data compared with its predecessor, AirMISR, which flew on NASA's ER-2 aircraft between 1999 and 2004 ( Diner, 1998b;Jovanovic 2001).
AirMSPI includes an 8-band (355, 380, 445, 470, 555, 660, 865, 935 nm) pushbroom camera that measures polarization in a subset of the bands (470, 660, and 865 nm).The camera is mounted on a gimbal and acquires imagery in a configurable set of along-track viewing angles ranging between +67° and -67° relative to nadir.As a result, near simultaneous multi-angle, multi-spectral, and polarimetric measurements of the targeted areas at a spatial resolution ranging from 7m to 20m (depending on the viewing angle) are derived.In the standard "step and stare" mode of data acquisition, AirMSPI collects data at nine fixed view angles (±67º, ±60º, ±47.5º, ±26.5º, and 0º), though these angles are programmable.In addition a "continuous sweep" mode has been implemented in which two simultaneous motions are part of the image creation: 1) linear due to the movement of the airborne platform, and 2) angular due to continuous back-and-forth along-track slewing of the gimbaled camera.In this mode, images with very high angular resolution of homogeneous areas of interest can be obtained.This paper describes the design and implementation of an automated processing system responsible for delivering multiangle intensity (I) for all spectral bands, and degree of linear polarization (DOLP) and angle of linear polarization (AOLP) for the polarimetric bands.The data are orthorectified and superimposed on a map projection grid.We also describe the basic instrument characteristics, the on-board data capture and commanding sequence, and discuss potential scientific applications.

2.1
AirMSPI flies in the nose of NASA's ER-2 aircraft at an altitude of about 20 km.To obtain polarimetric imagery with high accuracy, the AirMSPI camera includes a retardance modulator consisting of a pair of photoelastic modulators (PEMs) sandwiched between two achromatic, athermalized quarter wave plates.The changing retardance of the modulator causes the plane of linear polarization to rotate back and forth with a period of about 40 msec.For each pixel in the focal plane detector array, this modulation encodes information about intensity I as well as one of the linear Stokes parameters (Q or U, depending on whether the array is overlain by a polarization analyzer at 0º or 45º, respectively).The key to obtaining high polarimetric accuracy is that the ratios Q/I and U/I detected in this manner are independent of optical transmittance or detector gain.These relative measurements are then combined to produce DOLP.This technology is described in Diner et al. (2007) and Diner et al. (2010).
The AirMSPI camera is mounted on a gimbal inside a pressure vessel that exposes the instrument to ambient conditions at altitude.The gimbal is driven by an Aerotech actuator equipped with a precision angle encoder, enabling flexible operating and camera pointing modes.The ER-2 supplies 28 VDC power to the instrument.Instrument electronics including the flight computer and data acquisition equipment are mounted in a On-board processing system 2.2 The on-board processing system simultaneously records and timestamps instrument data from the AirMSPI camera, attitude and position from the ARINC-429 navigation board and view angle from the Aerotech gimbal controller.It also monitors the pilot cockpit switches and indicates instrument status via the cockpit control panel.Instrument health and safety are also monitored by the on-board processing system.A ruggedized Windows PC is used to coordinate, control and interface to all the instrument's subsystems.
The AirMSPI camera produces data at a rate of 25 MB/s over CameraLink to a dedicated IO Industries frame grabber.Directly attached to the frame grabber is 1TB of Solid State Disk (SSD) raid array for instrument data.Another 256GB of SSD storage is available for navigation data, gimbal data and instrument logs.This is enough storage to support a 10 hour flight.The SSD storage is configured as pluggable storage modules, so they can be easily swapped for quick instrument turn-around.
Accurate timestamps are necessary to relate image data, attitude and position data, and view angle for image orthorectification by the ground processing software.The ER-2 provides a time code derived from the GPS/inertial navigation system via the experiment interface panel.AirMSPI uses a Meinberg IRIG-B timecode receiver board to synchronize the instrument computer with the ER-2.All recorded data are timestamped with this time code.

2.3
Pre-flight planning includes determination of the preprogrammed gimbal pointing sequence to be implemented by the AirMSPI computer during a given flight.The first step is to determine the ER-2 flight lines and target locations, and to calculate the way points at which the pilot is to depress a cockpit switch that activates the gimbal motion and instrument data collection.Sequences are constructed from two basic instrument operating modes.In "step and stare" mode, the camera is pointed at the same target area at a fixed set of view angles beginning with the most forward view and then stepping aftward as the aircraft flies downtrack.The camera then slews forward and the sequence repeats for the next target, about 100 km downtrack.This mode is most useful when the highest possible spatial resolution is desired.Target area is ~11 km x 10 km.In "continuous sweep" mode, the gimbal slews back and forth.Because the gimbal moves continuously, spatial resolution is reduced to ~60 m due to smear and the set of view angles depends on location, but multiangle overlap coverage is greater, enabling better spatial coverage of targets with vertical extent such as multi-layered cloud field.AirMSPI's Aerotech actuator permits the slew rate to vary as a function of angle to keep the amount of smear constant.In both modes, imagery is oversampled as the cross-track sample spacing (7 m at nadir) is set by the detector pitch and the along-track sample spacing (8 m) is set by the image frame time.

2.4
Radiometric and polarimetric calibration are important for meeting AirMSPI's measurement objectives.The camera's radiometric transfer curve (relationship between illumination and digital output) is established in the laboratory by using a 1.65m integrating sphere as the illumination source.The integrating sphere also provides the illumination for determination of pixel-by-pixel gain coefficients.Independent measurements of the sphere intensity are obtained from a photodiode radiance standard calibrated against a NIST traceable incandescent lamp, and a scanning spectrophotometer.Dark offset levels to account for pixel-by-pixel variations in dark current are obtained in flight by recording data when the camera is stowed.
Polarimetric calibration compensates for instrumental polarization.A laboratory polarization state generator (PSG) was constructed for this purpose.Illumination from lightemitting diodes (LEDs) is depolarized, collimated, and transmitted through a tiltable plane-parallel glass plate to generate partially polarized light.A rotating high-extinction polarizer can also replace the tilt plate in order to provide a source of fully polarized light.Because of the great care taken in controlling instrument polarization, the AirMSPI camera operates as an imaging polarimeter with high DOLP accuracy even prior to calibration; however, refinement is necessary to meet the required DOLP uncertainty of < ±0.005.Rotation of the high-extinction polarizer in 10º steps provides the necessary observations to determine polarization calibration coefficients on a pixel-by pixel-basis (Diner, 2010).Once these coefficients are known, the calibrated Stokes parameters are determined.

3.1
A sequence of processing steps is needed to a) convert digital numbers obtained by very high temporal sampling of the modulation patterns into pushbroom frame-based intensity and polarization parameters, b) apply geometric, radiometric and polarimetric calibration, c) co-register data from all spectral bands and all viewing angles to a common Earth based map projection grid, and d) produce multiple superimposed layers of orthorectified intensities (I), degree of linear polarization (DOLP), and angle of linear polarization (AOLP).Processing is fully autonomous and initiated by a cam.run file which is essentially a logbook of on-board data collection operations.In addition to the on-board acquired imaging and pointing data a set of ancillary files needed for radiometric and polarimetric calibration as well as for georectification is prepared prior to flight.Primary data products are provided in HDF-EOS5 version and will be served to the user community from NASA's Langley Atmospheric Data Product Center (ASDC).

3.2
Products are generated from each of four stages of processing.The first stage (L1A1) reformats the raw L0 binary data into Hierarchical Data Format (HDF) with some minor processing, to correct for inconsistencies resulting from data loss or other errors in the raw L0 data.The resulting L1A1 product contains the raw data numbers per-pixel per-subframe.The second stage (L1A2) performs a linearization correction, without any data reduction, thus outputting an identical format product as L1A1.The third stage (L1B1) reduces subframes to frames, while solving for intensity and polarization.The resulting L1B1 product contains radiometrically calibrated intensity and Stokes parameters (Q and U) per-pixel per-frame.The final stage (L1B2) computes DOLP, AOLP, solar and view geometry, and map-projects the result onto a UTM grid.
Copies of the L1B2 data products will be sent to NASA Langley Atmospheric Sciences Data Center (Distributed Active Archive Center), where it is permanently archived and made available for other researchers.L1B2 products contain mapprojected data for each of 8 wavelengths: 935,865,660,555,470,445,380 and 335.All wavelengths include intensity, time, solar zenith, solar azimuth, view zenith and view azimuth.Wavelengths for which polarization is available (865, 660, and 470), also include DOLP, AOLP, Q, and U, where the last three are reported as relative to both scattering and meridional planes.For targets acquired with step-and-stare mode, separate product files are generated for each stare angle at a nominal resolution for that angle.Nominal resolutions are 7 meters at the nadir view; 10 meters at view angles within approximately 49 degrees of nadir; and 20 meters at view angles beyond 49 degrees.

3.3
Data reduction from as acquired sub-frames into frames, which represent image lines in the output product, requires a precise timing definition based upon the on oscillation frequencies of each PEM.Essentially, the retrieval of the stokes parameters Q and U is enabled by the modulation properties of the PEMs.A signal received by a 45 (or 0) degree polarization channel consists of a constant component proportional to intensity, plus a modulated component proportional Q (or U).The modulated components are modelled by a Bessel function, varying in amplitude as a function of time, with a repeat frequency configured to produce one cycle for each frame (approximately 44 ms).The AirMSPI camera sub-samples each frame at a rate of approximately 1.9 ms (~24 subframes), providing multiple signal measurements as a function of time.By combining the known Bessel modulation function and multiple subframe measurements, a set of basis functions are solved for by least-squares analysis to obtain estimates of intensity and Q or U, depending on the channel.

3.4
In nominal operating mode, AirMSPI will collect 13 layers of imaging information for each of nine view angles.Thus a total of 117 image layers, intrinsically displaced to start with, will need to be geolocated and orthorectified to satisfy the coregistration and spatial mapping subpixel accuracy requirements of 0.5 pixels or better.This assumes taking into account all sources of geometric distortions including: a) sensor nonidealities, b) variations in platform position, attitude and velocities, and c) panoramic and topographic effects related to the imaging geometry.As a part of the process, particular errors that need to be taken into account are the orientation of AirMSPI to the ER-2 INU/GPS frame of reference, and the errors in the supplied navigation data.
Processing algorithm relies upon the automatic identification of GCP's image chips in order to recover accurate viewing geometries prior to ray casting and resampling into a reference map.It is expected that AirMSPI will fly target areas that do not have available GCP's, such as over ocean or completely cloudy conditions.Thus our approach is to recover both static camera model and time varying pointing geometries by taking into account the slowly changing errors in the supplied navigation data whenever it is possible, i.e. when there are a sufficient number and distribution of the GCP's present.Such a retrieved static camera model can then be applied to the flight lines where no GCP's are available so that the accuracy of the resulting product will be limited only by the accuracy of the supplied navigation data.
It should also be noted that only after the co-registration is completed that the last steps in the processing pipeline, such as the application of the polarization calibration coefficients, the computation of the degree of linear polarization and the angle of linear polarization, can be accomplished.A part of the coregistration algorithm requires determining the Sun and view angle geometries explicitly and so they are reported in the product.

Geometric Model:
In order to maintain the ability to deal with various errors as specifically as possible, depending on the target conditions, we implemented a rigorous model of the viewing geometry.Essentially, to find the geolocation corresponding to a pixel's field-of-view, the pixel pointing direction is expressed in the geocentric coordinate system, as follows: In this model ‫ݎ‬ Ԧ is the pixel pointing direction relative to the camera coordinate system defined by the image line sample coordinates and the set of parameters defining the camera internal viewing geometry.Those are: 1) a set of polynomial coefficients modelled by Code V and used to convert real image coordinates to the ideal (i.e.paraxial), thus accounting for optical aberrations of the camera, and 2) the as-specified camera effective focal length and location of the principal point relative to the detector line array used to convert paraxial coordinates into the pixel pointing direction.
The rotational matrix ܶ represents the orientation of the camera frame of reference to the axis of gimbal assembly and nominally is equal identity.ܶ ത accounts for the rotation of the gimbal around its pitch axis with the angle provided by the assembly as a time-indexed sequence list.ܶ defines the orientation of the gimbal reference axis to the aircraft coordinate axis.This orientation, represented by the 3 static angles, needs to be calibrated post flight.Similarly, the transformation between the aircraft and Earth geocentric coordinate axis, ܶ ா , contains aircraft orientation data that needs to be adjusted post flight.In our model, corrections to the supplied aircraft data are formed as time dependent piecewise functions.This model is well suited for either calibration or error recovery via a simultaneous "bundle adjustment" (Mikhail, 2001), or as part of ray casting during orthorectification and resampling into the reference map grid.The navigation specifications presented here are our best interpretation of the information contained in the reference documents (LTN-92 and LTN-2001).The following plots illustrate the ER-2 navigation information as linked to a part of the AirMSPI acquisition sequence.

Ancillary
Figure 1.ER-2 Attitude Pitch and Roll angle linked to the AirMSPI gimbal pointing.The first time sequence between seconds 1400 -1700 represents a "continuous sweep", while remainder of the data line is a "step and stare" mode for the acquisition of nine view angles.A small change in the "stare" angle is intentionally commanded to increase common overlap.

Digital Elevation Model:
The AirMISR georectification process relies on the already available Digital Elevation Model (DEM).Almost the entire US area is covered by a DEM of sufficient resolution ranging from 1 m up to 30 m elevation postings.The maximum geolocation error resulting from the use of the existing US DEMs is less than 40 m for the most oblique view angles.Ground Control Imagery: For the extraction of the GCP's image chips, orthorectified imagery with a resolution of 10 m or better will be used.For the engineering flights we relied on the USGS DOQ 4 m data meeting National Map Accuracy Standards at 1:12,000 scale for 3.75-minute quarter quadrangles and at 1:24,000 scales for 7.5-minute quadrangles (USGS 2001).For flight missions outside the US the increasingly available high resolution DEMs and orthorectified imagery from commercial vendors will be acquired.

Algorithm and Results
: By assuming no errors in the overall camera pointing, co-registration and mapping will completely rely only on the explicit ray casting algorithm where the imaging ray for every pixel is projected to the intersection with underlying DEM to form an orthorectified image.
In order to maintain highest possible resolution quality, each individual pixel is represented by a configurable matrix of subpixel elements along with a configurable step size in the search for the best ray intersection with the surface.This part of the algorithm has been executed and validated during the processing of data from the early engineering flights.In those cases our on-board mechanism to properly timestamp data was not yet fully implemented.At the same time some loss of the navigation data due to a buffer overflow condition occurred.nm Intensity; Bottom -470, 660, 865 nm DOLP So, after some manual time alignments with existing navigation data, to remove gross errors, we projected the imagery without any pointing correction.Even though the absolute geolocation was significantly off target, the band-to-band co-registration within any "step and stare" view angle was accurate, as anticipated.This demonstrated that our camera internal viewing geometry model parameters are accurately predicted, specified, and implemented.Figure 2 shows imagery formed by various band combinations, demonstrating our ability to co-register to the highest possible resolution by properly accounting for the physical displacements of the detector line arrays in the focal plane and any optical aberration in the camera.
However, the full implementation of the algorithm must include recovery of the static (gimbal mechanism orientation to the ER-2 frame of reference) and the dynamic pointing as contained in the supplied navigation prior to final mapping.This was done based on the availability of the GCPs and suitable tie points which are automatically identified in the multi angular imagery using well known image matching algorithms (Gruen, 1996;  As implemented, simultaneous bundle adjustment solves for the parameters of interest by a non-linear least square fit with respect to this model: where the viewing vector ߩ ሬሬሬԦ is defined in eq. ( 1), ‫ܩ‬ and ܲ are three-dimensional coordinates in the Earth centred coordinate system of the GCPs and projection centres at the time of imaging, respectively.Static corrections represented by ܶ of eq. ( 1) are solved to be: Yaw = +0.084º,Pitch = -1.911º and Roll = -1.390º.Dynamic corrections modelled as piecewise linear functions with respect to time for positions ܲ and attitude angles ܶ ா are also obtained with particular results for Pitch and Yaw angle shown in Figure 4.
Once the geometric calibration of the in-flight data is completed and the viewing model is updated, the final ray casting step to orthorectify and map project the imagery is executed.Coregistration performance of the final product can be seen in the false color RGB image of Figure 5. Quantitatively, as obtained from the least square fit, the mean mapping error is expected to be around 1.3 m with standard deviation of around 11 m.It should be pointed out that these results were just obtained in time to be included in this article prior to the submission due date.They are obtained from a single flight data collection and not yet fully validated.As the development effort is still in progress, model and operational approaches underlying these results may not be exactly the same as those finally adopted.However, the current status is more than promising in terms of successfully completing development and having a system

SCENTIFIC APPLICATIONS
The Earth Sciences Decadal Survey (National Research Council, 2007) identifies an Aerosol-Cloud-Ecosystem (ACE) mission as a future NASA priority, with science objectives that include understanding the sensitivity of Earth's climate and hydrological cycle to aerosols and clouds as a function of amount and type.The report calls for "a highly accurate multiangle-multiwavelength polarimeter to measure cloud and aerosol properties [that] would have a cross-track and alongtrack swath with ~1 km pixel size."The rationale for this recommendation is that integration of multiple view angles, multispectral measurements, and polarimetry is a powerful method of sensing the microphysical and optical properties of airborne particles.A spaceborne MSPI instrument would be a candidate for the ACE payload, and AirMSPI serves as a Co-registration errors up to 2 km prototype for such a sensor.In addition, data collected by AirMSPI is applicable to aerosol and cloud remote sensing at much higher spatial resolution than would be obtainable from ACE.This is where the scattering feature known as the glory is found.Surrounding the colored rings is a bright, white region where DOLP is relatively high and spectrally neutral.This feature occurs at a scattering angle near 145º, and is known as the cloudbow.The detailed spacing of the supernumerary arcs is related to the droplet size distribution of the cloud particles.This image pair is an example of the type of information about particle microphysics contained in polarimetric imagery.

SUMMMARY
In parallel with all other technology developments towards advanced multi-angle, multi-spectral, polarimetric imaging from space, a data production system that focuses on airborne data acquisition is being developed.In this paper and introduction to the newly built AirMPSI instrument and its scientific applications as the context is given along with a report on the processing algorithms including post-flight geometric calibration operations.Given the instrument design, highly accurate co-registration and mapping is a critical prerequisite for subsequent science retrievals and must be accomplished autonomously as part of the ground processing.Results summarized here represent as expected performance for the data to be acquired in support of the Southeast Asia Composition, Cloud, Climate Coupling Regional Study (SEAC 4 RS) which will take place in August and September of 2012.This work is being done at the Jet Propulsion Laboratory, California Institute of Technology, under contract with NASA.
Copyright 2012 California Institute of Technology.Government sponsorship acknowledged.

Figure 3 .
Figure 3. Before adjustment: AirMSPI 660 nm Intensity layers (upper right corner): 1) Red: +60º, 2) Green: 0º, and 3) Blue: -47.5º.The data acquired on the latest engineering flight in January, 2012 over Fresno, California, are used to demonstrate the performance of the in-flight geometric calibration.In Figure 3 a subset of the 660 nm Intensity layers (upper right corner) from three widely separated viewing angles as projected to the UTM map projection grid are shown as a false color RGB image indicating the magnitude of the co-registration before adjustment.

Figure 4 .
Figure 4. Adjustment results: Piecewise linear time dependent ER-2 corrections, pitch attitude angle and position in the North direction.

Figure 5 .
Figure 5.After adjustment: AirMSPI 660 nm Intensity layers (upper right corner) 1) Red: +60º, 2) Green: 0º, and 3) Blue: -47.5º.ready to support scientific deployments near the end of summer, 2012.Remaining near term validation efforts include: a) evaluation of the stability of the static camera orientation across multiple flight campaigns, and b) further analysis on the best utilization of navigation data as related to the optimization of the representative piecewise functions.The upcoming SEAC4RS preparatory flights in the May/June, 2012 time frame will provide the necessary multi-flight data needed to complete the validations.Additional updates of the processing system also include expanded algorithms to support co-registration and mapping to a surface defined by cloud top boundaries.

Figure 6 .
Figure 6.AirMSPI intensity (left) and DOLP (right) imagery acquired during a "continuous sweep" sequence on August 31, 2011 off the coast of California.

Figure 6
Figure 6 is an example set of AirMSPI images acquired during a flight off the coast of Southern California on August 31, 2011.These images are from one pushbroom slew acquired during a "continuous sweep" sequence as the ER-2 was flying northward.The bowtie shape is due to the increase in linear swath width as the camera points off-nadir.The left image is a "true" color intensity view showing a field of stratocumulus clouds, with San Clemente Island near the top of the scan.The right-hand image is the same scene in polarized light, and is a color composite of DOLP at 470, 660, and 865 nm.Several notable features are apparent in the polarized view that are invisible in intensity.The circular colored fringes are so-called supernumerary bows, or interference fringes resulting from different optical paths of light at different wavelengths through the spherical cloud droplets.At the center of the rings is the backscatter direction (scattering angle = 180º) where the direction to the Sun is directly opposite the direction of view.This is where the scattering feature known as the glory is found.Surrounding the colored rings is a bright, white region where DOLP is relatively high and spectrally neutral.This feature occurs at a scattering angle near 145º, and is known as the cloudbow.The detailed spacing of the supernumerary arcs is related to the droplet size distribution of the cloud particles.This image pair is an example of the type of information about particle microphysics contained in polarimetric imagery.
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B1, 2012 XXII ISPRS Congress, 25 August -01 September 2012, Melbourne, Australia Lockheed-built ER-2 nose rack.Flight control software consists of data acquisition routines, gimbal operation instructions, and a main C&DH program.ER-2 attitude and position data are received by a Condor CEI-200 two-channel ARINC-429 board in the AirMSPI on-board computer.