SENSOR + + : SIMULATION OF REMOTE SENSING SYSTEMS FROM VISIBLE TO THERMAL INFRARED

During the development process of a remote sensing system, the optimization and the verification of the sensor system are important tasks. To support these tasks, the simulation of the sensor and its output is valuable. This enables the developers to test algorithms, estimate errors, and evaluate the capabilities of the whole sensor system before the final remote sensing system is available and produces real data. The presented simulation concept, SENSOR++, consists of three parts. The first part is the geometric simulation which calculates where the sensor looks at by using a ray tracing algorithm. This also determines whether the observed part of the scene is shadowed or not. The second part describes the radiometry and results in the spectral at-sensor radiance from the visible spectrum to the thermal infrared according to the simulated sensor type. In the case of earth remote sensing, it also includes a model of the radiative transfer through the atmosphere. The final part uses the at-sensor radiance to generate digital images by using an optical and an electronic sensor model. Using SENSOR++ for an optimization requires the additional application of task-specific data processing algorithms. The principle of the simulation approach is explained, all relevant concepts of SENSOR++ are discussed, and first examples of its use are given, for example a camera simulation for a moon lander. Finally, the verification of SENSOR++ is demonstrated.


INTRODUCTION
There is a need for simulations of remote sensing systems to assist the development process of new sensor systems.The simulation results can assist the optimization and verification procedures.It becomes possible to test algorithms and estimate important performance parameters like the signal to noise ratio.The goal is to provide a modular software for simulations of a broad spectrum of sensor types from panchromatic cameras to multi-or hyperspectral sensors.These simulations can cover the spectral range of light from visible to thermal infrared.Different scenarios, like remote sensing of the earth or observing space debris, are possible.

SENSOR SIMULATION
The idea, to split the sensor simulation into three parts (geometry, radiometry, and sensor), is taken from (Börner et al., 2001).The main extensions to this concept are the use of more general triangle meshes as geometric data input, and the enlargement of the simulatable spectral range to the thermal infrared.Every of the following three modules can work on its own, but a final sensor simulation will use all of them.

Geometry Module
The geometry module of the sensor simulation has to answer the question, where does every camera pixel look at?For this purpose, a ray tracing approach is used (Whitted, 1980).This uses a triangle mesh and a pinhole camera model as input and searches the intersections of every ray, constructed from the camera description for every pixel, with the triangle mesh.The nearest intersection is used as result.Furthermore, the same procedure can be applied to determine whether the area is shadowed or not, simply by using rays from the intersection points to the sun.The observed area is shadowed if there is another intersection of the mesh between the sun and the observed intersection point.For every camera pixel, the geometry module gives information about the intersection point, the corresponding surface normal, whether the point is shadowed, and the material of the point.Due to the lack of reflections and refractions during the ray tracing procedure, we can use a faster algorithm for the hidden surface determination.This algorithm is based on a z-buffer (Catmull, 1974).The shadow calculations are done by shadow mapping (Williams, 1978).To avoid the sampling problem, we use an irregular zbuffer (Johnson et al., 2004) for the shadow mapping.In this way, we achieve the same precision with the z-buffer algorithm as with the direct ray tracing approach.An example of an irregular shadow map is shown in Figure 1.
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B1, 2012 XXII ISPRS Congress, 25 August -01 September 2012, Melbourne, Australia The at-sensor radiance can be split into 5 parts.L ind , Lsc, and L th are only nonzero in the presence of an atmosphere.Actually, there can be one more indirect reflected component due to scattering from surrounding geometry.Fortunately, this is an insignificant contribution in most cases, otherwise it could be approximated by an ambient component.

Radiometry Module
The purpose of the radiometry module is to quantify the radiative transfer from the surface to the sensor.This is done for every intersection point corresponding to the sensor pixels.Fortunately, we have to consider only one light source, the sun, which can be seen as a directional light source.However in the presence of an atmosphere, the radiative transfer becomes a complicated process with a lot of scattering.Similarly to (Wiest, 2001), we can split the spectral at-sensor radiance into 5 radiometric components: where Every of these components is a spectral radiance.The direct reflected radiance can be calculated with where Esun = spectral irradiance of the sun ωi = unit vector pointing from the surface to the sun n = surface normal vector By using the spectral radiance of a blackbody B(T ) at a surface temperature T , we can calculate the emitted radiance The values of Esun, L ind , Lsc, L th , and τ are computed with MODTRAN (Berk et al., 1987) for the appropriate model parameters, especially the sensor zenith angle, the sensor height, the sun zenith angle, the visibility, and the model atmosphere.MOD-TRAN calculates the radiative transfer through the atmosphere of the earth regarding multiple scattering.
To simulate different materials, we can replace the Lambertian factor ρ/π in Equation ( 2) by the bidirectional reflectance distribution function BRDF (Nicodemus, 1965) with the unit vector ωo pointing from the surface to the sensor pixel.For our simulations, we consider only physically reasonable BRDFs, so the following conditions: the Helmholtz reciprocity and the conservation of energy have to be fulfilled.

Sensor Module
The final module converts the at-sensor radiance into a digital number for every sensor pixel.At this stage of the simulation, it is also possible to consider optical effects like the point spread function.Furthermore, different detector models can be used for the simulation.In the case of a quantum detector, we can compute the number of electrons which get excited by the incoming photons of one spectral channel (Börner et al., 2001): This number can be easily converted into a digital number.It is also possible to add noise to the result.For this purpose, one can take advantage of the Poisson distributed nature of n el .
Another detector type is a micro bolometer matrix which is used, for example, in the Mercury thermal infrared imaging spectrometer MERTIS (Helbert et al., 2005).MERTIS is a hyper-spectral sensor with 80 channels in the range of 7 µm to 14 µm.The purpose of this instrument is to measure and map mineralogical spectra of Mercury's surface in the thermal infrared range.In this case, we want to simulate the radiometric performance of the sensor and for this purpose, we only need the radiant flux (Paproth et al., 2010) The geometry part of the the sensor simulation of MERTIS currently consists only of the spectral assignment of the bolometer pixels.And the at-sensor radiance, Equation (1), is equal to the emitted radiance, Equation (3), L = Lem, because of almost no atmosphere on Mercury.The transmittance is simply τ = 1 and η also includes the grating efficiency of the spectrometer.

Moon Lander
One application of SENSOR++ was the creation of a camera simulation for a hypothetical moon lander including the simulation of about 100000 images.That was the main motivation for accelerating the ray tracing procedure of the geometry module by using a z-buffer algorithm.Another challenge was the handling of the large digital elevation model DEM of the moon.The DEM is gotten from the Kaguya mission (Kato et al., 2010) and consists of tiles in geographic coordinates.Theses tiles are transformed into triangle meshes with an adequate spatial resolution to minimize aliasing during the rendering procedure.
Due to almost no atmosphere on the moon, the at-sensor radiance calculation simplifies to Equation ( 2), L = L dir with τ = 1.To account for the opposition surge of the moon regolith, we use the Lommel-Seeliger law (Fairbairn, 2004) to describe the BRDF of the moon surface Actually, we use a weighted average of the Lambertian BRDF Equation ( 4) and the Lommel-Seeliger BRDF Equation ( 10) for the moon material simulation.It can be shown that the physically motivated constraints of Equations (5-7) are fulfilled by Equation (10). Figure 3 shows one of the simulated images with the expected maximum at-sensor radiance according to the viewing conditions: sun position, sensor orientation, and surface normals.
The resulting images are used in a simulation framework for the optical navigation of the moon lander.

Space Debris
In addition to the simulation of remote sensing scenarios, the modular concept of SENSOR++ enables us to simulate scenarios with the sun as the only significant light source generally.One of theses scenarios is the observation of space debris.The simulation of such a scenario can help to estimate the performance of an optical sensor for the identification of old or defect satellites during, for example, an orbit cleanup mission.In this context, we expect a high radiometric dynamic due to specular reflections.Therefore, we have to model a specular reflecting material.A normalized Blinn-Phong (Blinn, 1977) BRDF based on the halfway vector ωi + ωo: is used for this task, with the parameter p ≥ 0 to adjust the strength of the specular reflection.Notably, this becomes Equation (4) for p = 0.One can show that the Equations (5-7) are also Figure 3: Camera simulation for a moon lander.The shadows are generated by using an irregular shadow map, see Figure 1.
The triangle meshes of the different visible moon DEM tiles are put together, so that, only one continuous mesh without holes is rendered.
Figure 4: Simulated camera image with a simple satellite model under extreme viewing conditions, so that there is a strong specular reflection.The middle part of the satellite is modeled as a Lambertian reflector and the panels are specular reflectors.
fulfilled by Equation ( 11).Identically to the moon lander application, the at-sensor radiance is just L = L dir with τ = 1.In Figure 4, one resulting image with the maximum at-sensor radiance can be seen.The high dynamic range is already visible, but this can be better seen when the satellite is rotating.

VERIFICATION
To begin with, we want to verify the geometry module by extracting a digital elevation model from our simulation result of the moon lander application.For this purpose, we use the calculated intersection points and transform them back to geographic coordinates.The resulting DEM is then compared with the original DEM.A difference image of these DEMs is shown in Figure 5.
As one can see, the differences are evenly distributed around zero, so there is no sign of a systematic error.The main deviation is due to the different sampling of the extracted and original DEM, which results in aliasing in the difference image.But on average, the difference is small with less than 10 cm compared to the sensor height of ≈ 20 km.
Finally, the sensor simulation of MERTIS is verified by comparison of the experimentally determined signal to noise ratio SNR and the simulated SNR (Paproth et al., 2010), which is the main performance parameter of MERTIS.By using the radiant flux, Equation ( 9), we can calculate the signal to noise ratio with the noise equivalent power NEP of the bolometer detector.The experimentally determined SNR is gotten from measurements with a MERTIS-like breadboard.Figure 6 shows that the simulated results are in good agreement with the experiment.So this model can also be used for more complicated simulations like the simulation of mineralogical spectra to estimate, for example, the performance of a spectral feature extraction algorithm.

CONCLUSION AND FUTURE WORK
To sum up, the modular sensor simulation concept, SENSOR++, was presented.It is possible to simulate remote sensing scenarios for different sensor types from visible to thermal infrared.Several bidirectional reflectance distribution functions were provided for radiometrically correct simulations of differing surface materials.Furthermore, applications of SENSOR++ and the verification of parts of this concept were illustrated.In the near future, the focus will be on simulations with atmospheric components contributing to the at-sensor radiance.These will be used to test and optimize algorithms for atmospheric compensation or fire detection.

Figure 1 :
Figure 1: This irregular shadow map was used to produce the shadows in Figure 3. Every cell of the grid can contain an arbitrary number of light rays.Exactly one light ray corresponds to a camera pixel and only this ray is used for the shadow calculation of the pixel.This results in shadows without aliasing.

Figure 5 :Figure 6 :
Figure5: Image of the differences between an original Kaguya DEM tile and a DEM tile extratced from the camera simulation.This tile is also part of the scene in Figure3.The averaged difference is smaller than 10 cm.