TRAFFIC FLOW ESTIMATION FROM SINGLE SATELLITE IMAGES

Exploiting a special focal plane assembly of most satellites allows for the extraction of moving objects from only one multispectral satellite image. Push broom scanners as used on most earth observation satellites are composed of usually more than one CCD line – mostly one for multispectral and one for panchromatic acquisistion. Some sensors even have clearly separated CCD lines for different multispectral channels. Such satellites are for example WorldView-2 or RapidEye. During the Level-0-processing of the satellite data these bands get coregistered on the same ground level which leads to correct multispectral and exactly fitting pan images. But if objects are very high above the coregistering plane or are moving significantly in between the short acquisition time gap these objects get registered on different points in different channels. Measuring relative distances of these objects between these channels and knowing the acquisition time gap allows retrieving the speed of the objects or the height above the coregistering plane. In this paper we present our developed method in general for different satellite systems – namely RapidEye, WorldView-2 and the new Pléiades system. The main challenge in most cases is nevertheless the missing knowledge of the acquisition time gap between the different CCD lines and often even of the focal plane assembly. So we also present our approach to receive a coarse focal plane assembly model together with a most likely estimation of the acqusition time gaps for the different systems.


INTRODUCTION
Most satellite-borne push broom scanners consist of more than one CCD line.Often different spectral channels but mostly the multispectral and PAN sensor CCDs are mounted on different places in the focal plane assembly of the instrument.Due to this construction feature the acqusition of the different CCD lines is not exactly simultaneous.In the production process of the level-1 satellite data these bands get coregistered on a specific ellipsoid height or on a digital elevation model (DEM).While most areas of the two bands fitting together within fewer than 0.2 pixels moving objects or high clouds will not fit onto each other.Specially in the RapidEye imagery the missing coregistration of clouds can easily be seen due to the cyan and red edges at opposite sides of a cloud.Also objects moving during the short acquisition time gap get registered on different pixels in the different channels.
Figure 1: Section 2.1×1.5 km from a RapidEye scene of southern bavaria (north of Füssen) containing clouds and a plane Fig. 1 shows a part of a RapidEye scene containing clouds and a plane travelling from east to west.The different positions of the plane in the individual multispectral bands of the sensor is clearly visible.Also the colored border of the clouds is evident.While the different positions of the plane results from a combination of two effects -the movement and the height above ground -the border along the clouds is only due to the height above the ground or more precise: the coregistration plane.
In this paper we show how to exploit this effect in WorldView-2, RapidEye and Pléiades imagery to detect and derive moving objects like cars, trains or airplanes from the imagery.In WorldView-2 images cars travelling at a speed of about 60 km/h show a shift of about 4 pixels or 8 meters in the multispectral image between e.g. the green and yellow channel while static objects have shifts below 0.5 pixels (the manually measuring accuracy).
In Rapid-Eye images a plane flying across the acquisition direction of the sensor shows a shift of about 108 pixels or 540 meters between the red and the green band.Between the blue and green band there are still 13 pixels or 65 meters (all measured in orthorectified imagery).Cars on a highway show up moving about 12 pixels (60 meters) between the green and red band.In the same Rapid-Eye scene clouds show a shift of about 45 meters between the red and green band in acquisition direction.Assuming not moving clouds allow together with the estimation of the acquisition geometry and -times the estimation of cloud heights.Also -if a plane flies across the acquisition direction -speed and height of the plane are separable and can be retrieved independently.Planes flying along the acquisition direction mix up these information and for retrieving one the other has to be estimated -e.g. if the height of the plane is estimated the speed may be calculated.
This paper focuses mainly on the estimation of the time gap for the RapidEye sensor since this measure is as unknown as the exact focal plane assembly which was extracted from the resulting imagery and some sparse information.
Beside exploiting these effects also the time distance between the band acquisitions is estimated for the sensors and for WorldView-2 and Pléiades images also a first approach of an automatic detection of moving traffic is shown.

Sensor composition
The WorldView-2 multispectral instrument consists of two CCD lines acquiring in the first the standard channels blue, green, red and the first near infrared band and in the second the extended channels coastal blue, yellow, red edge and the second near infrared band.These two CCD lines are mounted on each side of the panchromatic CCD line.Therefore the same point on ground is acquired by each line at a different time.Fig. 2 and fig. 3 show the focal plane assemblies (FPA) of WorldView-2 and the Pléiades push broom scanner instruments respectively.In table 1 from Kääb (2011) the time lags for the sensor bands are given.The Pléiades FPA is similar but consists only of one multispectral and one panchromatic sensor line.The main gap exists only between the multispectral bands and the pan channel where the latter is also mounted in a curvature around the optical distortion center (marked with a × in the figure).

Preliminary work
The first one exploiting the time lag between different bands in a very high resolution (VHR) push broom scanner was Etaya et al. (2004).He used QuickBird images of 0.6 m GSD panchromatic and 2.4 m multispectral and found a time gap between these bands of about 0.2 s.
Also M. Pesaresi (2007) used QuickBird imagery with also a time lag of 0.2 seconds between the panchromatic and the multispectral bands.
Tao (Tao and Yu, 2011) proposed 2011 in an IGARSS paper first the usage of WorldView-2 imagery for tracking of moving objects.He calculated from a plane arriving at the Shanghai airport a time delay between the Coastal Blue Band on the second multispectral sensor line and the Blue Band on the first multispectral sensor line of about 17.5 m/80 m/s = 0.216 seconds.
Delvit (Delvit et al., 2012) described in his work on "Attitude Assessment using Pleiades HR Capabilities" the Pléiades focal plane (as shown in fig.3).Here the panchromatic/multispectral shift is significant: 19 mm in the focal plane, which means 1 km on ground or a time delay of 0.15 seconds or in turn also a 1.5 mrad stereoscopic angle.He also describes the maximum offset between two multispectral bands as 6 times smaller (maximum 3 mm).The 1.5 mrad stereoscopic angle means a height of about 300 m corresponds to 0.5 m shift (1 GSD of the pan channel).In turn using a matching accuracy of about 0.1 pixels allow for the extraction of a DEM with an uncertainty of 120 m (0.1 × 4 × 300 m for the multispectral GSD pixel size).
Finally Leitloff (2011) gives in his PHD thesis a short overview of more of these methods and proposed also some approaches for automatic vehicle extraction.

METHOD
Most images acquired by push-broom-scanners with focal plane assemblies like those shown in fig. 2 to 4 cause small time gaps between image bands in different mounts as illustrated in fig. 5.The 900 × 300 m section of a RapidEye image in fig.6 contain some moving cars along the highway.These cars can be seen as red and cyan dots.The cars on the highway verify clearly the acquisition order: first blue/green and red after the main time gap.This shift is due the car moving some distance (60 m on the highway) between the acquisition of the bands.A first estimation of the time gap assuming a speed of about 120 km/h on the highway leads to a relatively large ∆t of 1.8 sec which is about ten times the order of the time gaps found in VHR imagery from e.g.QuickBird, WorldView-2 or Pléiades.A first point of reference is a statement from RapidEye AG itself: "This means that the bands have imaging time differences of up to three seconds for the same point on the ground,..." (RapidEye (2012), p. 11).Therefore the result for the time delay between the blue and the red band should be approximately 3 seconds.
In general we use for better results of the measurements the distances between red and green band since the green band is less noisy and spectrally closer to the red band.To estimate this height a simulation of a geographic cloud shadow was developed using both the satellite viewing azimuth and incidence angles and the sun azimuth and zenith angles.This is nearly 1.3 times the value of 3 seconds as stated by Rapid-Eye.Repeating this procedure for 9 different clouds with clear shadows we gain a statistical result of 3.46 ± 0.51 seconds for the time lag between the red and the green channel.

Calibration Rapid-Eye -Cars
The car measurement follow fig.6.For the analysis 67 cars on highways, 12 cars on ramps, 28 cars on secondary roads, 25 cars on city roads were measured.The results are: The error for the speed estimation was estimated as 20 km/h, the measurement error was estimated as 5 m (1 pixel).Taking into account these estimation and measurement errors together with the statistical error σ∆t provides the overall error ε in tab. 2.

Calibration Rapid-Eye -Planes
Deriving ∆t from planes is more complicated than using clouds or cars since in flight direction of the satellite also the travelling height of the plane above ground gives an additional shift.As shown in fig. 10 we have to split all distances and also the travelling speed of the plane in components across (dc, vc) and along (da, va) the satellites acquisition direction αs.The movement of the plane across the flight direction dc is independent of the influence of the FPA and therefore only depending on the speed component in this direction and the time lag ∆t between the bands (we take for all measurements d the red and green band as stated above): In flight direction of the satellite the displacement da is composed from the speed component in this direction va • ∆t and the height of the plane above ground (see above section "clouds"): The flight directions v (heading) and the distances d = (dx, dy) were measured directly, the along and across values were calculated using αs.Tuning the absolute value of the speed v gives directly the requested ∆t and also solving the equation above for H f the height above ground of the plane.For the results in tab. 3 we tuned the estimated speeds to give somehow reasonable flight heights H f of the planes.
Table 3: Plane measurements in RapidEye image, v is the heading of the plane in degree (north=0, east=90, . . .), The positions of plane 4 in the red and green band are below 0.5 px in flight direction of the satellite and so below the measurement accuracy.For this dc < ε the ∆t = dc/vc = 0 and no height H f may be calculated.The resulting times give a mean ∆t = 2.56 ± 0.42 s.The measurement error of the position ε d = 0.06 s, the uncertainty from the speed/height estimation of εv = 0.23 s and the statistical standard deviation of σ∆t = 0.34 s sum up to an overall error of ε = 0.42 s.The time distance between the stereo pair ∆t13 = 43 s is known while the time distance between the acquisition of the two multispectral CCD arrays ∆tyr is unknown.Measuring all positions P and assuming a constant velocity v = v1 = v3 = vy = vr the searched ∆tyr can be calculated as

Calibration
The assumption of constant velocity can be proved coarsly by verifying v1 = v3 or |P1y − P1r| = |P3y − P3r| at the both endpoints of the stereo-acquisition.Repeating this measurements for many cars and checking the measured speeds v with the roads on which the cars are measured for plausibility gives a result of ∆tyr = 0.297 ± 0.085 s.The error is resulting from an inaccuracy of 1 second in the acquistion times of the images and a measurement inaccuracy of 0.5 pixels (or 1 m).This results in an error εv of about 3 km/h in the estimated speed between the stereo images and such all together a measurement error of ε∆t = 0.062 s.
Together with the statistical error of σ∆t = 0.059 s we gain the overall error of ε = 0.085 s.

Calibration Pléiades
The calibration of the Pléiades time gap ∆tms,pan works in the same way as for WorldView-2.But here the calculation has to be done between the pan channel and the multispectral channels as shown in fig.

CONCLUSION AND OUTLOOK
In this paper we presented a new method for exploiting the special focal plane assembly of most earth observation satellites based on push broom scanners to extract moving objects from only one single multispectral image.For deriving the correct speed of the detected moving objects the exact time gap between the acquisitions of the different bands used has to be known.For Rapid-Eye this time was estimated using moving cars, cloud heights and planes.For satellites capable of aquiring short-time in orbit stereo images like WorldView-2 or the new Pléiades system an absolute calibration of the inter-band time gap of one image is possible from using a stereo pair.
Implementing this method the whole traffic flow of a complete satellite scene can be extracted to provide a full large-area situation snapshot as extension to long time but only point wise car measurements of local authorities.
Applying automatic image matching and object detection algorithms may help in future to speed up the process of large area traffic monitoring from satellites and is subject to future investigations.

Figure 3 :
Figure 3: Focal plane assembly Pléiades As shown in fig. 4 the RapidEye focal plane assembly consists of five separate CCD lines -one for each band.They are grouped in two mounts: the blue and green on one and the red, red edge and the near infrared band on the second.

Figure 4 :
Figure 4: RapidEye focal plane assembly (FPA), g: gap between lines, Dp: distance between packages, D * : maximum distance between lines, D: distance red-green So the main gap can be found between the blue/green and the red/RE/NIR bands.The nominal orbit height for RapidEye is given as 630 km.Using the NORAD two-line-elements (TLE) for exact calculation gives an orbit height of 637.5 km. and an average speed of 7.53902 km/s resulting in an orbit of about 97 minutes length.

Figure 5 :
Figure 5: Principle of acquisition geometry of image bands separated in a FPA Since for RapidEye the exact geometry of the focal plane assembly (FPA) was not known several approaches were investigated to estimate the time lags between the bands of this satellite:

Figure 7 :
Figure 7: Acquisition geometry of clouds in RapidEye imagery, left: side view, right: top view In a first step the borders of clouds like in fig.8 may be used for this purpose.As shown in fig.7 the (exaggerated) time delay ∆t in aquring the same point on a cloud border results in a lateral shift ∆s on ground in the band-coregistered level-1 images.The time delay ∆t may be expressed as

Figure 8 :
Figure 8: Example of a cloud in the RapidEye image near Schongau, section 3.1 × 3.1 km, left RapidEye image, right ellipsoid DEM ranging from 745 to 845 m (WGS84)

Fig. 9
Fig. 9 shows the cloudmask derived from the image section in fig.8, left and simulated cloud shadows for different cloud heights h e c ranging from 1000 to 2000 m above ellipsoid.As can be seen in the figure a cloud height on DEM level (h = 800 m in this area) will drop no shadow outside of the cloud.Raising the height above the DEM height shifts the cloud shadow to the north east.Measuring the distance of the cloud to the shadow in the Rapid-Eye images gives a distance of dsc = 1052±10 m.The simulated

Figure 9 :
Figure 9: Cloud mask from previous image and simulated shadow masks for cloud heights (above ellipsoid) 1000, 1500, and 2000 m cloud height corresponding to this distance is h e c = 2019 ± 12 m above ellipsoid or hc = 1219 ± 15 m above ground.Using the satellite height Hs = 631900 m − 800 m = 631100 m and speed vs = 7539 m/s together with a measured cloud border of ∆s = 55 ± 10 m gives for the example cloud a

Figure 10 :
Figure 10: Acquisition geometry of planes in RapidEye imagery, left: side view, right: top view Rapid-Eye -Relative distances in FPA Analyzing a flying plane in all five channels of a RapidEye image as shown in fig.11 let us derive the absolute distances between all channels and also the relative distance of the channels in the FPA (see fig. 4 and tab.4).So the assumption for constant g (see fig.4) between all channels can be assured and a D = dNIR−G + 2g = 8.5g or Dp = dNIR−G − 2g = 4.5g can be derived.

Figure 12 :
Figure 12: Time calibration for WorldView-2 stereo imagery; left: a car with positions in red and yellow channel of the first stereo image; right: the same car in the second stereo image Fig.12shows the principle of the absolute calibration using a stereo image pair: For this approach cars are searched in the imagery for which a constant travelling speed between the acquistion of the two stereo images (19 or 43 seconds) is highly probable -e.g. on highways.Let the two images be 1 and 3 and let the multispectral channels used be r (red) and y (yellow).So we have positions of a car P1r, P1y, P3r and P3y.Please remember the focal plane geometry of WorldView-2: the red and the yellow bands are located in the two different multispectral CCD arrays on each side of the panchromatic CCD array.For such cars the following travelling speeds can be calculated: 13.For the calibration a multi-stereo image set from Melbourne acquired 2012-02-25, 00:25 was available in the scope of the "Pléiades User Group" program.The result for Pléiades based on measuring 14 cars is ∆tms,pan = 0.16±0.06s (σ∆t = 0.05 s, measurement inaccuracy of 1 m: ε d = 0.04 s).

Figure 13 :
Figure 13: Time calibration for Pléiades stereo imagery; left: a car with positions in pan and a multispectral channel of the first stereo image; right: the same car in the second stereo image