A PROCEDURE FOR SEMI-AUTOMATIC ORTHOPHOTO GENERATION FROM HIGH RESOLUTION SATELLITE IMAGERY

The General Directorate of Surveying and Mapping (G DSM), under the Ministry of Municipal and Rural Affa irs (MOMRA) is responsible for the production and dissemination of accurate geospatial data for all the metropolitan cities, towns and rural settlements in the Kingdom of Saudi Arabia. GDSM ma intains digital geospatial databases that support t he production of conventional line and orthophoto maps at scales ran ging from 1:1,000 to 1:20,000. The current procedur s for the acquisition of new aerial imagery cover a long time cycle of three or more years. Consequently, the availability of recen tly acquired High Resolution Satellite Imagery (HRSI) presents an attractive alte rnative image data source for rapid response to upd ated geospatial data needs. The direct sensor orientation of HRSI is not accurate en ough requiring ground control points (GCP). A field survey of GCP is time consuming and costly. Seeking an alternative approach, a research study h as recently been completed to use existing image an d d ta base information instead of traditional ground control for the ortho pr jection of HRSI in order to automate and speed up as much as possible the whole process. Based on a series of practical experi ments, the ability for automated matching of aerial and satellite images by using the Speeded-Up Robust Features (SURF) algorithm is de monstrated to be useful for this task. Practical re sults from matching with SURF validate the ability for multi-scale, multi-sen or and multi-season matching of aerial and satelli te images. The matched tie points are then used to transform the satellite ort hophoto to the aerial orthophoto through a 2D affin e coordinate transformation. GeoEye-1 and IKONOS imagery, when geo-referenced th rough SURF-based matching and transformed meet the M OMRA Map Accuracy Standards for 1:10,000 and 1:20,000 scale. However, a similarly processed SPOT-5 image does n t meet these standards. This research has led to the development of a simpl e and efficient tool for the geo-referencing of HRSI of 0,5m to 1m ground sampling distance (GSD) that can be used for updati ng map information. The process completely eliminat es the need for any ground control as well as image measurements by human oper ators.


INTRODUCTION
The Kingdom of Saudi Arabia has been covered by small and large scale topographic mapping using photogrammetric procedures.In 1950 the Ministry of Petroleum initiated topographic mapping of the entire kingdom at the 1:50,000 scale.Unlike classical triangulation, the geodetic reference frame at that time was established by traverses using electronic distance measurement technology.The resulting network of 700 monumented geodetic stations was adequate to meet the required accuracy for national mapping at the desired scale, but is not suited for mapping at larger scales needed for urbanized and settlement areas.Today, large scale urban mapping efforts are assigned to the Ministry of Municipal and Rural Affairs (MOMRA).For this task MOMRA developed a new geodetic network.This resulted in the establishment of a modern GPS based geodetic reference frame that forms the MOMRA Geodetic Datum 2000 (MGD-2000).Latest photogrammetric technologies have been implemented to meet kingdom wide mapping requirements and to maintain the large scale topographic map database.Approximately 25,000 km² of the settled areas of the country have been covered by 1:5,500 scale aerial photographs in a series of aerial survey projects, and approximately 300,000 km² have been covered by 1:45,000 scale aerial photographs.In addition a large part of the mountainous area has been covered with 1:22,500 scale aerial photography.Using current photogrammetric technology and methods, high quality topographic maps at 1:1,000, 1:2,500, 1:10,000 and 1:20,000 scales have been produced.The Map Accuracy Standards currently followed in MOMRA are given in Table 1.The required horizontal standard deviations with 0.12 to 0.14mm in the map scale are quite challenging.

Map scale
SX, SY SZ 1:1,000 0.15 m 0.30 m 1:2,500 0.35 m 0.50 m 1:5,000 0.60 m 0.75 m 1:10,000 1.25 m 1.50 m 1:20,000 2.50 m 3.00 m Table 1: MOMRA required standard deviations for mapping Aerial survey, line mapping and geospatial database compilation projects take from 2 to 3 years to complete, and in the meanwhile, significant changes in the urban infrastructure are taking place.The usual impact of such rapidly occurring changes is that although the geodata are spatially precise and accurate, they often do not reflect the current status of land on the map.Growth in population, rapid industrialization and migration from remote areas to cities cause the urban areas to grow at a much faster rate than the corresponding growth in population (Figure 1).Therefore, the existing large scale topographic base maps at the different scales 1:1,000, 1:2,500, 1:10,000 and 1:20,000 need to be frequently updated.The time cycle for the current procedure of awarding and implementing the contracts for the acquisition of new aerial imagery and checking it by new geodetic surveys and subsequent photogrammetric data processing and map compilation is by far too long.This approach is both very slow and costly for frequent revision in map database while new changes are constantly going on.Consequently, alternative technologies must be explored.
Figure 1.Size of Ar Rijadh in 1980, 1990, 2000and 2010 The availability of recently acquired High Resolution Satellite Imagery (HRSI) presents an attractive alternative image data source not requiring time consuming photo flights.To also avoid the costly field survey of GCPs, as an alternative the ground control information may be automatically extracted from existing orthophotos using an image matching algorithm.The use of HRSI comprises the question which ground sampling distance (GSD) is appropriate for the required map scales.This includes the semantic information -which details can be identified and are requested for the map product -and the geometric accuracy required by the actual map accuracy standards (Table 1).Issues of object identification in Saudi Arabia, which mostly exhibits relatively large buildings in planned residential areas, surrounded by walls, are less complicated than for small buildings without separation in unplanned areas found in many other countries.In general, as a rule of thumb, 0.1mm GSD in the map scale is required for topographic mapping, corresponding to 0.5m GSD for 1:5000 and 1m GSD for 1:10000 map scales (Jacobsen 2012), so under the special Saudi conditions even larger map scales may be possible with respect to semantic details.The paper describes the results of an investigation to answer the questions raised in the last paragraph with a focus on the required and the achievable geometric accuracy and on automation.More details are available in (Alrajhi 2013).

METHODOLOGY OF AUTOMATIC CONTROL POINT EXTRACTION AND TEST AREAS
In the context described above a choice of HRSI of 0.5m, 1.0m and 2.5m GSD supported by GCPs has been made to cover the most commonly used MOMRA mapping scales ranging from 1:1,000 to 1:10,000.It is known that GCPs are essentially needed to supplement the vendor provided RPCs during HRSI processing to achieve positional accuracy required for mapping at large scales from HRSI (Gordecki and Dial, 2003).The questions to be answered are these: what are the required accuracy, number and spatial distribution of GCPs for the largest scale for mapping from HRSI of 0.5 meter GSD?Since the MOMRA geospatial database maintains seamless digital orthophoto coverage with 0.5m GSD that has been derived from aerial photography at 1:45,000 scale, it was proposed to use such aerial orthophotos to serve as control.
The question which matching procedure to use is partially answered by analysing the problem -due to different image scales, different sensors and different seasons, in which the images were acquired a robust scale invariant method is required.First tests showed, the Scale Invariant Feature Transform -SIFT (Lowe 2004) in general is suitable and can be used also under the difficult conditions of deserts, but finally the Speeded-Up Robust Features (SURF) algorithm (Bay et al. 2006) has been selected because of faster processing.The superior performance of SIFT against image rotation changes is not important because the matched images are not rotated against each other.
To simplify the search for corresponding key points, the satellite images have been transformed to orthophotos using available rational polynomial coefficients (RPCs) for image orientation and the same height model as for the aerial orthophotos.The uncertainty of the direct sensor orientation may lead to small deformations of the satellite orthophotos causing shifts against the correct geo-position, but the geometric offsets of the direct sensor orientation are limited and in the close neighbourhood of the finally selected key points the terrain is dominantly flat.Thus, an iteration of the procedure with improved relation between the height model and the satellite images is not required.Three test areas have been used, namely "Riyadh", a densely urbanized south-western part of Riyadh Metropolis as well as the adjoining suburban area which is undergoing rapid new development, "Al-Muzahimiah", including several cultivated farm parcels with significant variation in topography in a rural landscape, and "Huraymila" with old style farms with a lot of palm trees presenting a landscape ranging from acacia covered wadi to dramatic escapement and wide plains.Figure 2 shows the wide range of the test areas covered in this study.In the first two test areas GeoEye-1 (0.5m GSD), IKONOS (1.0m GSD) and SPOT-5 supermode (2.5m GSD) images have been used, in the last test area only GeoEye-1 and IKONOS images were available.In total 19 satellite images have been used.

Automatic identification of ground control points
To create a reference for the automatic image orientation, manual measurements of a varying number of corresponding points in both satellite and orthophotos have been conducted.As mentioned above the satellite images were first orthorectified using the RPCs, before correcting the remaining bias.The results were then checked at independent check points (Table 2 and Figure 3).For bias correction a 2D-affine transformation was employed, as it is known from experience that the affinity parameters of the bias correction are mostly significant and improve the final results, compared against bias correction using a 2D shift only.

Number of GCPs
GeoEye Of course also the reference aerial orthophoto with 0.5m GSD, (resampled from 0.63m GSD, as the original analogue images were scanned with a pixel size of 14 µm), is not free of errors.It is probably for this reason that a clearly better accuracy has been reached with 15 as with just 5 GCPs.Without GCPs the accuracy of the direct sensor orientation dominates the results; and the resulting discrepancies are not acceptable for the required map accuracy.For GeoEye-1 images the RMS differences at independent check points based on bias corrected RPC-orientation exceed one GSD, but when taking into account that the aerial orthoimage based on a similar GSD must be expected to have a similar geometric accuracy, an estimation of the GeoEye-1 error component for 15 GCP of 0.65݉/√2 = 0.46m seems to be justified.Thus a standard deviation of slightly better than one pixel has been reached.For the automatic determination of the satellite image orientation the SURF algorithm has been used.Since the effectiveness of the matching approach is independent of the size of the two images being matched, but rather depends on the image texture, it was decided not to use the entire satellite image scenes for this experiment.Instead, 800 x 800 pixel patches were extracted from the satellite orthophotos to reflect differences in texture resulting from differences in landscape.Test samples were selected that are representative of typical agricultural, urban and rural landscapes in Saudi Arabia.All three types of satellite imagery, namely GeoEye-1, IKONOS and SPOT-5 imagery were tested.As typical for feature based matching like SIFT and SURF a high number of key points are identified (Figure 4) which have to be reduced to corresponding points (Figure 5) by key point description and geometric plausibility checks.The different imaging conditions of analogue aerial and satellite scenes and the time difference of 3 to 5 years did not cause any problems with the identification of a satisfactory number of corresponding points (Figures 5 and 6).For operational reasons the number of tie points (used as GCPs) has been limited to 200.This number has been reached in all tests.The high number of available GCPs reduces the influence of random GCP discrepancies extracted from the aerial and satellite orthophotos, mainly caused by tie point identification and unfavourable influence of the height model used for the orthophoto generation.The large number of matched tie points, well distributed over the image provides strong least squares based adjustment solution for the subsequent 2D affine transformation.In order to strengthen the solution, large discrepancies between the two sets of coordinates were considered to be blunders and were iteratively eliminated.A special problem occurs for the urban area: conjugate points can be misplaced due to building heights (being different to the heights of the height model used for orthoprojection) and different viewing direction.The employed satellite scenes have an incidence angle of approximately 25°; the aerial images were of course taken as vertical images.With building heights of 7m to 10m, the resulting displacement is between 3.3m up to 4.7m.These values are typically large enough to be identified as blunders and thus eliminated in the 2D affine transformation, and thus do not influence the results.Figure 6.Sample of identified tie points, above urban, below rural, left GeoEye-1, right aerial orthophoto For operational reason the reference orthophoto has been downsampled to the resolution of the satellite orthophotos prior to matching -for IKONOS and SPOT-5 supermode to 1m, respectively 2.5m GSD.This step is not directly required for SIFT and SURF but it improves the matching slightly and has advantages for simple geometric checks for corresponding tie points.A drop in the number of matched points was observed with an increased difference in image resolution.In the case of SPOT-5 imagery, however, it was noticed that matching of the original satellite image with the aerial images resulted in a much larger number of blunders.Table 3 and Figure 7 contain the results of this experiment for GeoEye-1, IKONOS and SPOT-5 supermode, respectively.After matching and 2D affine transformation, points visible in the images should have same coordinates.To check the results corresponding check points were manually selected and their coordinates were compared.The table contains the RMS values of these independent check points.
For GeoEye-1 and SPOT-5 the results for the urban and rural area are significantly better than those for the agricultural area, for IKONOS all three areas deliver similar accuracies.While the reason for this observation is not clear beyond doubt, the poor texture in the agricultural areas depicted in the GeoEye-1 and the SPOT-5 images is suspected to be responsible for this result.
When comparing Tables 2 and 3 it can be seen that in general the automatically achieved results are comparable to those obtained by human measurement with approximately 5 GCPs.
GeoEye-1 (0,5m GSD) 3. Root mean square coordinate differences at independent check points -tie points determined by matchingscene orientation by bias corrected RPC Figure 7. Root mean square coordinate differences at independent check points -tie points determined by matchingscene orientation by bias corrected RPC

Accuracy of mapping
In addition to the scene orientation problems, described above, the geometric accuracy with which common map features such as buildings, boundary walls, fences, roads, parcel boundaries, etc. can be extracted from geo-referenced imagery by on-screen digitizing has been evaluated.Such features were mapped from existing aerial orthophotos of 0.5m GSD and the geo-referenced satellite orthophotos (Figure 8).The coordinates of wellidentifiable points along these features, such as building or fence corners, etc. were compared.The RMS values for the difference in the X-and the Y-coordinate for the same map points are summarized in Table 4.
As before the reported results refer to differences between the points identified in both images.Assuming again equal accuracy for aerial and GeoEye-1 images, the accuracy of the GeoEye-1 images is a factor of √ better than the indicated values.With respect to the underlying GSD the achieved results are very satisfactory.

RMS [m]
GeoEye-1 IKONOS-2 SPOT-5 The clearly visible and large objects in all used reference areas did not cause problems with object identification, so in contrast to topographic mapping in other countries (Jacobsen 2012) the object identification was not the limiting factor for generation and updating the geospatial data base.

CONCLUSION
The map accuracy standards used by MOMRA (Table 1) are very high in relation to common international standards which are in the range of 0.2mm up to 0.3mm in the presentation map scale.These MOMRA standards have to be compared with the on-screen digitizing accuracy (Table 4) and the required orthophoto accuracy (only the standard deviations for X and Y are important for the analysis).
It could be shown in this study that the existing 0.5m resolution aerial orthophoto database can effectively be used as control for geo-referencing of recently acquired satellite imagery of 0.5m to 1m resolution to generate and update 1:10,000 scale line maps and orthophotos with 0.5m GSD.In order to generate these map products, coordinates of number of control points that are well distributed over the image, need to be obtained through manual measurements on the reference orthophoto and the new satellite image; bias corrected RPC orientation and height model data for the orthoprojection of the satellite image are employed in a pre-processing step.About 4 to 5 hours of a trained operator time is required to complete this process under the difficult conditions, especially in desert areas.
A more efficient alternative for the geo-referencing of the new satellite image is to orthoproject them based only on the vendorsupplied RPC and height model data in order to obtain an approximately geo-referenced satellite orthophoto.This orthophoto is then matched with the corresponding reference orthophoto using the SURF-based matching approach that generates a large number of corresponding points, usable as GCPs.SURF matching and the orientation processing only take a few minutes and results in a geo-referenced satellite image that also meets MOMRA map accuracy standards for the 1:10,000 map scale.A production rate of 5 to 7 satellite images in one day can be obtained without the need for field survey or manual measurement of image coordinates.It is estimated that if orthophotos of higher positional accuracy becomes available for control, the proposed procedure will be equally applicable for generating orthophotos of correspondingly higher accuracy.
There are several uses of map data where the completeness and the correctness of the map data is of primary interest while the geometric accuracy is of secondary importance.For example, due to the rapid pace of urbanization in Saudi Arabia, the planning for continuing urban development requires frequent updating of the land-use map data which is most commonly compiled from MOMRA's 1:2,500 scale map database.Such land-use planning can be greatly facilitated if MOMRA can rapidly deliver map data that has been updated using recently acquired very high resolution satellite imagery that captures the changes in the urban infrastructure.The fact that in this case, the newly mapped features do not meet MOMRA's geometric map accuracy standards for 1:2,500 scale is not of concern for land-use planning; they represent topologically correct current information.
Since 2012 digital aerial imagery of 0.1m GSD has been acquired for the entire urbanized area in Jeddah Region using modern digital aerial cameras.This has been supplemented with coverage at 0.2m GSD for the remaining rural and mountainous areas of the region.The 0,.2m GSD digital imagery is currently being processed for the production of orthophotos at scale 1:2,500, and orthophoto maps and line maps are being generated from the 0.1m GSD digital aerial imagery.Based on the experience in this research it can be confidently concluded that by matching the 0.5m GSD satellite imagery with 0.2m GSD aerial orthophotos using the proposed SURF-based matching procedure should provide updated map data that meets MOMRA map accuracy standard at least for the scale 1:5,000.This will enable MOMRA to rapidly respond to any requests for the delivery of updated map data at scales ranging from 1:5,000 to 1:10,000 that fully meet MOMRA map accuracy standards.It will, of course, be subject to the timely availability of recently acquired satellite imagery of appropriate resolution.The human eye can resolve approximately 8 pixels/mm (0.125mm/pixel) at the usual reading distance.Based on this fact 0.5m GSD orthophotos can be used up to a scale number of 0,5m/0.125mm= 4000.If the internationally used standard deviation of 0.2mm up to 0.3mm in the map scale or 1m up to 1.5m for the scale 1:5,000 is taken into account, also under the current conditions a mapping and orthophotos for 1:5,000 are possible.

Figure 2 .
Figure 2. Samples of different landscape types

Figure 3 .
Figure 3. Root mean square differences at independent check points based on orientation with manually measured GCPs for different test areas

Figure 8 .
Figure 8. Building feature mapped with aerial and satellite imagery

Figure 11 .
Figure 11.Updated map overlaid with aerial orthophoto of 2007 (left) and GeoEye-1 orthophoto from 2010 used for mapping (right) Figures 9 up to 11 give an impression of the amount of changes in the suburbs of Riyadh within one year, respectively 3 years.The clearly visible and large objects in all used reference areas did not cause problems with object identification, so in contrast to topographic mapping in other countries (Jacobsen 2012) the object identification was not the limiting factor for generation and updating the geospatial data base.

Table 4 .
Root mean square differences of coordinates of mapped features