ORTHORECTIFICATION OF BRIDGES FROM HIGH RESOLUTION SATELLITE IMAGES

High-resolution satellite imagery requires high-resolution height reference data to create orthorectified images. High-precision digital surface data and LiDAR data are expensive to acquire quickly and accurately. Facility databases, on the other hand, are very efficient because once established, the information can be utilized persistently. The temporal resolution between the structure in the database and the captured image affects the ortho resolution created. Landmarks such as bridges are rarely subject to changes after construction. This makes facility database a suitable material to generate orthophotos of bridge structures. In this paper, we used a bridge database to perform orthorectification for structures with height variations. We removed relief displacement from satellite imagery at different angles and verified orthorectification feasibility for bridges of different heights and lengths. The orthoimage was overlaid with a topographic map to verify the orthorectification visually. As a result, we confirmed that the facility database was able to generate orthophotos of the bridges. The length and slope of bridge structures indicated the need for precise database updates in the future research.


Introduction
In urban areas, changes in small-sized urban objects continuously occur.To observe these changes over a wide area, highresolution satellite images are required (Piero et al, 2004).However, in satellite images, distortions of the image arise from height differences among urban objects and the ground surface.These distortions are severe when images are acquired at an offnadir angle.They must be orthorectified as they cause errors in the monitoring and change detection process (Qin et al, 2003).Additional elevation data for urban objects is necessary to remove relief displacement and to generate orthorectified images.Raster data such as digital surface model (DSM) and digital elevation model (DEM) or LiDAR datasets are primarily employed in orthorectification.High-resolution LiDAR data is predominantly utilized for the distinct identification of urban objects (Pradhan et al., 2021).The use of such data has spurred active research in urban monitoring.Acquiring such data at a high resolution presents economic and temporal challenges.Producing dense elevation data for small size objects entails significant costs.These costs increase as the coverage area expands and data is updated more periodically.Unlike raster data case, facility database regularly builds information on changes within urban areas at certain intervals.To generate more efficient elevation data and orthoimages, research on true orthorectification using a building database has been conducted (Amhar et al, 1998).The majority of object types comprising urban areas is buildings.Hence, true orthorectification of urban areas is mainly performed on these (Zhou et al, 2005., Chen et al, 2007).Buildings in downtown areas undergo frequent creation and demolition activities.In order for this information to be reflected, the building database used for orthorectification also requires periodic updating.Orthoimages using height information in the building database were significantly influenced by the database update frequency.Unlike regular buildings, it was useful to use database information for landmarks that did not change continuously.Similarly, landmarks such as bridges also have a low update frequency and little impact on accuracy related to database update issue.The use of the facility database is an appropriate way for bridge orthorectification in terms of accuracy and efficiency.The height of buildings mainly changes within their footprints by certain planar surfaces.In contrast, structures like bridges and overpasses may exhibit varying heights within long and narrow regions.Also, a bridge is a structure that floats in the air.This can cause other types of unnatural relief displacement and reduce the positional accuracy of the image.Therefore, it is necessary to conduct orthorectification studies on structures where such continuous height changes occur (Švec et al, 2014).Additionally, bridges are essential elements of urban architecture.To establish efficient transportation routes and urban planning within limited areas, the construction of bridges utilizing threedimensional space is imperative.In Seoul, South Korea, there are 28 river crossing bridges located in the middle of the city.Besides, there are also large and small overpasses and bridges connecting roads.Therefore, bridges are one of the important structures that require monitoring when generating orthoimages.In this study, we conducted precise orthorectification of bridges using a constructed bridge database.Instead of DSM, DEM and height values from the database were used.This enabled the generation of more accurate orthorectified images for bridges in urban areas within the coverage of the satellite imagery.To identify bridge objects at the individual level, we employed highresolution K3A satellite imagery with a resolution of 0.55 meters in our experiments.Additionally, we used three images to confirm the possibility of removing relief displacement that occurred at various imaging angles.The accuracy and performance of the generated orthorectified images were assessed by comparing them with digital topology map.

The construction of bridges
Bridges are typically constructed to span between two distinct points.Occasionally, there might be a variation in height or an obstacle between these points.Consequently, bridges exhibit a continuous horizontal change in height.It is divided into two parts, parallel and variation, depending on the presence of height alterations.In the parallel parts, height variation is minimal or non-existent, with the points constituting the bridge having similar height values.This area can be modelled to a uniform surface, such as a rooftop of buildings.Conversely, the variation parts experience rapid changes in height.Thus, additional distinct process is necessary to calculate the precise relief displacement position for this changing segment.Figure 1 shows the degree of change in height value from the orthogonal view depending on the part.

Types of bridges
Prior to orthorectification, we categorized bridge objects slated for calibration into three groups: overpasses, bridges, and curved roads, based on distance and slope metrics.Overpasses serve as short-distance connectors between points, exhibiting minimal height fluctuation due to their proximity.However, primarily composed of parallel components, overpasses manifest relief displacement, seemingly levitating even at lower altitudes.Bridges link points across extensive spans, such as rivers, with their structure facilitating gradual height transitions or distinct separation between parallel and variant segments.Curved roads are mainly built to connect other roads.For this reason, they are predominantly constructed in constrained environments, adept at swiftly connecting areas with varying elevations.This type of bridge and road often features sharp curves, even with minimal height deviations.As shown in plot on Figure 2, curves that change rapidly have more inflection points.The more midpoints there are in a graph, the more smoothly the curves appear to be connected.Bridges also needs a connection point on the location with rapid height and position change.The height value of each changing point is required for precise orthorectification.Bridges with gentle slopes can be approximated as polygons with uniform height, whereas those with steep inclines experience rapid height variations.Enhanced accuracy in orthorectification is achieved by acquiring height values for multiple points.Consequently, we partitioned individual bridges into multiple segments, delineated by inflection points occurring during frequent changes in slope.

Figure 2. Type of bridges (above) and inflection point changes according to slope (bottom).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-1-2024 ISPRS TC I Mid-term Symposium "Intelligent Sensing and Remote Sensing Application", 13-17 May 2024, Changsha, China

Methodology
The methodology for this experiment is as follows.First, we get the height values for the bridges and terrain.Then we perform inverse mapping to the three-dimensional ground coordinates with those heights and calculate the image coordinates.We create an ortho image using the pixel value of the original image coordinates.

Inverse Mapping
To perform precise orthorectification we need to know the 3D ground coordinates of the ground surface and bridges.For this, three-dimensional positional information of bridges is obtained from the bridge location information database.3D coordinates of the ground surface and bridges are transformed into image coordinates of input satellite images through rational functional model (RFM).The RFM is a generalized model that defines the relationship between images and ground coordinates using rational polynomial coefficients (RPCs).Equation ( 1) represents the relationship between ground coordinates and image coordinates.Equation ( 2) is a polynomial used in RFM, and each equation consists of 20 RPC coefficients.
where xi, yi = image coordinates of origin image a0~ a19 = RPC Coefficients 0, Y0, Z0 = coordinates of projection centre X, Y, Z = ground coordinates The coefficient of RFM reflects the attitude angle of the satellite and its relationship with the ground point.Since this is similar to a projected coordinate system, ellipsoidal height is required for calculation.Different image coordinate points are estimated depending on the Z value of the input ground coordinate point.Through this, the image coordinates of the location where the relief displacement occurred can be calculated.

Interpolation
In this paper, we utilize interpolation in two types.The first interpolation is employed to estimate heights of the bridges.An object with a constant height requires only one height value.With the value, all locations where relief displacements occur can be identified.In contrast, bridges have a height value that changes continuously.We need the height values for each change within the area corresponding to the bridges.However, Information of bridges in the database is built as a polygon.Each polygon has a single height attribute value.We do not know the individual height values of every pixel where a bridge located.Therefore, the first interpolation is applied using the height values of the vertexes to calculate the height values of the pixels.
For the second case, we used inverse distance weighting interpolation with the corner of the bridges as shown in Equation (3).Since the number of known points is four or more or their shapes are diverse, interpolation is performed using inverse distance weights.N is the total number of points that makes up one bridge.We estimate the height of the unknown point using the distance between a point with known height and the unknown point inside the bridge.
where   = Height value at point k  = number of known points ⅆ(,   ) 2 = distance between  ⅆ The second interpolation is conducted on pixel brightness values using the pixel value from the images when creating orthoimages.
It is used to estimate the precise pixel value from the mapping image coordinates.The pixels in the original image are always square unlike a polygon of bridges.To accurately correct the ortho values from the calculated image coordinates, we use bicubic interpolation

Orthoimages Generation
Figure 3 shows the sequence of generating a true orthoimage after removing relief displacement.After estimating the image coordinates where the relief displacement occurred, the orthorectified coordinates of the corresponding point are calculated.In the case of orthorectified coordinates, they correspond to the orthogonal position of a specific object in the image.It can be calculated using the two-dimensional plane coordinates of the ground point used in the previous step.We input the pixel value of the object area where relief displacement occurred through inverse mapping to orthorectified coordinates.Blank processing is performed to remove the existing relief displacement area.To create natural orthoimages, blank areas are filled with the brightness values of nearby pixels.In this paper, we generated double-mapped images without blank processing to check the extent of relief displacement in the images.

Used Data
For our experiments, we used KOMPSAT-3A imagery which was pansharpened to 0.5m.Three images were selected covering similar area of Seoul, Republic of Korea.All images with different off-nadir angle were orthorectified.Table 1 shows the imaging angle for each image.The bridge database used was the 'Seoul bridge location information database' and the 'National bridge standard database'.The 3D coordinate information of the bridge was combined to the bridge height value and the bridge location information.Bridge location information was constructed mainly for simple, straight bridges.We created polygons for roads that existed in the standard database.The height value of the created polygon was estimated through iterative mapping through RFM based on ground elevation values.Vertexes values were extracted from the polygon and used.
Figure 4 shows the images and the polygons of the bridge database used.Bridges, overpasses, and curved roads that were commonly included in the images were selected.As shown in Figure 4, the green polygon is a bridge, the blue polygon is an overpass, and the red polygon is a curved road.The number next to the polygon indicates the maximum height of the bridge.The points in the lower image were extracted from the vertices of the polygons in the upper image.

Distance error
The magnitude of the relief displacement varies with the distance, angle, and height of the subject from the shooting location.We compared the influence of shooting angles and object heights on relief displacement.This was done to see how the angle of acquisition affects the displacement in the image, and to generate images with the displacement removed from various images.When one ground coordinate changed in height, the corresponding image coordinates were calculated.the height values range from 0m to 30m in 5m intervals.Based on the image coordinates at 0m, the image distance was calculated as the height value of the ground coordinates changed.
The relief displacement increased radially from the image center coordinates.In images A and B, the relief displacement occurred in the same upward-right direction.However, in case of image C, it occurred in the upper-left direction.Although the degree was different for all three images, as the height increased, the distance error in the col and row directions increased.The increase in relief displacement as the constant height for each image increased was confirmed to be uniform for each image.
As the off-nadir angle increased, we observed that the parallax displacement error also increased.However, the increment according to the angle was insignificant compared to the increment in height of the object.In the case of a bridge with a height of up to 30m, distance errors of approximately 25, 27, and 28 pixels may occurred for each image.

Validation
Figure 5 shows the double-mapped images after bridges orthorectification for the experimental area.To check whether the orthoimages were created properly, visual analysis was conducted using before and after orthoimages and aerial orthoimage.Figure 6 shows crop images comparing the before and after images for each bridge type.The accuracy of the location of the discontinuity was verified by overlaying a topographic map and orthoimage.As shown in both the Figure 5 and 6, the red line represents topographic map contour.The yellow line indicates the error of bridges region where the relief displacement occurred.
In the image, the larger the off-nadir angle, the larger the relief displacement.However, orthorectification results was confirmed that relief displacements of the bridges were eliminated in all three images.The piers of the bridge are obscured by orthorectified bridges in double-mapped images.
We checked the naturalness of the positional edited relief displacement and compared the results between bridges types.The bridge was long, straight and had little change in height, so the orientation was properly established.For curved roads with accurate inflection point height values, the relief and displacement could be found in the same shape.In contrast, the overpass had a rapid increase in height over a short distance.
Curved roads had continuous changes in height in one direction, but in the case of overpasses, rise and fall occurred simultaneously within one bridge.Therefore, there was a rapid change in height compared to the distance, and orthorectification was not properly performed using only the height value between two points.The larger the height deviation and the shorter distance of the bridge, the more section points were required.However, it was found that for shorter intervals, the roads can appear disjointed due to differences in height values within the road.
The bridge database did not include information on connections and nearby roads.It could be seen that the connection between the bridge and the road was unnatural intermittently.It was believed that it would be possible to generate orthoimages while maintaining connectivity if orthorectification was performed using an existing road database.

Conclusions
We performed orthorectification on several types of bridges objects and checked the results.It was confirmed that orthorectification for the structure was easily performed using the existing construction database.Therefore, orthoimages could be efficiently generated without DSM of individual structures.The possibility of various bridge orthorectification was confirmed by using images taken at diverse angles as input data additionally.In this process, it was confirmed that the deviation of relief displacement within the image was more affected by the height of the structure than the off-nadir angle.In addition, since the database once built was continuously used for multiple images, it indicated the need to build a precise leg database.
In the case of roads with steep slopes, the rapidly changing height was not reflected by interpolation, and separation was observed as a result of orthorectification.In this case, if orthorectification was performed by narrowing the vertex spacing of the bridge object, smoother results were obtained.However, overly dense vertex construction can reduce database efficiency.Therefore, it is necessary to establish a database construction interval according to relief displacement and height information for future research.The bridges used in this experiment did not overlap each other in orthogonal positions.When bridges built at different heights in the database intersect, incorrect relief displacements may be estimated for the overlapping areas.This part requires another analysis based on the occlusion area between structures, as well as the relief and displacement removed during the process of generating the realistic photorealistic image.

Figure 1 .
Figure 1.Whether to interpolate based on parts.

Figure 5 .
Figure 5.The result of generating double-mapped images for each image for the experimental area.

Table 1 .
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-1-2024 ISPRS TC I Mid-term Symposium "Intelligent Sensing and Remote Sensing Application", 13-17 May 2024, Changsha, China Shooting angle for each image used Figure 4. Polygons of Bridges in the test area.

Table 2 .
Image distance error according to changing shooting angle and height