“VERTICAL ACCURACY EVALUATION OF THE FOREST AND BUILDINGS REMOVED COPERNICUS DEM (FABDEM) OVER THE PHILIPPINES”

: This study evaluated the accuracy of the Forest And Buildings removed Copernicus DEM (FABDEM) over the Philippines. The evaluation was conducted at the country and island levels using 17,013 Geodetic Control Points (GCPs) partitioned into 3 independent sets. Results suggest that the FABDEM generally underestimates terrain elevation in the Philippines, as exemplified by a mean error of -1.44 m which is the average among the three sets of the GCPs used. Negative errors are primarily dominant in Luzon Island in the northern part of the Philippines, and positive errors are prevalent in the Visayas and Mindanao islands. At the country level, the DEM has an average Root Mean Square Error (RMSE) of 4.74 m and average linear errors of 7.80 m and 9.80 m at 90% and 95% confidence levels, respectively. At the island level, the DEM's accuracy varies, with some islands having RMSEs as low as 3.03 meters and as high as 5.80 m. This DEM is also most accurate at elevations less than 100 m and slopes less than 2 degrees. Care must be taken when applying the FABDEM in an archipelagic country like the Philippines due to several factors, such as the general tendency of this DEM to underestimate elevations, the dominance of negative errors in the northern part of the country, the differences in accuracies of elevations from one island to another, and the pronounced influence of elevation and slope to its accuracy. Nevertheless, the FABDEM is one of the most accurate among several freely available global DEMs covering the Philippines. Its accuracy in Mindanao Island is comparable to AW3D30 and superior to SRTM


INTRODUCTION
The FABDEM is a recently released global digital elevation model (DEM) that was generated by removing building and tree height biases from the Copernicus GLO 30 DEM (COPDEM30).Version 1 of the data was released to the public for noncommercial use in 2021, with one arc-second grid spacing (approximately 30m at the equator).The developers of the FABDEM reported that it has a mean absolute error of 1.61 in urban areas, 2.88 m in forests, and 2.55 in boreal forests.The Root Mean Square Errors (RMSEs) of this DEM range from 2.33 m (urban) to 6.66 m (boreal forest), which makes it more accurate than existing global DEMs (Hawker et al., 2022).
As the buildings and trees no longer exist in the FABDEM, it can be considered a Digital Terrain Model (DTM).With such characteristics and reported high accuracy, the FABDEM is attractive for applications requiring terrain elevation information.However, its quality and accuracy, including its suitability for these applications, must be evaluated.
This study aims to evaluate the vertical accuracy of the FABDEM over the Philippines using geodetic control points (GCPs).The specific objectives are: (i.) to determine the spatial distribution and characteristics of the errors, (ii.) to provide an estimate of its accuracy using such measures as the Root Mean Square Error (RMSE) and Linear Errors at 90% and 95% confidence levels, (iii.) to establish the consistency of FABDEM's accuracy at both the country and island levels.

2.1.1
FABDEM Data: Version 1 of the FABDEM tiles covering the Philippines, in TIFF format, were downloaded from the University of Bristol Research Data Repository at https://data.bris.ac.uk/data/dataset/25wfy0f9ukoge2gs7a5mqpq2 j7 (Hawker and Neal, 2021).All the tiles span 113 0 to 117 0 east longitude and 4 0 to 23 0 north longitude.
A total of 103 1 0 x1 0 tiles, each with a dimension of 3600 x 3600 pixels, were mosaicked in ArcGIS -ArcMap 10.8, preserving its native spatial resolution of 1 arc second and World Geodetic System 1984 (WGS84) coordinate reference system (Figure 1).It was ensured that the mosaicking procedure did not change the geographic location and elevation values.Random checking of the elevation values in the original tiles and the mosaicked version showed that this was the case.

GCPs:
A total of 17,088 GCPs were initially considered for the vertical accuracy evaluation.They belong to the Philippines' Geodetic Control Network, established using Global Navigation Satellite System (GNSS) equipment and techniques and are maintained by the National Mapping and Resource Information Authority (NAMRIA).The GCP data, in MS Excel format with the station name, was manually encoded using the available GCP information at the Philippine Geoportal (https://geoportal.gov.ph).These GCPs were established following the standards and specifications in the Revised Manual of Land Surveying Regulations in the Philippines (DENR, 1998) and the manual on land survey procedures (DENR, 2010).These GCPs are permanently marked on the ground strictly in accordance with surveying regulations and are protected to ensure their integrity through time, considering that they are mainly used as reference points for land surveys in the Philippines.Such characteristics make them advantageous in DEM accuracy evaluation.
The spreadsheet file of the GCP data with the WGS84 coordinates (latitude, longitude, and ellipsoidal height) was imported in ArcGIS -ArcMap 10.8, and a point GIS Shapefile was created.As the FABDEM's vertical datum is based on EGM2008, the ellipsoidal heights were converted to EGM2008 orthometric heights or elevation (H) by using the equation: H = h -N, where N is the geoidal undulation or geoid height.The N for each GCP was extracted through bilinear interpolation from the 1' EGM2008 model raster file downloaded from the Agisoft LLC website at https://www.agisoft.com/downloads/geoids/.Agisoft LLC converted this global geoid model from US National Geospatial-Intelligence Agency (NGA) data.
The FABDEM elevation at each GCP location was extracted through bilinear interpolation.The difference between the FABDEM elevations and the GCP elevations were calculated, and the values represent the errors (Ghilani, 2017).The preliminary error statistics, particularly the mean and the standard deviation, were determined.Using these statistics, the range for potential blunders and outliers were computed using the 3-sigma rule.This means that blunders and outliers are those GCPs where their error values are outside the range of mean ± 2.968*standard deviation (Ghilani, 2017).Using this range, only 17,014 of the original GCPs were retained.
The final GCPs were then partitioned into 3 independent sets of equal number (i.e., 5671 GCPs per set), making the final total of the utilized GCPs to 17,013.The extra GCP was excluded.The partitioning aims to have a robust estimation of the DEM's vertical accuracy and to provide a range of values of the accuracy measures.This was also done considering that a single set of validation points may not give a representative estimate of the DEM accuracy.Several sets make it possible to determine the consistency of the DEM's accuracy.The partitioning also ensures that the GCPs in each set are well distributed, considering that there are high concentrations of GCPs in several areas than in others.
For accuracy evaluation by island, each GCPs were assigned to the island where they belonged.Only the 10 largest islands of the Philippines were considered in the evaluation.
The summary of the GCPs used in the evaluation are shown in Table 1 and Table 2.

Accuracy Evaluation
The accuracy evaluation of FABDEM comprises comparing the actual elevations of the independent sets of Geodetic Control Points (GCPs) to their corresponding FABDEM elevation values.From the error values, the minimum, maximum, and the mean errors were calculated for each set.The Mean Absolute Error (MAE), Root Mean Square Error (RMSE), LE90, and LE95 were also determined.Table 3 presents the list of equations used in the evaluation.
The calculations and analysis were conducted at the country and local (site-specific, e.g., island) levels.Doing so would help us understand the FABDEM's overall accuracy and its accuracy across different sites.The latter is crucial to determine considering the archipelagic nature of the Philippines.The relationship between the FABDEM elevations of the GCPs with their actual elevations were also explored, including the relationships of the computed errors with elevation and slope.3. List of equations used in the accuracy evaluation.

FABDEM Error Characteristics
Figure 2 presents the spatial distribution of FABDEM errors computed at the Set 1 GCP locations.It is noticeable that negative errors are mostly dominant in the northern part of the Philippines, and positive errors are prevalent in the middle and southern parts.
The frequency histograms of FABDEM errors over the Philippines (Figure 3) further illustrate the dominance of negative errors, accounting for more than 60% of the GCPs used.The most frequent errors lie between approximately -7 m to +4 m.Beyond these values, the error frequency decreases.Such observation is confirmed by the plot of the errors versus the GCP elevations (Figure 4), which indicates that the FABDEM, when assessed at the country level, generally underestimates actual elevation.While the errors seem to decrease with elevation, this relationship depicted in the error plot is not strong enough to provide a clear relationship between the errors and elevation.The negative errors exist regardless of the elevation.One notable observation is the concentration of GCPs with errors from -10 to 0 meters at elevations up to 500 meters.When the errors are plotted versus slope (Figure 5), ± 10 m errors are mostly concentrated in FABDEM pixels with 0 to 10 0 slopes.

Country-level Accuracy of the FABDEM
At the country level, the minimum errors observed using the 3 sets of GCPs range from -40.61 m to -35.60 m, with an average of -37.63 m.The maximum errors range from 35.65 m to 37.81 m, with an average of 36.95 m.On the other hand, a negative mean error was obtained in all GCP sets, with an average of -1.44 m, indicating the FABDEM to underestimate elevation.The mean absolute error was found to have an average of 3.16 m across all GCP sets.
Regarding RMSE, the values obtained from the three sets appear to be nearly consistent, ranging from 4.65 to 4.85 m and an average of 4.74 m.The average vertical accuracy of the FABDEM in terms of LE90 and LE95 were 7.80 m and 9.30 m, respectively.

Island-level Accuracy the FABDEM
The accuracy of the FABDEM at the island level significantly differs from those obtained for the whole country (Table 5, Figure 6, Figure 7, and Figure 8).
The largest absolute minimum and maximum errors are found in the Luzon and Mindanao, which are the top 2 largest islands of the Philippines.While at the country level, the FABDEM tends to underestimate elevation, this is not entirely the case when evaluated at the island level.The mean error values range from -2.95 m to 1.13 m and are positive in 5 of the 11 islands considered in the analysis, indicating overestimation of elevation in these sites, particularly in Siargao (1.Table 5. Statistics of the FABDEM errors when evaluated in selected islands of the Philippines.The values are the average of the results using the three sets of GCPs (unit: meters).

FABDEM Accuracy Based on Elevation and Slope
Since the relationships of the FABDEM errors with elevation and slope are challenging to see using the error plots shown in Figure 4 and Figure 5, an error analysis using elevation and slope ranges was conducted.The summaries of FABDEM accuracy statistics computed using all (17,013) GCPs located at different elevation ranges and slope classes are presented in Table 6 and Table 7.
To allow a definitive evaluation of how the accuracy changes, the GCPs were grouped using elevation ranges with an interval of 100 m.More than 70% of the GCPs were located at FABDEM elevations less than 100 meters.At this range, the MAE and RMSE are at their lowest, with values of 2.83 m, and 4.12 m, respectively.Based on Figure 9a, the RMSE of the FABDEM generally increase with increasing elevation.An RMSE of less than 5 meters is found in elevations less than 400 m. the GCPs located at different slope classes (unit: meters).
When evaluated according to slope classes, the smallest values of MAE and RMSE of 2.65 m, and 3.87 m, respectively, were found for slopes less than 2 degrees.With increasing slope, the FABDEM RMSE also increases (Figure 9b), indicating a linear relationship between them.The RMSE in slopes ≥ 10 degrees is more than twice that in flat terrains (i.e., slope less than 2 degrees).The FABDEM's error is largest in slopes ≥ 15 degrees, with an RMSE of 9.28 m.
Figure 9. FABDEM RMSEs computed using all the GCPs located at different elevation ranges and slope classes.

Comparison of FABDEM Accuracy with Hawker et al (2022)
This vertical accuracy evaluation of the FABDEM is perhaps the first to be done at the country level, particularly for the Philippines.A major finding of this study is that the FABDEM generally underestimates elevation in the Philippines, as exemplified by a mean error of -1.44 m obtained from the three sets of GCPs used.While this value, including the MAE, differ from those obtained by the FABDEM developers (Table 8), the average RMSE obtained by the study (4.74 m) is within the published range of 2.33 m -6.66 m.
One of the possible reasons for the differences in the accuracy values would be the method and data used in the evaluation.In their validation, the developers mainly utilized referenced DEMs, specifically LiDAR data from 12 countries, as well as random samples of ICESat best estimates of the ground terrain from locations around the world (Hawker et al., 2022).On the other hand, the differences can also be explained by the accuracy of the random forest regression model in removing buildings and forests in the COPDEM30 covering the Philippines.It was not clear from Hawker et al. (2022)

On the Differences of Accuracy at the Island Level
This study demonstrated that the FABDEM's errors and accuracy differ at the island level.Mostly negative errors were found in GCPs located in Luzon Island, while positive errors persist in several islands in the Visayas and Mindanao.The values of the RMSEs are also not consistent across the islands, with some islands having RMSEs as low as 3.03 meters and as high as 5.80 m.
In an evaluation conducted by Santillan and Makinano-Santillan (2016) using 274 control points in north-eastern Mindanao, the RMSEs of AW3D30 (5.68 m), SRTM-30m (8.28 m), and ASTER GDEM2 (11.98 m) are higher than that of the FABDEM that was obtained by the study for the whole Philippines (4.74 m) and Mindanao (4.65 m).
Another study by Santillan and Makinano-Santillan (2017) evaluated the vertical accuracy of AW3D30, SRTM DEM V3, and ASTER GDEM2 covering the whole of Mindanao.The RMSEs obtained were 4.32 m, 5.16 m, and 9.80 m, respectively.This makes the FABDEM nearer in accuracy to the AW3D30, and more superior than SRTM DEM V3, at least for Mindanao Island.

On the Effects of Elevation and Slope on the FABDEM Accuracy
The quantitative analysis of RMSE according to elevation ranges and slope classes demonstrated the pronounced effects of elevation and slope on the FABDEM accuracy.This DEM is most accurate at elevations less than 100 m and slopes less than 2 degrees.Its accuracy degrades as the elevation increases and as the terrain surface becomes steeper.It is hypothesized that the result of obtaining better accuracy in flat and lowly elevated areas can be attributed to the fact that most of the GCPs are in relatively open spaces, with few obstructions from buildings and vegetation.Hence, the terrain information of these GCPs is better captured by the FABDEM.Another plausible explanation can be the better performance of the random forest regression model in removing the buildings and vegetation in these specific areas than in the others.

CONCLUSIONS AND FURTHER STUDIES
It can be concluded that the FABDEM is by far one of the most accurate among several freely available global DEMs covering the Philippines, with average RMSE, LE90, and LE95 of 4.74 m, 7.80 m, and 9.30 m, respectively, based on 17,013 GCPs.However, care must be taken when applying the FABDEM in an archipelagic country like the Philippines due to the following reasons, namely (i.) the general tendency of this DEM to underestimate elevations, (ii.) the dominance of negative errors in the northern part of the country, (iii.) the differences in accuracies of elevations from one island to another, and (iv.) the pronounced influence of elevation and slope to its accuracy.
As the FABDEM is essentially a re-processed or re-analyzed version of the COPDEM30, a separate vertical accuracy evaluation of the latter over the Philippines using the same GCP datasets can be one of the ways to explain the differences in the accuracy obtained by this study from that of the developers.At this time, the differences in the vertical accuracy of the COPDEM30 and FABDEM over the Philippines have not yet been established.Doing so may highlight the accuracy and limitations of the random forest regression model in removing buildings and forests in the COPDEM30 covering the Philippines.On the other hand, specific evaluation of the FABDEM in urban and forest areas would also be worthwhile to establish this DEM's accuracy level further.
Moreover, only GNSS-measured GCPs were utilized in the vertical accuracy evaluations.Noting that these GCPs are at strategically selected and thoroughly evaluated locations, they may not reflect the real accuracy of the FABDEM.This limitation can be overcome by utilizing reference DEMs of higher spatial accuracy, especially those generated by Light and Detection and Ranging (LiDAR) technology.
Recently, newer versions of global DEMs have been released, and some are continually updated.Comprehensive accuracy evaluations and comparisons of these DEMs with FABDEM are crucial to establishing which among them is the most accurate elevation source for various applications, such as in hydrological analysis and simulations, flood modelling and hazard mapping, geological hazard analysis, and landslide mapping, among others.

Figure 2 .
Figure 2. Spatial distribution of FABDEM errors at Set 1 GCP locations.Green colors indicate negative errors while yellow to red colors indicate positive errors.
13 m) followed by Leyte (0.93 m) and Mindanao (0.64 m).Underestimations are evident in Luzon (-2.95 m), Cebu (-1.04 m), and Mindoro (-0.94 m).On the other hand, the mean absolute error ranges from 1.78 m (Panay) to 3.86 m (Palawan).Mean absolute errors are greatest in Palawan, Cebu, and Luzon.The average FABDEM RMSE at the selected islands ranges from 2.73 m to 5.83.It is the most accurate in Leyte, having the lowest LE90 and LE95 values of 4.49 m and 5.35 m, respectively.It is least accurate in Palawan and Cebu, with RMSEs greater than 5 m and linear errors greater than 9 meters.

Figure 3 .
Figure 3. Histogram of FABDEM errors over the Philippines according to the set of GCPs.

Figure 6 .
Figure 6.FABDEM mean errors at selected islands of the Philippines.The values illustrated are the average of the mean errors calculated from the three sets of GCPs.

Figure 7 .
Figure 7. FABDEM MAEs at selected islands of the Philippines.The values illustrated are the average of the MAE calculated from the three sets of GCPs.

Figure 8 .
Figure 8. FABDEM RMSEs at selected islands of the Philippines.The values illustrated are the average of the MAE calculated from the three sets of GCPs.

Table 1 .
Number of GCPs used in the FABDEM accuracy evaluation.

Table 2 .
Basic statistics of the EGM2008 elevations of the GCPs (unit: meters).

Table 4 .
Country-level statistics of the elevation differences between FABDEM and the three sets of the GCPs (unit: meters).

Table 6 .
Beyond this, the RMSE increases, reaching a maximum of 13.49 m at elevations greater than 2000 meters.The FABDEM accuracy statistics computed using all the GCPs located at varying elevation ranges (unit: meters).

Table 7 .
The FABDEM accuracy statistics computed using all

Table 8 .
Hawker et al. (2022)o trained using data covering the Philippines.Comparison of the accuracy evaluation results with the accuracies obtained by the FABDEM developers as reported inHawker et al. (2022).