Relative truth value acquisition method of heterogeneous surface based on the Improved Point Spread Function

The acquisition of pixel-scale relative truth value is important for remote sensing validation.The Point Spread Function (PSF) has been widely used in the field of the acquisition of relative truth value of heterogeneous surface. In this study, we propose an Improved Point Spread Function (IPSF) based on the Median Pixel Variability Weighted (MPVW) method and PSF to acquire relative truth value of heterogeneous surface. Firstly, the size of variance and clustering window are confirmed based on the pixel scale information of the satellite product to be validated. Secondly, the PSF suitable for heterogeneous surface is selected from 5 PSFs. Thirdly, the IPSF is constructed according to the MPVW and the PSF suitable for heterogeneous surface. Finally, the IPSF is used to acquire relative true value at pixel-scale of heterogeneous surface from airborne hyperspectral image. This study shows: (1) Good correlation between relative true value obtained using IPSF and reference value is indicated as R 2 values among 256 channels reach 0.985, structural similarity (SSIM) ranges from 0.992 to 0.998, and peak signal noise ratio (PSNR) more than 35dB. (2) Better accuracy performance is observed in the acquisition of relative truth value of heterogeneous surface for IPSF than PSF. Compared with conventional PSF, the R 2 , PSNR and SSIM of the result of IPSF are increased by about 1.34%, 9% and 0.7% on the average. (3) Significant advantage of IPSF is also reported in reducing the deviation compared to PSF, as the average root mean square error (RMSE ave ) is reduced by 30.37% and the average mean absolute error (MAE ave ) is reduced by 35.98%, respectively. (4) Comparing the RMSE and MAE of PSF and IPSF result, the RMSE and MAE of the result of IPSF become smaller. The RMSE decreases from 3 ~ 195 to 2 ~ 131. The MAE changes from 2 ~ 138 to 1 ~ 89. Overall, the IPSF proposed in this paper can effectively calculate the relative true value of heterogeneous surface, which can provide reference for the validation of multi-spectral and hyper-spectral satellite products.


Introduction
The acquisition of pixel-scale relative truth value for the satellite remote sensing product to be validated over heterogeneous surface is crucial for validating this product (Wen et al., 2023), significantly influencing the outcomes of such validations.Based on the current research findings related to relative truth value acquisition, from the perspective of reference data sources, methods for acquiring relative truth value can be categorized into two types: those based on groundmeasured data (Wu et al., 2020) and those based on highresolution data (Liu et al., 2016).However, due to the significant scale difference between ground-measured point data and satellite product areal data, as well as the presence of surface heterogeneity, the method of acquiring relative truth value based on high-resolution data is more suitable for heterogeneous surface.
Currently, the Point Spread Function (PSF) (Bai et al., 2019) is the commonly used method for acquiring relative truth value on heterogeneous surface from high-resolution data.Susaki employed PSF to transform a 30m ETM+ albedo product into a 250m resolution version (Susaki et al., 2007).Rutan used PSF to convert MODIS albedo data into a 30km resolution product, subsequently validating the CERES albedo product (Rutan et al., 2009).Wu regarded the PSF outcome as a 'real' value for lowresolution albedo product (Wu et al., 2016) .Mira devised a method combining Gaussian and triangular PSFs to create a Gaussian-like PSF, enabling the computation of a 1km resolution albedo product from an 8m FORMOSAT dataset (Mira et al., 2013).Peng introduced an elliptical-Gaussian PSF and, utilizing a 30m HJ albedo product, calculated a 1km resolution albedo product (Peng et al., 2015).
The critical factor in acquiring accurate relative truth value lies in determining the spatial variability within each pixel of remote sensing imagery (Ge et al, 2019).However, articulating spatial variability entails a multifaceted process (Li at el, 2016).The PSF, and its enhanced variants, such as the Gaussian-like PSF and the elliptical-Gaussian PSF, only provide a rudimentary approximation of spatial variability, focusing on the spatial response characteristics at the satellite pixel scale.This approximation, however, can be significantly refined.Current research on the acquisition of relative truth value across heterogeneous surface via high-resolution data reveals that a strong spatial autocorrelation is exhibited because the information of each pixel in a remote sensing image is affected by the surrounding pixels.In contrast to the PSF, the Median Pixel Variability Weighted (MPVW) method offers a quantitative representation of surface heterogeneity, focusing on the prevalent spatial autocorrelation among spatial entities (Bai et al., 2021).Therefore, this paper aims to develop an Improved Point Spread Function (IPSF) that holistically considers surface heterogeneity by integrating the MPVW method with the PSF.The goal for the IPSF is to describe surface heterogeneity through two aspects: spatial response characteristics and local spatial autocorrelation.And it can increase the PSF's capacity to depict surface heterogeneity accurately and reduce the uncertainty of relative truth value.

The Study Area
This study area is located in Matiwan Village, Xiongzhou Town, Xiong County, Baoding City, Hebei Province (Figure 1), situated on a gently sloping plain characterized by expansive, open terrain.The elevation of this area ranges from 7 to 19 meters above sea level.The central geographic coordinates are E116.059691°,N38.943524°.The designated core experimental area is more than 2000m × 1000m.Within this zone, the diversity of surface cover is notable, with more than 20 distinct types of land features, including but not limited to, residential structures, peach and poplar trees, meadows, robinia pseudoacacia, vegetable plots, and maize fields.Such diversity highlights the area's significant surface heterogeneity, rendering it particularly suitable for research on acquiring relative truth value of heterogeneous surface.The study area is shown in Figure 1.

High-resolution airborne hyperspectral data
This high-resolution airborne hyperspectral remote sensing image of Matiwan Village, used in this study, was acquired using the visible and near-infrared imaging spectrometer designed by Shanghai Institute of Technical Physics, Chinese Academy of Sciences on October 2017 (Cen et al., 2020).The spectral range of image is 391-1002nm, with 256 bands and a spatial resolution of 0.5m.The image size is 3750 × 1580 pixels(Figure 2).The land cover types labeled here are 20, exhibiting strong surface heterogeneity.After preprocessing such as radiation correction, geometric correction, and image mosaic and clipping, the high-resolution airborne hyperspectral surface reflectance product with high geometric and radiation accuracy was obtained.And it is suitable for acquiring the relative truth value of surface reflectance at the satellite pixel scale over heterogeneous surface.

Reference data
The area weighted method is one of the most widely used methods for acquiring relative truth value at the pixel scale of heterogeneous surface (Markham at el, 2023).The reference value (Figure 3) , employed to validate the accuracy of relative truth value over heterogeneous surface in this study, is calculated using the area weighted method combined with this high-resolution airborne hyperspectral data.The spectral range of reference value is 391-1002nm, with 256 bands and a spatial resolution of 8m.The airborne land cover map used in the calculation process was classified using random forest classification according to the distribution of land types in the study area.The total classification accuracy is 97%, which significantly ensures the authenticity of the reference value.More details about the experimental data can be found in the word of Cen et al (Cen et al, 2020).

Relative truth value acquisition method of heterogeneous surface
Surface heterogeneity impacts the accuracy of relative truth value.Therefore, it is necessary to comprehensively consider the heterogeneity of the land surface and to accurately map the heterogeneous characteristics of each pixel of remote sensing imagery for acquiring accurate relative truth value.This study introduces an IPSF, which describes surface heterogeneity through two aspects: spatial response characteristics and local spatial autocorrelation.Using the IPSF, we acquire an 8m resolution satellite surface reflectance product (8m resolution relative truth value) from the 0.5m resolution airborne hyperspectral image.The overall technical process is as follows: Firstly, the size of clustering window of airborne hyperspectral surface reflectance data and the variance of the Gaussian PSF are confirmed based on the pixel scale information of the satellite surface reflectance product.Then, an analysis of the applicability of PSF is conducted to select an optimal PSF for heterogeneous surface.Based on the MPVW and the PSF suitable for heterogeneous, we construct an IPSF method to acquire relative truth value at pixel-scale of heterogeneous surface.And the R 2 , PSNR, SSIM, RMSE, and MAE between the relative truth value and the reference value are compared and analyzed to validate and evaluate the performance and superiority of the IPSF method.The overall technical process of acquiring relative truth value of heterogeneous surface is shown in Figure 5.
Figure 5.The overall technical process of acquiring relative truth value at the pixel scale of heterogeneous surface.

Point spread function
The Point Spread Function (PSF) represents a weighted function characterizing the variance in optical sensors' responses to detected target signals, thereby elucidating the imaging process of sensors with respect to detection targets.It examines the radiation contributions from high-resolution pixels encapsulated within low-resolution pixels, facilitating a straightforward quantitative depiction of surface heterogeneity at the low-resolution pixel level.This function mitigates, to some extent, the issue of spatial response non-uniformity across heterogeneous surface.And PSF is routinely employed to delineate the spatial response features inherent to the pixels of satellite products and to derive the relative truth value of heterogeneous surface at a low-resolution scale from highresolution data.The commonly used PSF include 5 types: Rectangular PSF, Circular PSF, Gaussian PSF, Cosine PSF, and Triangular PSF.The specific equations for these PSFs are provided in Equations 1-5, and their respective shapes are illustrated in Figure 6.
The pixel value of the satellite product can be understood as the convolution of the radiation values of ground objects in the pixel with the PSF.Thus, acquiring relative truth value based on the PSF is to realize the spatial response weighting of ground objects at different positions in the pixel (clustering window) through PSF, then acquiring the pixel-scale relative truth value(Equation 7).

Point spread function applicability analysis
The spatial responses characterized by the 5 types of PSFs are different, which results in differences in their applicability of heterogeneous surface.To analyze the applicability of these 5 PSFs to heterogeneous surface and to select the most suitable PSF for heterogeneous surface, an analysis of the applicability of PSF is conducted.Firstly, each of the 5 PSFs is used to acquire pixel-scale relative truth values of satellite surface reflectance product of heterogeneous surface.Then, the mean and standard deviation between the acquired relative truth values and the reference values are calculated to validate the accuracy of the relative truth values, thereby evaluating the applicability of each PSF for heterogeneous surface.The PSF that achieved the highest accuracy in relative truth value is selected as the most suitable for heterogeneous surface.
Due to the fact that the spatial resolution of the satellite surface reflectance product to be validated in this study is 8m, and the spatial resolution of the airborne hyperspectral image is 0. From Figure 8, it can be seen that the standard deviations of the relative truth values acquired by the 5 PSFs follow a trend similar to that of the original airborne hyperspectral data, but are generally lower overall.Among them, the standard deviation associated with the Gaussian PSF is closer to that of the original airborne hyperspectral data, indicating that the Gaussian PSF retains more spectral information of ground objects than the other PSFs, thereby making it more suitable for heterogeneous surfaces.Therefore, the article selects the Gaussian PSF as the PSF suitable for heterogeneous surface.
where  (xi, yi) = the value of a certain reflectivity value in the local 17×17 window of the airborne hyperspectral image wi = the weight n = the pixel number of the local 17×17 window A = relative truth value of the local 17×17 window To enhance the PSF's capability to represent spatial heterogeneity, the study employs the MPVW method to improve the PSF suitable for heterogeneous surface, and constructs an Improved Point Spread Function (IPSF).This enables the IPSF can simultaneously deal with spatial heterogeneity from two aspects: the spatial response characteristics of sensors to ground objects and the local spatial autocorrelation of ground objects.The IPSF can be calculated as follows: where fMPVW = the MPVW method fPSF-Gaussian = the Gaussian PSF a, b = the coefficients, with values 0.5 and 0.5

Evaluation indexes
To further quantitatively compare and evaluate the accuracy of the relative truth value of heterogeneous surface acquired using IPSF, 5 common evaluation indexes, including R 2 , peak signal noise ratio (PSNR), structural similarity (SSIM), root mean square error (RMSE) and mean absolute error (MAE), are used for a comprehensive assessment.The R 2 reflects the correlation between the relative truth value of heterogeneous surface and reference value.The closer R 2 is to 1, the higher the correlation between the relative truth value and reference value of heterogeneous surface, indicating higher accuracy of the relative truth value.
The PSNR quantitatively evaluates the overall deviation between the relative truth value and reference value of heterogeneous surface based on their correlation.Generally, the higher the PSNR, the greater accuracy of the relative truth value and the lesser the distortion.The PSNR calculation formula is as follows: The image structure contains the main information of the image, and SSIM can quantitatively interpret the structural information.
From the perspective of image composition, the SSIM function has three parts: luminance, contrast and structure (Bai et al, 2019).The SSIM values are less than or equal to 1.And the closer the SSIM is to 1, the greater the similarity between the acquired relative true value of heterogeneous surface and the reference value.It is defined as follows: = parameters that are used to adjust the relative importance of the three components, with values 1,1 and 1 = constants that are used to avoid instability, with values 0.0001,0.0001and 0.00005 RMSE and MAE are often used to measure the deviation between the relative true value of heterogeneous surface and its reference value.The smaller the RMSE and MAE values, the lesser the deviation between the relative true value and reference value of heterogeneous surface.The RMSE calculation formula is shown in Equation 19.The calculation formula for MAE is presented as Equation 20.

Results and Discussion
The R 2 between the relative truth value of heterogeneous surface acquired before and after PSF improvement and the reference value is shown in Figure 9(a).It is observed that the trends in R 2 for the relative truth value of heterogeneous surface acquired by PSF and IPSF and the reference value are essentially consistent.However, the R 2 for the relative truth value acquired using PSF is above 0.969, while that based on IPSF reaches above 0.985.This finding illustrates that both before and after PSF improvement, the relative truth values of heterogeneous surface have high R 2 with reference values, but the R 2 for the relative truth value acquired by IPSF is higher than that of PSF, closer to 1, indicating a higher correlation of the relative truth value acquired by IPSF with the corresponding reference value.Figure 9(b) shows the improvement in R 2 (↑R 2 ) after PSF improvement.It is found that the R 2 improvement for the first 134 channels is slightly lower, around 0.8%-1.2%;while for the latter 117 channels, the R 2 improvement is more significant, above 1.6%.Therefore, IPSF increases the R 2 by about 1.34% on the average.For PSNR, Figure 10(a) shows us that the PSNR of the relative truth value of heterogeneous surface acquired by PSF and IPSF compared to the reference value first increases, then decreases, and finally stabilizes.At the same time, the PSNR of the relative truth value after improvement is higher than that before improvement.Before improvement, PSNR ranges between 31-41 dB, with an average of 35 dB; after improvement, it spans 35-45 dB, with an average of 38dB.Before the improvement, the number of channels with PSNR above 40dB is only 36; after the improvement, the PSNR of the first 56 channels all reaches over 40dB.Before the improvement, the number of channels with PSNR higher than 35dB is 126, whereas after the improvement, the PSNR of all 256 channels is higher than 35dB.Overall, IPSF has significantly increased the PSNR between the relative truth value and the reference value.To analyze the improvement in PSNR across each channel, a PSNR improvement figure was created, as shown in Figure 10(b).The improvement in PSNR varies greatly across channels.The PSNR can be improved by more than 10% in the first 7 channels and the last 124 channels.In channels 35-49 and 90-123, the improvement effect is low, under 6%; the improvement effect in the remaining channels is between 6% and 10%.At the same time, the 256th channel has the highest improvement effect, which can reach 13.86%; the 110th channel has the lowest improvement effect, which is 4.5%.In sum, PSF has led to an approximate 9% overall increase in PSNR.
(a)PSNR (b)↑PSNR Figure 10.The PSNR and the PSNR improvement For the SSIM index, as observed in Figure 11(a), the relative truth value of heterogeneous surface acquired both before and after the PSF improvement exhibit high SSIM values with the reference values, with all channels having an SSIM of over 0.984.However, compared to PSF, the SSIM of the relative truth value acquired by IPSF and the reference value is generally higher.After improvement, SSIM increases from a range of 0.984-0.993to 0.992-0.998,with the average SSIM across channels rising from 0.987 to 0.994.This indicates that the correlation between the relative truth value of heterogeneous surface acquired after PSF improvement and reference value is significantly better than before the improvement.To further analyze the improvement in SSIM across various channels, a map illustrating the SSIM improvement effect is drawn, as shown in Figure 11(b).From this figure, it is evident that channel 126 shows the least improvement in SSIM, with an increase of about 0.44%; channel 247 shows the most significant improvement, with an increase of about 0.99%.Meanwhile, channel 1 and channels 74-130 shows lower improvements, under 0.5%; channels 2-73 and 131-139 shows improvements between 0.5%-0.8%;and the latter 117 channels exhibited better improvements, above 0.8%.Therefore, the relative truth value of heterogeneous surface acquired by IPSF are closer to the reference value and of higher quality than those acquired before the improvement, overall resulting in an approximate 0.7% increase in SSIM.For RMSE, Figure 12(a) presents that the RMSE between the relative truth value of heterogeneous surface, before and after PSF improvement, and the reference value, remains consistent.However, the RMSE for the relative truth value of heterogeneous surface acquired using IPSF is lower than that acquired using PSF, aligning with the higher correlation between the IPSF-acquired relative truth value and the reference value.A chart illustrating the reduction in RMSE following improvement is shown in Figure 12(b).From this chart, it is evident that the reduction in RMSE across various channels generally follows a trend of initial decrease followed by an increase.The greatest reduction occurs in channel 1, with a decrease of about 45.59%; while the smallest is seen in channel 113, with a decrease of about 20.54%.Among the 256 channels, the reduction effect exceeds 40% in 11 channels; ranges between 30% and 40% in 133 channels ; and is less than 30% in the remaining 112 channels.On average, IPSF reduces the RMSE by approximately 30.37%.It can be found that before PSF improvement, the RMSE ranges from 3 to 195; after improvement, it ranges from 2 to 131.This indicates that IPSF effectively reduces RMSE, significantly lowering the deviation between the relative truth value and reference value.In summary, compared to PSF, IPSF more effectively reduces the deviation between the relative truth value of heterogeneous surface and reference value, decreasing the RMSE to 69.63% of its original value.The result for MAE is shown in Figure 13(a), indicating that the MAE for the relative truth value of heterogeneous surface acquired using IPSF is significantly lower than that acquired using PSF.Further analysis of MAE across different channels, reveals that before PSF improvement, MAE ranges from 2 to 138; after improvement, it narrows to 1 to 89, demonstrating IPSF's effectiveness in reducing deviation from the reference value.Figure 13(b) illustrates the MAE reduction trend, consistent with the RMSE reduction, initially decreasing before increasing.The most significant reduction occurs in channel 1, with a decrease of about 45.2%, while the smallest reduction is observed in channel 113, decreasing by approximately 30.49%.Among the 256 channels, 57 channels show a reduction greater than 40%, 139 channels have a reduction between 35% and 40%, and 60 channels show a reduction less than 35%.Overall, compared to PSF, IPSF can reduce the MAE by about 35.98%.In conclusion, compared to PSF, IPSF can more accurately describe the spatial heterogeneity of the land surface, achieving higher R 2 , SSIM, PSNR, and lower RMSE, MAE.

Conclusions
The study addresses the deficiency of the PSF, which characterizes spatial heterogeneity only through the sensor's spatial response to ground objects.By integrating the MPVW method with the PSF, an IPSF is developed, thereby increasing the PSF's capacity to accurately depict surface heterogeneity.Then the relative truth value of heterogeneous surface is acquired using IPSF.And the R 2 , PSNR, SSIM, RMSE, and MAE are calculated to evaluate the accuracy of the relative truth value.Conclusions are as follows: (1) The relative truth value of heterogeneous surface acquired using IPSF demonstrates high consistency with the reference value.The R 2 value for all 256 channels exceeds 0.985; PSNR is above 35dB, ranging between 35-45dB, with 56 channels exceeding 40dB.In addition, SSIM remains between 0.992 and 0.998, very close to 1.
(2) Compared to PSF, IPSF performs better in acquiring relative truth value of heterogeneous surface.Indexes such as R 2 , PSNR, and SSIM have all shown significant improvements.R 2 has increased by about 1.34% overall, with the improvement effect in the latter 117 channels exceeding 1.6%.PSNR has increased by about 9% overall, with the improvement effect in the first 7 channels and the last 124 channels reaching more than 10%.And SSIM has increased by about 0.7% overall.
(3) IPSF also shows a significant advantage in reducing deviation.Compared to PSF, the relative truth value of heterogeneous surface acquired based on IPSF show a reduction in RMSE and MAE by 30.37% and 35.98% respectively, effectively reducing the deviation between the relative truth value of heterogeneous surface and reference value.Among 256 channels, the best reduction effect is observed in channel 1, with RMSE and MAE reducing by about 45.59% and 45.2% respectively; the least reduction effect is seen in channel 113, with RMSE reducing by about 20.54% and MAE by about 30.49%.
(4) The range of RMSE and MAE values for each channel has noticeably narrowed after improvement.Specifically, RMSE has decreased from 3-195 before improvement to 2-131 afterwards; MAE has changed from 2-138 before improvement to 1-89 thereafter.
Overall, IPSF demonstrates a certain improvement effect, enhancing PSF's capability to depict surface heterogeneity.The relative truth value of heterogeneous surface acquired using IPSF exhibit a higher linear correlation, greater similarity, elevated PSNR, and reduced deviation.Thus, the IPSF method discussed in the paper has a stronger capacity for relative truth value acquisition of heterogeneous surface compared to the PSF method, which will provide technical support for the validation of satellite remote sensing products over heterogeneous surface.

Figure 1 .
Figure 1.Map of the study area in the Matiwan Village.

Figure 2 .
Figure 2. True-color display of the high-resolution airborne hyperspectral image in study area.

Figure 3 .
Figure 3. True-color display of 8m resolution reference value in study area.

Figure 4 .
Figure 4.The airborne land cover map in study area.
PSF, Circular PSF, Gaussian PSF, Cosine PSF, and Triangular PSF x, y = pixel coordinates in the clustering window x0, y0 = center pixel coordinates in the clustering window  (x, y) = pixel value at (x, y) coordinates M = half side length of the clustering window  = variance of Gaussian PSF A = relative truth value of the clustering window (a)Rectangular PSF (b)Circular PSF (c)Gaussian PSF (d)Cosine PSF (e)Triangular PSF Figure 6.Five point spread functions (The axis units are in pixels of the airborne hyperspectral image).
5m, the clustering window size is set to 17 × 17 airborne hyperspectral pixels.At the same time, according to equation 6, the variance of the Gaussian PSF is set at 3.4.Upon determining the clustering window size and variance, the 5 types of PSFs are applied within equation 7 to calculate the relative truth values.Then, the means and standard deviations between these acquired relative truth values and the reference values are calculated.The mean values of the relative truth values acquired by the 5 PSFs and the mean value of the original airborne hyperspectral data are shown in Figure 7. Analysis of Figure 7 reveals that the variation trends in mean values across the 5 sets of relative truth values are essentially consistent and very close to the original airborne hyperspectral data.This indicates that the overall radiation characteristics of the relative truth values acquired by the 5 PSFs are quite uniform.Figure 8 displays the standard deviations of the relative truth values acquired by the 5 PSFs and the standard deviation of the original airborne hyperspectral data.

Figure 7 .
Figure 7.The mean values of the relative truth values acquired by the 5 PSFs and the mean value of the original airborne hyperspectral data.

Figure 8 .
Figure 8.The standard deviations of the relative truth values acquired by the 5 PSFs and the standard deviation of the original airborne hyperspectral data.3.3 Point spread function improvement The Median Pixel Variability Weighted (MPVW) method provides a quantitative representation of surface heterogeneity, focusing on the prevalent spatial autocorrelation among spatial entities.It takes the intermediate reflectivity value in the local 17 × 17 window of the airborne hyperspectral image as the reflectivity value of the dominant spatial characteristics of the local window.And spatial autocorrelation and spatial variability can be measured by the variance between the reflectivity value of the dominant spatial feature and the reflectivity values of each pixel of the local window.Meanwhile, the reciprocal of the variance is usually taken as the weight.The farther a reflectivity pixel value is from the reflectivity value of the dominant spatial features, the larger the variance is and the smaller the weight is.In the MPVW method, the reflectivity value of the dominant spatial feature is determined through the following steps.The reflectivity values of all pixels in the local 17 ×17 window of the airborne hyperspectral image are sorted, and when the reflectivity values of some pixels are equal, only one of them participates in the sorting.If the total number of reflectivity values participating in the sorting in the local 17×17 window is odd, then the intermediate reflectivity value is taken as the reflectivity value of the dominant spatial feature of the window.If it is even, then the mean of the two intermediate reflectivity values is taken as the reflectivity value of the dominant spatial feature as follows:             square error fmax, fmin = the maximum and minimum values of the relative truth value yi= the relative truth value of heterogeneous surface i y ˆ= reference value n = the number of relative truth value 18) where G = the relative truth value of heterogeneous surface A = reference value l(G,A), c(G,A), s(G,A) = the luminance comparison function, the contrast comparison function and the structure comparison function Figure 11.The SSIM and the SSIM improvement Figure 12.The RMSE and the RMSE reduction Figure 13.The MAE and the RMSE reduction