High-detail and low-cost underwater inspection of large-scale hydropower dams

The article presents a practical method that combines low-cost camera systems with remotely operated vehicles (ROVs) to accomplish a comprehensive but economically feasible underwater survey of large hydropower infrastructures. Typically, inspecting reservoirs entails draining them off to allow for visual inspections, which are time-intensive, pose risks to operators' safety and are associated with generation losses. In this regard, ROVs are a much safer and more efficient alternative to traditional methods. The study was conducted at the Pack reservoir in Austria, where a reference framework was set up using terrestrial laser scanning and checkerboard markings for the above-water components. A ROV equipped with a GoPro camera and lighting system for the underwater recordings has been employed. Via a close-range photogrammetric approach, it was possible to generate 3D point clouds of the submerged infrastructure with a survey-grade accuracy level. Various strategies were explored to perform bundle block adjustment (BBA), among these were strategies where ground control points (GCPs) were used, strategies without the use of GCPs but pre-calibrated initial camera parameters and strategies with a combination of using both GCPs and pre-calibrated camera parameters in the BBA. The deployment of an inspection technique using low-cost sensors that can generate highly detailed three-dimensional models of submerged infrastructure areas is presented and discussed, allowing easy detection and localization for maintenance inspection, all while being cost-effective. The paper strengthens the suggestion of best practices that optimize camera settings, considering the effect of electronic image stabilization, suggesting its avoidance, and using advanced calibration methods.


Introduction / Motivation / State of the art
The maintenance of hydropower plants involves complex routine inspections.Relevant submerged parts of hydropower plants are checked at empirically determined intervals, on the one hand in the dry state with conventional surveying and on the other with divers in a submerged state.Due to the depth and for safety reasons, diving is not always possible or advisable.Both dives and the draining of submerged plant components are time-consuming and costly.In addition, dives always pose a risk to the divers themselves.Remotely piloted underwater vehicles, also referred to as Remotely Operated Vehicles (ROVs), equipped with special inspection devices, can make a decisive contribution to more efficient, safer, and qualitatively improved inspections in hydropower in the future (Capocci, 2017;Groemer, 2022).For inspections that correspond to the state of the art, ROVs are already established for visual inspections or surveying tasks in the offshore sector, e.g., in the gas and oil industry or underwater archaeology (Chemisky, 2021).In addition, ROV-based close-range photogrammetry shows high potential in infrastructure inspection (Menna, 2018).Underwater photogrammetry is an established technique for monitoring large and complex structures, as shown for coral reefs.The advantage of the photogrammetric approach is the ability to produce spatially detailed, accurate, and nondestructive measurements.Challenging factors are water turbidity, light patterns created by the reflection of the sunlight on the water's surface, and motion blur caused by moving the ROV too quickly.(Nocerino, 2020).Flat ports, as used in this experiment, also introduce refraction and distortion, which cause a departure from the classic photogrammetric mathematical model.(Maas, 2015) Against this background, we investigate the possibilities of ROV-based close-range photogrammetry with low-cost camera systems in this contribution.It can be shown that underwater inspection and mapping are feasible with decimeter (dm) accuracy compared to reference data from terrestrial laser scanning.
Currently, dams are inspected at fixed intervals, with an expert visually inspecting the dam in its drained state.This process is expensive, and the corresponding power plant is unavailable during the draining and refilling.Because of the energy transition, hydropower plants are playing an increasingly important role in making the electricity system more flexible, facilitating the integration of volatile photovoltaic and wind energy.Therefore, they play an essential role in enabling the decarbonization of the energy sector.Pumped hydroelectric storage plants especially have advantages in energy storage, such as fast response speed, flexibility to start and stop, the capability to track load changes, and the maintenance of voltage stability (Rehman, 2015).If the inspection method using ROVs and close-range photogrammetry can stretch these intervals and reduce the downtime of the power plant, an essential contribution to the energy transition is being made.

Test Site -Creation of the Reference Framework
The hydropower dam of the Pack reservoir -a 200m wide and 30m high gravity dam located in Styria, Austria -was chosen as the object of investigation.The location of the dam is shown in Figure 1.In dry conditions during a draining process, the reference system has been prepared using the following steps: In the first step, 77 checkerboard markings were placed on the waterside using a crane with a gondola to reach every part of the dam.Depending on the local concrete structure, either the markings were glued to the surface or a template was used to spray the marking on the dam.Especially on rough concrete parts, the spray option proved to be very useful.In Figure 2, the placement of the markings using the crane is shown.Figure 3 shows the markings on the dam.
Figure 2. A crane with a gondola was used to reach all parts of the dam to place the markings.In the next step, the 77 markers evenly distributed on the waterside of the dam were accurately measured with a total station (Leica TS16).These markings (cf. Figure 3) are the reference frame for the photogrammetric bundle block.After that, the dam was surveyed using a RIEGL VZ-400i terrestrial laser scanner (TLS) (Figure 5).Within the TLS campaign, the whole dam was recorded both on the waterside and on the air side.
The TLS was geo-referenced using the checkerboard markings.
The difference between the TLS and the markings has been evaluated by meshing the TLS and computing the cloud-tomesh distance between the markings and the meshed TLS using CloudCompare (CloudCompare, 2022).The error of the TLS compared to the markings was mean μ = 0.001 m with a std.Dev σ = 0.004 m and is in the millimeter (mm) range (Figure 4).This deviation is an order of magnitude smaller than the expected accuracy of the photogrammetric approach and can therefore serve as a trustworthy reference for validating the results.
The accuracy of the GCPs measured with the total station was evaluated in x, y, and z directions and has the following values:  Figure 5. TLS of the hydropower dam (height of wall: 30 m).

Underwater Recording
After measuring the reference framework and completing the dam inspection, the reservoir was again filled with water.Three months later, the dam was recorded in a submerged state.The underwater recordings of the waterside of the dam have been produced using the camera and lighting setup on the side of the ROV, which can be seen in Figure 6.Close-range photogrammetry was chosen as the inspection method.The camera was mounted at the side of the ROV (Figure 6) because steering the vehicle forward-looking is more stable than moving laterally.The ROV camera was mounted on the same side, for navigation and distance keeping.For photogrammetric purposes, only the recordings of the GoPro 10 camera were used.The GoPro camera was placed in a camera housing (flat port) with an additional battery pack inside.Details about the camera, the housing, and the battery pack are listed in Table 2.With the battery pack, the potential total recording time is more than 5 hours.An underwater area of around 1.000 m 2 has been surveyed by capturing seven 200 m long horizontal strips.Four dives on two separate days were necessary to survey the area.After each dive, the ROV was lifted from the water to change the camera battery and the battery pack.Adjacent strips were 70 cm vertically apart.The depth was kept with the auto-depth function and the integrated depth sensor.The surveyed area is shown in Figure 7.

Part
The turbidity in the reservoir was measured using a portable turbidimeter (Brand: BriSunshine) for turbidity measurement of liquids with a range from 0 -200 NTU.The turbidimeter is based on measuring scattered light from the infrared light source.The turbidity has been measured five times; the results are shown in Table 4.

Underwater Point Cloud
We extracted the frames from the videos recorded during the four drives.Using the roughly 25,000 frames, an underwater point cloud in Agisoft Metashape Version 2.0.1 (Agisoft, 2022) was created.39 of the 77 markings were contained in the surveyed area (Figure 7).Underwater point clouds were generated using different approaches, either using the markings as Ground Control Points (GCPs) or without GCPs.The other variants are described in detail in the following sections.In this approach, 39 markings were measured and used as GCPs during the bundle block adjustment (BBA).The GCPs have been introduced with different accuracies in six separate processing versions, and the GCP accuracies were varied from 1.e-1 m to 1.e-6 m.This is shown in more detail in Table 6.While the actual GCP coordinate accuracy is around 1.e-3 m, the GCP accuracy variation aims to test the GCPs' influence on the bundle block.A 3D point cloud was generated and compared against the TLS by computing M3C2 distances (Lague, 2013) using CloudCompare for each processing version.In this paper, we used the following settings to calculate M3C2 distances: normal radius = 0.5m, projection radius = 0.5 m.It has been found that the nominal-actual deviations were in the range of several meters using GCPs with an accuracy of 1.e-1 m or 1.e-6 m.The deviations could be reduced to the decimeter range when setting the GCP accuracy to 1.e-3 m.Introducing the GCPs with an accuracy of 1.e-3 m also yielded the smallest deviation to the TLS reference both in mean and standard deviation.The results of this evaluation can be seen in Figure 10; the corresponding values are listed in Table 6.

Approach 2: BBA without GCP usage
In a second approach, the camera parameters were fixed from the best-fitting model of Approach 1 (i.e., Model 4), and a BBA was tested and calculated without using GCPs.The respective camera parameters are listed in Table 5.This approach anticipates a scenario where the camera is calibrated either in the lab or in situ, e.g., using a frame containing a set of reference points.The underwater point cloud was created without any GCPs in the BBA, but using the camera parameters of the best point cloud (Model 4 from Approach 1) was a success.However, the point cloud was slightly mis-scaled; it was too big.We suggest using the plans usually available for dams to scale the 3D model and increase the accuracy.When comparing this result with Figure 10, one can see that the deviation is comparable to the best model created using GCPs.This positive result motivates us to employ the optimum camera calibration so that it is possible to create an exact underwater model without using GCPs.The advantage of this is that the work-intensive step of placing and measuring GCPs would not be necessary.

Approach 3: BBA with GCP and camera parameters.
As a final approach, the model with the slightest deviation from the reference set (Model 4) has been used as a starting point, and additional approximations of the camera parameters, namely the pixel size of the GoPro 10 sensor and the focal length have been introduced.By running another optimization in Agisoft Metashape, Model 9 was created.This model has the lowest total error, the lowest mean error, and the lowest standard deviation.The following values were used: Pixel size: 0.001607x0.001607mm; Focal length: 2.92 mm.

Summary and Results of the three Approaches
Table 6 gives an overview of the nine different models created in the three approaches described above.The total error was calculated in Agisoft Metashape and represents the GCP's absolute coordinate deviation from the corresponding underwater point cloud.Model 4 was the best result created in Approach 1 and was further optimized in Approach 3 by introducing approximations of the camera parameters in the BBA.Table 7 shows the results created using CloudCompare.Here, Model 4, Model 8, and Model 9 have the smallest errors.Again, Approach 3 generated the best result, which optimized Model 4 by introducing good camera approximations before the optimization.
This concludes the results generated so far and leaves us with several ideas on improving the process, with possible optimizations in every step from creating the underwater dataset to generating the point cloud.7. The mean and standard deviation for the nine models calculated in CloudCompare by analysing the M3C2 distance.

Conclusion and Outlook
By introducing GCPs in the bundle block adjustment, the deviation between the reference model and the underwater 3D point cloud was significantly reduced.
The creation of the underwater point cloud worked very well; the recording was done in one try without experience with an area of this size.As described in the following part, the camera settings were not optimal, and the recording was done only once.It is more remarkable that the model is complete, and the underwater point cloud shows a high level of detail when considering that the following camera settings were not set optimally for photogrammetric purposes (see Table 3):

Figure 4 .
Figure 4. Deviation between checkerboard markings and TLS mesh in [m].

Figure 7 .
Figure 7. Reference model of the dam with the underwater 3D point cloud (blue).

Figure 9 .
Figure 9.A part of the textured underwater 3D model.The red circles highlight the markings in this part of the dam.

Figure 10 .
Figure 10.Comparison of the distribution fitting of M3C2 distances between the underwater model / TLS using different GCP accuracies in the BBA.

Table 1 .
Accuracy of the GCPs measured with the total station.

Table 2 .
Specifications of the ROV and camera system Figure6.Cameras and lights, mounted on the side of the ROV.

Table 3 .
EXIF data of the recorded videos.

Table 5 .
Camera calibration parameters were used for the underwater point cloud created without GCPs.

Table 6 .
Variation of the GCP a-priori accuracy in Agisoft Metashape for weighting the GCPs differently during the BBA.