Lighting Model for Underwater Photogrammetric Captures

Photogrammetry is an established technique for producing 3D representations of submerged structures in shallow, naturally lit environments. Natural light is not available in more extreme environments such as in the deep ocean or submerged caves, which are major applications for photogrammetric survey. Additionally, these environments are often accessed with resource-limited sensor platforms, necessitating efficient use of power constraining the level of artificial illumination that can be deployed. A method to estimate the amount of light needed to achieve sufficient image quality in underwater photogrammetric acquisition systems is presented.


Introduction
Photogrammetry, and particularly underwater photogrammetry, relies heavily on clear and sharp images to enable feature detection and localization (Bobkowska et al., 2021, Burdziakowski andBobkowska, 2021).Recent developments in underwater photogrammetry applications are increasing the path loss of light in the imagery, whether due to increasing standoff distances, turbid environments, cloud cover, time of day, caustics, etc (Song et al., 2022).With many of these new applications occurring in a limited light environment (such as benthic or overhead environments), motion will typically be a major contributor to image blurriness.Dynamic elements of ocean environments such as surge, swell, and current will contribute to camera motion.
In photography, we can adjust parameters such as shutter speed, aperture, ISO, and illumination to achieve the desired exposure.In underwater photography, many of these parameters are constrained due to the low light environment, thus, bringing additional light allows more flexibility in achieving appropriate image quality.Modeling the required lighting allows us to determine a design's ability to meet mission requirements.Recent research primarily focuses on optimizing the light placement to reduce loss due to backscatter (Song et al., 2021) or color correction (Song and Baik, 2023).
We can eliminate motion blur by limiting the exposure to the time it takes for the camera's projected image to traverse 1 pixel, meaning that, the exposure is constrained such that the maximum blur is 1 pixel.This constraint will drive selection of lens aperture and ISO value.Additional light can also be brought to bear on the scene, which will allow more ideal lens aperture and ISO values.Insufficient light will result in too large an aperture (shallow depth of focus) or too high of an ISO value (increased graininess).Excess light will result in a smaller aperture (larger depth of focus) and lower ISO value (reduced noise), up to practical limits.
To simplify the formulation, we assume the following: • Scene is planar and normal to the optical axis, and is uniformly reflective.
• Artificial illumination is uniform over the beam width and visible scene.
• Image degradation due to backscatter is not considered.
• Camera is infinitely sharp.
• Camera is moving parallel to the scene and towards the "up" direction of the image.

Definition of Terms
We denote source parameters with the subscript Σ.
• θΣ 1-D angular width of the light source beam in rad • Φv,Σ Illumination source luminous power in lm • zΣ Illumination source altitude over scene in m • AΣ Planar area of illumination at the scene in m 2 • ΩΣ Illumination source solid angle (2-D angular width) in sr We denote camera parameters with the subscript c.
• fc Camera lens focal length in m • Nc Camera lens aperture size in f-number • Ic Camera exposure index ("Film Speed") in ISO units We denote the projected image parameters with the subscript I.
• wI Camera projected image width in m We denote scene parameters with the subscript S.
• rS Scene reflectivity as a value from 0 to 100 in %

Model Derivation
We first compute the projected areas of both the camera view and the artificial illumination.The illuminated area we compute using Equation 1.The projected camera view area is computed with Equation 4ΩΣ = 2π 1 − cos θΣ 2 (5) We next compute the solid angle ΩΣ in sr using Equation 5.This provides us a measure of the "size" of the illumination beam.We also compute the solid angle Ωc in sr of the camera view using Equation 6.
We then need to compute the light attenuation down through the water column.Idso and Gilbert provide a model for this, shown in Equations 9 and 10 ( Idso and Gilbert, 1974).We apply these to the illumination sources present in the scene (artificial light and solar), resulting in Equation 11.
Since no scene reflects 100% of light, we account for some loss of light due to scene absorption with scene reflectivity rS in %.
The reflected luminous flux Mv,S from the scene is given by Equation 12.
Lv,S = Mv,S Ωc (13) We assume that the scene is well lit.Thus, the luminance of the scene is given by Equation 13.Since the light traveling from the scene to the camera is still subject to attenuation, we need to again take into account this path loss.The scene luminance from the camera's perspective is given by Equation 14.
ISO 12232:2019 provides the math to relate Lv,S, Ic, and tc, shown in Equation 15and Equation 16(International Organization for Standardization, 2019).We can rearrange these to facilitate computing the scene luminance, shown in Equation 17. Ev,SrSe We can then formalize the entire artificial light model with respect to illuminance in Equation 21.We discuss these terms and their significance and semantics in section 4.

Model Discussion
We can broadly break up Equation 21 into a few major groups.
Ev,S is the light from the artificial light source.
Ictc are the camera exposure parameters.
We can intuit the left side of Equation 21 as the "amount" of light coming into the camera, and the right side of Equation 21as the camera parameters required to achieve a nominally exposed image.
The relationship between the aperture size Nc, ISO speed Ic, and exposure time tc is what we expect from photography.Imaging a brighter scene requires decreasing tc or Ic, or making the aperture smaller (increasing Nc).Correspondingly, doubling tc (increasing by one stop) can be compensated by closing the aperture (reducing by one stop) or by halving the ISO speed (decreasing by one stop).Figure 2 shows an example of the relationship between focal length fc and the camera solid angle Ωc, for which the expanded model is given in Equation 8.Over the domain of possible values of fc, wc, and hc, arctan will have a range of [0, π 2 ), and as a result, Ωc has a range of [0, 2π) sr (half of a sphere).An example curve of this behavior is shown in Figure 2. Due to this, fc, wc, and hc have a very limited influence on the exposure behavior of the camera system.If we expand the illuminance terms, we get the following: If we look at a scenario in which we only have artificial lighting, the model becomes Often, we want to apply this model to see how much light we need for a particular environment.The exposure time tc is limited by motion blur.This can be computed from the camera velocity vc in m s −1 , pixel pitch pc in m, focal length fc in m, and camera altitude zc in m as shown in Equation 28.
Rearranging the artificial light model (Equation 27) results in Equation 29.
Since the light from the artificial source is not collimated and experiences appreciable spread over distance, we have a significant dependency on the artificial light source altitude zΣ in Equation 29.The z 2 Σ term significantly limits the impact of artificial lighting at significant standoff distances such that it is more effective to be closer to the scene than it is to double the amount the light carried.This is demonstrated in Figure 4.
Looking at this model from the perspective of how much light is required to illuminate a given scene, we get the curves in Figure 5. Current LED technology can achieve on the order of 200 lm W −1 to 300 lm W −1 .Thus, in order to achieve millisecond exposure times at a range of 10 m with ISO 1600, we need to generate on the order of 1 × 10 6 lm, requiring approximately 4 kW, which would be about enough to boil a liter of 5 • C water in 2 min.

Model Validation
We have conducted some experiments to validate this model, and anticipate continuing to validate this model.Since it is difficult to precisely measure rS and z d , the majority of our tests will assess whether the model holds general trends and produces reasonable rS and z d .As this model is intended as a design estimation tool to predict lighting needs, it need not be exact.This first major test was conducted on 2023-05-15 at the university's pier, shown in Figure 6.We attached a 20 MP machine vision camera 4 and 40 000 lm of dive lights 5 to a floating rig then placed a checkerboard target at various depths.At each depth stop, we executed several captures with varying exposures (0.143 ms to 51.2 ms) and gain values (4 dB to 32 dB).A subset of these data are shown in Figure 7.
Once we captured these data, we examined the exposure of each image, accepting only those whose peak pixel intensity was between 0.5 and 0.7 of full dynamic range.This results in the data shown in Figure 7.
We can visually estimate the Secchi distance at approximately 6 m -we cannot discern the edges of the checkerboard in the imagery at this depth.If we assume 30 % reflectivity and an experimentally determined ISO/gain mapping of Ic = 18 × 10 0.050g , then we get a very similar curve, as shown in Figure 7.
From the data, we see that we consistently get shorter exposure time at scene depths approaching the Secchi depth.This indicates that light reflecting from backscatter is impacting the scene exposure.If we fit the model to data collected up to 70 % of the Secchi depth, we determine the best fit Secchi depth z d to be 14 m and best fit scene reflectivity rs to be 9 %.The resulting curves and model differences are shown in Figure 8.We conducted another set of tests on 2024-03-22 at the university's pier.For this experiment, we used a Nikon D780 with a 24 mm f/1.8 lens in a dive housing with 4 BigBlue 10 000 lm dive lights attached approximately 0.5 m on either side of the camera housing.We placed a checkerboard target at the base of the pier, then dove the camera in a vertical transect above the target while continuously capturing images.The camera was configured in aperture priority, ISO 8000.We measured the Secchi distance at approximately 2.8 m using a Secchi diskthis was also confirmed using the checkerboard.If we assume a 30 % reflectivity, we get the curves shown in Figure 9.
In these data, we see that the model better fits the data.If we again fit the model to the data collected up to 70 % of the Secchi depth, we determine the best fit Secchi depth z d to be 14 m and the best fit scene reflectivity to be rs to be 2.7 %.The resulting curves and model differences are shown in Figure 10.
In both experiments we conducted, the model appears to overestimate the amount of light required to fully illuminate a scene, especially as the distance to scene approaches the Secchi depth (i.e.limit of visibility).Since this model does not account for light reflected due to backscatter, it falls apart when backscatter begins to dominate the reflectivity of the scene.Additionally, features in the scene will likely be indistinguishable when imaged close to the Secchi depth, which will likely cause the photogrammetric model to fail.

Conclusion
In this paper, we propose and validate a mathematical model to estimate the amount of light required to achieve a well exposed image using only artificial illumination for underwater photogrammetry.This model is parameterized by Secchi depth, total artificial light, and camera exposure parameters.Experimental validation show that the model tends to imitate the exposure behavior in conditions where backscatter does not dominate the reflectance, and is otherwise off by less than 10 ms.This is likely enough to provide an engineering estimate to determine an appropriate amount of light.
Experimental validation indicates that this model begins to break down when imaging near the Secchi depth.The data suggests that more light is being reflected by the scene, which the model is not accounting for.In both experiments, the turbidity was due to fine particulates in the water column.In all likelihood, this model will also break down in the presence of large particulates in the water reflecting light.Additional work using different attenuation and reflectance turbidity models will assist this model in being more accurate in those regimes, however, such environments are not conducive to quality photogrammetry.
One particular application of interest is photogrammetric survey during times when ambient light is available.Availability of ambient light would affect the light entering the aperture of the camera, so modifications would be required to assess the contribution of ambient light.A reformulation of this model to allow arbitrary illumination sources, or to allow an additional arbitrary luminous source, would allow using the solar illuminance curves provided by Jones and Condit to estimate the maximum light required by a dive team (Jones and Condit, 1948).

•
Figure 1.Diagram of terms.Red elements are associated with the artificial illumination source.Blue elements are associated with the reflected light and camera. 1 See ISO 12232:2019(E) Section 4.1 2 See ISO 12232:2019(E) Equation B.2 and B.4 3 See ISO 12232:2019(E) Section 4.1

Figure 2 .
Figure 2. Camera solid angle vs focal length for various sensor sizes

Figure 3 .Figure 4 .
Figure 3. Sample pixel size (Ground Sample Distance) and shutter speed limit curves.Cameras in order are: Olympus TG-6 with 25 mm lens, Sony a7R IV with 35 mm lens, Nikon D780 with 24 mm lens, Lucid ATX204S with 8 mm lens, and GoPro Hero 10

Figure 6 .
Figure 6.Lighting Test Rig deployed at UC San Diego's Ellen Browning Scripps Memorial Pier

Figure 7 .
Figure 7. Accepted exposure values and predicted exposure values over depth from Lucid ATX204S

Figure 8 .
Figure 8. Accepted exposure values and best fit predicted exposure values over depth from Lucid ATX204S

Figure 9 .
Figure 9. Exposure values and predicted exposure values over depth from Nikon D780 Scene luminance as seen by the camera in cd m −2