POSER: an oPen sOurce Simulation platform for tEaching and tRaining underwater photogrammetry

Underwater photogrammetry presents unique challenges due to the optical properties of water that, if not correctly taken into account, might affect the quality of the survey and the related 2D and 3D products. It is recognized nowadays the importance to train newcomers to underwater surveying, and extend and consolidate the knowledge of best practices for underwater data acquisition. Starting from this consideration, we propose the development of POSER, a 3D simulation framework designed to facilitate the teaching of underwater imaging principles. The project, an ISPRS Educational and Capacity Building Initiative, is built upon the open-source platform Blender, incorporating realistic modelling of the physical properties of water, including light refraction, scattering, and absorption phenomena, to simulate underwater surveying conditions. We foster a learning-by-doing approach, providing users with ready-to-use application scenarios inspired by real-life case studies. They will cover a range of application fields, from marine ecology to archaeology and subsea metrology, and allow users to address the complexities of underwater surveying practices. This paper introduces POSER to the community, presenting its educational vocation and describing its constituent components.


Introduction 1.1 A simulation software for teaching underwater photogrammetry
Underwater photogrammetry has become an essential technique for studying and understanding the underwater world in many application fields, such as exploration and mapping, industry and metrology, engineering, archaeology, ecology, etc.Nevertheless, producing accurate three-dimensional measurements underwater is still a challenge if compared to photogrammetric applications on land.In this paper, we present the development of POSER, a 3D simulation framework with the educational mission of supporting the teaching of underwater photogrammetry.Through the generation of computer-aided teaching and learning materials, POSER will help the comprehension of the physical properties of the underwater environment with its challenges, hereafter briefly described.
Water, especially seawater, is characterized by a high electrical conductivity, requiring electronic apparatuses and imaging sensors to be enclosed in a waterproof housing, mounting an optically transparent interface, usually shaped in the form of a flat or hemispherical port.The water and the ports of the housing affect the image formation, possibly introducing, among others, aberrations and distant-dependent geometric distortions.
The design of the camera housings and ports, in terms of mechanical and optical characteristics, is mainly ruled by the high density of water, approximately 800 times greater than air, which requires the enclosure to be appropriately dimensioned to withstand the significant forces exerted by the water pressure over its shell as the depth increases.
The refractive index of water varies as a function of temperature, salinity, pressure, and light wavelengths, thus requiring the imaging system to be optically calibrated for specific operative conditions.The presence of suspended particles due to organic and inorganic matter, such as phytoplankton, sediment, or pollution, causes scatter and backscatter phenomena that reduce water clarity.The wavelength-dependent absorption of light produces color distortions that are a function of both water depth (when ambient sunlight is available), distance from the camera and from the artificial light sources.
The height of the sun, depending on the location on Earth, time of day and season, and the sea conditions, influences the actual amount of light that is refracted, absorbed, or reflected upward.Scatter, backscatter, and wavelength-dependent light absorption reduce the amount of light reaching the underwater environment.Light is more absorbed in rough seas, whereas it is highly reflected in calm waters that act as a mirror surface.
Light rays are refracted by wavy surfaces, producing bright patterns known as caustics.These patterns are unfavorable for photogrammetric applications because they could affect the automatic extraction of 2D features in images and produce poorquality object texture.The intensity of light caustics depends on Sun elevation, water turbidity, and depth, with the effect significantly reduced after a few meters of depth.
Figure 1.The ISPRS POSER project will use a mix of reality-based and simulated underwater environments to support teaching the main aspects and challenges encountered in underwater photogrammetry.A calibration test object (left, Menna et al., 2017) is virtually inserted in an underwater 3D scene in Blender (center and right) to simulate an underwater calibration of a camera enclosed in a waterproof housing mounting a flat port.The simulated images are generated using a physically based ray-tracing approach that can accurately simulate physical phenomena such as refraction, scattering, backscattering, and volumetric absorption.The resulting image (right) shows heavy pincushion radial distortions and a dominant bluish color.
A broad technical literature exists on the physical properties of water and its influence on image formation for underwater photogrammetry applications (Morel, 1974;Jaffe, 1990;Bryson et al., 2013).However, these resources can be overwhelming and challenging for new practitioners, especially those with limited technical background.For the above reasons, novel teaching paradigms are increasingly in demand.These may leverage advanced digital technologies, such as computer simulations, augmented and virtual reality, and game-based learning approaches, to cite a few.
In recent years, the potentialities of simulation platforms have become increasingly evident both for research purposes (Nakath et al., 2022;Zwilgmeyer et al., 2021;Wang et al., 2019) and for educational goals (Luhmann et al., 2022).Among the different reasons, we can include the possibility of simulating environments that are difficult to access, performing a statistically significant number of tests, lowering costs, and systematically varying and controlling the different variables involved.Still, simulation solutions present limitations, as evidenced by the fact that they remain a very active research topic.At the same time, these solutions have undeniable potential in the educational context or in cases where researchers and practitioners have limited access to real-world resources.

Objective and goals of the POSER project
POSER, an ISPRS-funded Education and Capacity Building Initiative aims to build an educational simulation platform to create a learning environment for students and practitioners.In this frame, idealized scenarios and datasets for underwater photogrammetry will enable users to actively learn surveying skills and best practices by isolating relevant variables within a closed environment.So, while the project primarily pursues educational purposes, it will also contribute to improving simulation platforms applied in research contexts.
Working as a cross-disciplinary team of geomatics experts, oceanographers, archaeologists, data analysts, and instructional designers, we have devised a highly versatile empirical framework to test and compare the standard methods used in underwater structure-from-motion (SfM) and multi-view stereo (MVS) reconstruction, to help establish and communicate best practices, and to educate the community regarding the fundamental principles at play and demonstrate the effects of improper planning and data acquisition strategies.We employ a customizable open-source digital simulator, made publicly available, to begin the communal, cross-disciplinary development of rigorous standards for field-captured data.
POSER will use a mix of reality-based and simulated underwater environments to cover the main aspects and challenges encountered when planning and executing underwater photogrammetry surveys.The results of these simulations produce experimental image datasets (Figure 1), which can be used to help understand basic underwater imaging concepts, such as ground sample distance (GSD), image overlap and side lap, and artificial lighting configuration, to name a few.These datasets can then be processed using different photogrammetric software solutions and analyzed to show and compare differences in the potentially achievable accuracy.The rest of the paper explains the POSER simulator's main functionalities, along with preliminary results and relevant examples from different underwater application fields.

POSER simulator in Blender
A preliminary project of an underwater simulator was implemented in Blender to plan and analyze common camera networks utilized by NOAA, the US National Park Service, and Scripps Institution of Oceanography researchers (Figure 2, Giuseffi, 2020).The simulator shows the camera's movement over a scene in real time, with an adjacent view of the camera's local view.The rendered images are then processed in an SfM+MVS application, allowing a straightforward comparison of the 3D reconstructions.POSER builds upon this first experience, adding rigorous yet flexible modelling of water-airglass interfaces in Blender (Nakath et al., 2022).This component is necessary for assessing the impact of residual systematic errors introduced by non-centered dome ports and flat ports (Menna et al., 2020;Nocerino et al., 2021;She et al., 2022;Rofalski et al., 2022) and extending the analyses to optical aberrations such as field curvature with dome ports and astigmatism with flat ports (Menna et al., 2016(Menna et al., , 2017)).In POSER, an extension to cylindrically shaped ports is also introduced.
The reasons behind choosing Blender1 as a simulation platform are manifold.Blender is a free and open-source 3D computer graphics software toolset used for different purposes such as film animation, modelling, art, 3D printing, texturing, motion graphics, simulations, and interactive 3D applications.Based on an OpenGL GUI interface, it is cross-platform and runs on Linux, macOS, and Windows systems.It is highly customizable and features extensive Python API scripting that can control any tool available in the software.Moreover, it is supported by an active community worldwide with countless tutorials and examples available on the internet.Blender offers different rendering engines, among which the Cycles engine provides realistic physically-based rendering (PBR), volumetric rendering with absorption, scattering, subsurface scattering, support for different camera models from rectilinear to fisheye, depth of field, motion blur, different lighting models, to cite a few.Though Blender, with its multitude of options and interfaces, can be quite daunting for new users, POSER will incorporate custom menu interfaces within this system, providing a simplified means to manipulate relevant parameters that might otherwise be scattered across many different menus.It will also take advantage of the Python scripting functionality, enabling lightweight re-configuration and sharing of complex scenes.In this way, an experimental model, composed of many individual variables, can be easily exported, reviewed, and replicated externally.POSER will also leverage virtual reality (VR) functionalities available in Blender through the VR Scene Inspection add-on that exposes and extends the native virtual reality features of Blender in the user interface.
Multiple pre-designed scenarios (see Section 4) are envisaged in POSER to cover different fields, such as marine ecology, archaeology, and subsea industrial metrology.The application of VR will provide more immersive photogrammetry teaching and training sessions to reach and engage a broader audience.The compelling idea is to create accessible applications both "at home" and "in the classroom", enhancing knowledge transfer on best practices for underwater data acquisition.This trend is supported by the increasing availability of VR/AR/MR devices on the market and their growing adoption in various sectors, including education, scientific research, and tourism.Furthermore, using these technologies offers numerous advantages, including the ability to simulate challenging environmental conditions safely and in a controlled manner, enabling the users to tackle complex challenges more effectively.The availability of VR/AR platforms for learning underwater photogrammetry allows for reaching a broader and more diverse audience, facilitating the sharing of knowledge and collaboration among different communities interested in the documentation and conservation of underwater scenarios.

Simulated virtual scenes: different challenges from marine ecology to archaeology and subsea metrology
Whether by SCUBA diving or using an ROV, the time available for underwater operations is always limited, either because of decompression sickness (DCS) risks or the costs associated with support vessels and personnel needed for remote operations offshore.Therefore, underwater virtual environments envisaged in POSER can be used to optimize the design of image acquisition operations and train different aspects of underwater photogrammetry.Depending on the application field and the main goal of the photogrammetric survey, POSER will be used to simulate the different case studies and the associated environmental challenges.For example, in monitoring the growth and health of coral reefs, practitioners would need to plan an appropriate ground sample distance to fulfill the ecological analyses and design a proper camera network acquisition.Indeed, depending on the shape of the corals, standard aerial-like nadirlooking imaging networks may not be enough to fully model the 3D structural complexity of some coral species (Figure 3).
In underwater archaeology, before beginning any excavation phase, it is necessary to provide a complete site plan and accurate photographic documentation.These are critical operations to decide how to proceed with the excavation (Calantropio et al., 2021).After delimiting the area, setting the origin of a stable reference system, and starting with the first phase of excavation, the excavated area is covered with a grid square (site gridding), allowing the division of the excavation area into smaller squares.
Figure 3. Virtual scene (left) built using 3D models of different coral species from the Smithsonian 3D digitization collection2 used in this example for teaching the different levels of surveying complexity associated with various coral species (e.g., branching corals vs massive).A Seriatopora hystrix branching coral (middle and right) is selected as a case study to test different camera network configurations (a spherical spiral in the figure) and understand the level of completeness reached in the 3D reconstruction.Due to self-obstructions, the red and green parts in the reference 3D model are not reconstructed in the photogrammetric 3D model.
Photogrammetric surveys are then repeated during the excavation phase.The most relevant challenges for photogrammetric acquisitions are suspended particles in the aftermath of the excavation, dive time associated with the site's depth, and occlusions due to the size and shape of the assets.For the above reasons, it is clear how a simulation tool will help understand the site's constraints beforehand, allowing for a reasoned acquisition step, reducing dive time, and increasing the safety of the involved operators.
In the Oil & Gas industry, subsea metrology is required to document and monitor underwater infrastructures or support building operations, for example, by accurately measuring underwater assets' relative position and orientation (pose).Typically working at great depths, in these scenarios, the survey operations are carried out in complete darkness and, often in turbid water, using an ROV.In these cases, POSER will be used to explain the peculiarities of these scenarios, simulating, for example, the waterproof housings with thick dome ports needed to withstand huge pressures.
Also, it will help in understanding the best lighting configuration to minimize backscattering in turbid waters and plan the proper camera network for 3D inspection, from complex structures to long pipes.Figure 4 shows an example of a 3D model of a pipeline with a water volume only around the camera to improve the performance of the volumetric rendering.

Refraction
The refractive effects introduced by water and the ports used in the waterproof housings are among the most important differences encountered in underwater photogrammetry with respect to terrestrial and aerial photogrammetry.These affect the geometry of recorded images and thus may strongly influence the accuracy of 3D measurements.Additionally, scatter, backscatter, selective wavelength light absorption, and light caustics (mainly in shallow waters) are other key optical and physical phenomena that must be understood before dealing with underwater photogrammetry surveys.
In POSER, we start from the developments presented in GEODT (Nakath et al., 2022), where the raytracing approach used in the Cycles rendering engine was validated for dome and flat ports using a real laboratory tank and its 3D-modelled digital twin.Following this approach, a flat port underwater can be easily modeled in Blender by inserting a planar mesh surface between the camera and the 3D scene.A glass material shader is then given to the planar surface with an index of refraction (IOR) intended as the ratio between the medium in which the camera is located (air) and the next one encountered towards the object space.Therefore, if the thickness of the flat port is neglected, the IOR will be 1/1.33 (1 for air and 1.33 for water).Conversely, two surfaces are introduced if the thickness is to be considered.For the first surface, going from air to glass, the IOR will be 1/1.52 (in the case of a BK-7 optical glass); for the second surface, going from glass to water, the IOR will be 1.52/1.33(Figure 5).Similarly, a dome port can be modeled using single or double concentric spherical surfaces.Virtually, with this approach any shape of the port can be considered.In POSER, we plan to add other port shapes, such as the cylinder, as it is often used in small micro ROVs for inspection (e.g., the Deep Trekker 3 ).
The presence of flat, dome, and cylindrical ports not only affects the geometry of projection, for example, introducing distortions, but they also influence the focusing (on a closer and smaller virtual image) and image quality in terms of aberrations, such as field curvature and astigmatism, which negatively impact the accuracy of image measurements (Menna et al., 2016(Menna et al., , 2017)).In POSER, we will provide the possibility of rendering the simulated images, including the optical aberrations caused by the different port shapes.Moreover, single, stereo, and multicamera configurations will be allowed to feature several lens projection models, such as rectilinear and fisheye lenses.
Since Blender does not model the chromatic aberrations, we render the synthetic images three times, one per RGB channel, using the proper IORs (function of the different wavelengths).
We then combine the three image channels in an RGB image, as visible in Figure 5: a full-frame camera mounting an 18 mm lens is virtually mounted in a waterproof housing and evaluated on a specifically designed test object (Menna et al., 2017) using three different ports, i.e., flat, dome, and cylindrical.Water caustics are created by sunlight reaching and refracting at the water surface or on highly reflective walls and submerged bottoms.In some cases, artificial lighting, either immersed or emersed, can also be the source of such a phenomenon, for example, when surveying a semi-submerged cave (Nocerino et al., 2019).

Flat (EP off the planar surface)
Camera-to-object distance 1.3m, EP 3cm from the inner surface of the flat.
Flat thickness of about 15mm Corner blur due to astigmatism.Strong chromatic aberrations visible.
Dome port inner radius 7cm thickness 1 cm Corner blur due to field curvature

Cylinder (EP off-axis forward)
Camera-to-object distance 1.3m.Cylinder inner radius 5 cm, thickness 1 cm.EP off the cylinder axis with EP about 2 cm from the inner surface Corner blur from field curvature and astigmatism.Chromatic aberrations visible In POSER, the water caustics are generated using the physicalbased ocean modifier and random Voronoi textures, visible as temporally changing light patterns on the object of interest.

Water turbidity and influence on image contrast
In survey planning for photogrammetry applications above the water, the amount of air between the camera and the subject is usually disregarded.On the contrary, this is an essential aspect in underwater photogrammetry, where light absorption and scattering have a major impact on the image contrast with respect to imaging in air, thus effectively reducing the spatial resolution of the images (i.e., the ability to discern between adjacent image points without them being coalesced) and the image observation accuracy (Codevilla et al., 2015;Garcia and Gracias, 2011).Moreover, the selective distant-dependent color absorption introduces color distortions (Jaffe, 1990) and strong color casts typically varying from blue (tropical waters) to green (lakes).Using the volumetric rendering in Blender for scattering and absorption, the user can include scatter, backscattering, and water-selective color absorption.Figure 6 shows the underwater test object illuminated by two LED lights at about 30cm on each side of the camera in a scene simulating green water like those typically encountered in lakes.Two setups are considered to give the same ground sample distance, but the one on the left is taken with a wide-angle lens, while the one on the right uses a longer focal length that requires the camera to be more distant from the test object.The longer distance implies more water and, consequently, scattering volume between the camera and the subject, thus resulting in an image with less contrast and more intense color dominance given by the wavelength selective absorption.

Learning by Doing with Poser
Simulation-based learning is generally understood to have significant positive effects on acquiring new complex skills (Chernikova 2020).Simulations can help to deliver experiences and meaningful examples in a scalable way, which can more easily translate from abstract theory to practical application within real-world environments.Within the POSER simulator, exemplified scenarios with case studies derived from real projects in marine archaeology, ecology, and subsea metrology are envisaged.For each topic, an ad-hoc Blender project file is designed and prepared.Hereafter, some examples of possible topics are reported.

Imaging basics in underwater photogrammetry
One of the first fundamental concepts taught in photogrammetry classes is the image scale and how it affects the level of details visible in the recorded scene as a function of the characteristics of the imaging sensor, lens, and distance from the subject.The ground sample distance is defined as the pixel size expressed in ground (object space) units.Figure 7 shows an example of a 3D model of a French Polynesian coral reef from the Moorea Idea Project (Nocerino et al., 2020) that will be released within the POSER simulator.Although a pretty simple and intuitive concept, it can be difficult for beginners to understand how to choose between a camera sensor and lens properly, adapt the distance D to achieve a specific GSD, and how this affects the final 3D reconstruction.This example is an introductory exercise where the students compute the correct camera height from the sea bottom, given its focal length and pixel size, to meet a specific GSD, namely 1 and 3 mm in the figure.Precompiled Blender file examples where the scene is already loaded will be released.The student can interactively modify the camera distance from the reef and check the achieved GSD using the metric resolution targets spread on the underwater scene.Besides understanding the level of detail associated with a specific GSD, this exercise fosters the good practice of having reference control and validation methods in the scene to quickly verify the captured data directly on-site.

Underwater camera calibration
Students can practice the concepts behind camera calibration using different calibration objects, from the planar checkerboard to the 3D test object shown in Figure 1.They can analyze the image quality of the specific system used (dome, flat, cylindrical ports) and design and test the impact of the camera network on the determinability and accuracy of interior and additional calibration parameters.

Accuracy degradation due to unmodelled refractive effects
The presence of a flat port (or an uncentered dome port) introduces systematic errors that grow with increasing EP offset from the flat port (or dome port center).To allow users to experiment with this critical concept, an exercise is prepared in POSER where a wide-angle lens (18mm full frame equivalent) is mounted behind a flat port (EP 3cm off the flat).About 2500 nadir images are simulated over a 55x7 m 2 plot (Figure 8).The SfM + MVS with camera self-calibration performed in Metashape 4 results in a deflected shape of the reef with errors in the range ±0.5m for the presence of the unmodelled systematic errors (Figure 8) in a pure nadir network without oblique images 4 https://www.agisoft.com/(Nocerino et al., 2014;Menna et al 2020;James and Robson, 2014).

Refraction by waves and water caustics in underwater and through water applications
POSER can simulate underwater scenarios with caustics both under and through the water when the camera acquires images from above the water surface as when surveying in shallow clear water using a UAV (Figure 9).The students can practice with the challenges of orienting the simulated images with different wave heights, depths, and visibility conditions and test different algorithms for caustics removal (Agrafiotis et al., 2023) and feature extraction and matching, from traditional to deep learning ones (Morelli et al., 2024).

SCUBA diver-operated surveys: training the safety of diving during underwater photogrammetry surveys
Due to the increased pressure underwater, inert gases in the breathed mix dissolve in the body tissues proportionally to the surrounding pressure.As long as the diver remains at the increased pressure for a time less than the so-called nondecompression limit (NDL), an ascent to the surface at a slow ascent rate (i.e., less than 9m/min) is always possible and presents minimal decompression sickness risks (DCS).The NDL reduces significantly with the pressure with only 10 minutes available at 40m depth without decompression obligations with respect to about 90 minutes available at 15m depth.
Figure 10.A dive profile related to a SCUBA dive operated photogrammetric survey exported from POSER simulator and imported in the opens source software Subsurface dive log.
Moreover, the gas breathed in the tank provides only a limited dive duration.For example, with a standard surface air consumption rate (SAC) of 20l/min, a diver can stay about 15 minutes at a depth of 40m or 45 minutes at a depth of 15m.
Through Python API, we enable the export of the dive profile associated with the simulated pathways in POSER needed to carry out a specific camera network.These can be imported into dive logging software applications, such as the open-source free software Subsurface5 , or further optimized using pathfinding algorithms (Mangeruga et al., 2020).This would allow for further analyses of the dive profile, including verification that the planned dive is safe both in terms of DCS and sufficient breathing gas. Figure 10 shows the dive profile of the same example described in subsection 4.2, where a real 55x7m 2 reef area in Moorea is surveyed by underwater photogrammetry.In the simulated survey, 20 parallel strips for a total length of the path of about 1100m are to be carried out at a depth of about 15m with a swimming speed of about 0.2 m/s.The resulting dive of about 104 minutes and its profile is analyzed in the Subsurface dive log, revealing decompression obligations and indicating the need for a larger volume tank (double 12l tank, SAC 15 l/min) to carry out the survey in a single dive.

Conclusions
This paper introduced POSER, an oPen sOurce Simulation platform for tEaching and tRaining underwater photogrammetry.
Funded by the ISPRS Educational and Capacity Building Initiatives, its main aim is to provide the scientific community with an easy-to-use yet scientifically sound platform to learn the fundamentals of underwater photogrammetry and train with the main challenges encountered in surveying operational practice.
The paper presented the main aspects of the project, motivating the decision to use Blender as a development platform and describing how critical water properties, such as refraction, light absorption, and dropoff, are incorporated.To foster a learningby-doing approach, we are designing and developing several ready-to-use application scenarios.The users are expected to experiment with different photogrammetric acquisition strategies, including single, stereo and multi-camera rigs, and test their influence on the attainable accuracy, also critically reasoning with the distribution of control and scaling elements, such as scale bars and markers.Although POSER's primary connotation is education and training, it is worth emphasizing the great potential also in terms of research and experimentation that such a platform can offer the community.We are currently working on expanding the 3D datasets incorporated into POSER.These datasets may also be sourced from the 2023 ISPRS NAUTILUS project6 , and, at the same time, datasets created with POSER may contribute to the NAUTILUS web portal7 .We foresee that the POSER simulator will be used as teaching support in ISPRS-related underwater photogrammetry events such as tutorials and summer schools (Balletti et al., 2023).In future development, POSER's team will work towards the possibility of trainers integrating their datasets to cover the specific teaching aspects they want to cover in their classes.

Figure 2 .
Figure 2. A preliminary version of the POSER simulator in Blender shows the 3D scene and the moving camera input (left) in a side-by-side view, as well as Agisoft Metashape point confidence values (right), after processing the images of the same synthetic environment with varying acquisition patterns (lawnmower with parallel and cross strips, spiral strip, parallel strips).

Figure 4 .
Figure 4.An example of a subsea metrology scene in POSER where different lighting configurations, turbidity levels, and geometry of the pressure housing ports can be tested.A cube around the camera used is introduced for rendering purposes, allowing testing volumetric scattering and absorption (up) more efficiently.The resulting image is characterized by limited visibility and a green strong color cast (down).

Figure 5 .
Figure 5. Different simulated setups are used to assess the image quality with flat, dome, and cylindrical ports.The figure reports technical information on the setup and entrance pupil (EP) position.

Figure 6 .
Figure 6.Example of reduced contrast due to wavelength selective absorption and scattering.

Figure 7 .
Figure 7. 3D model of a coral reef plot in Moorea, French Polynesia, used in POSER to interactively teach underwater photogrammetry fundamentals such as GSD, overlap, and side lap.

Figure 8 .
Figure 8. 3D deformations caused by a flat port with nadir network.

Figure 9 .
Figure 9. Simulation of an image of a shallow underwater archeological site with Roman columns generated through the water as if the image was acquired by a UAV.The image displays typical distortions and water caustics caused by waves.