3D PHOTOGRAMMETRIC MODELS FROM HDR IMAGES: THE CASE STUDY OF THE FONTANONE VISCONTEO IN BERGAMO

: The world is a complex subject whose dynamic range between dark and light cannot be captured by digital cameras. The range of light that can be measured and recorded by a single exposure, is too limited and adapted only for the average light values of the scene. This paper focuses on the integrated surveys activities undertaken on the Fontanone Visconteo in Bergamo, to improve its knowledge and assess its conservation status. The specific lighting conditions of the environments (in particular, of the tank) have given the perfect opportunity for testing solutions aimed at the reconstruction of the clouds from different HDR processing to highlight the diverse surfaces’ treatment and understand the sequence of build actions. The purpose is to provide a thought-provoking discussion about the delicate issue of high-quality work that can be obtained from HDR images (12 bit) and RAW files such as TIF, DNG, and JPG (8 bit).


INTRODUCTION
The Palazzo dell'Ateneo, a significant monument sited in the Upper Town of Bergamo, stands between the Cathedral of Sant'Alessandro and the church of Santa Maria Maggiore, in the area occupied, in Roman times, by the civil forum.During the lordship of Giovanni, Archbishop of Milan, and his brother Luchino Visconti, among the numerous public and beautification works carried out, it was decided to provide the ancient bourg with a large reserve of water, capable of supplying it in every season.Fontanone Visconteo is the name given to this large water tank that was constructed right on Platea S. Vincentii, where in the past there was probably already a fountain, as evidenced by old documents and the most recent finding of a lead water pipe.The reservoir -so called due to its large capacity (43.800 brenta bergamasche equal to approximately 3.000 m³) -collected the water coming from the Vasi aqueduct (also called Castagneta), of probable Roman origin but certainly already existing in 1200.The channel followed a route of 3344 meters which started from the Noce Source and reached the Baluardo di Sant'Alessandro, intercepting six springs (fig.1).The cistern was partly underground but also had an emerging portion with rows of alternating light and dark marble throughout it.Its achievement can be dated to 1342, as recalled by a large white marble plaque bearing, together with the names of the clients, even those of the builders (Giovanni da Corteregia and Giacomo da Correggio, perhaps Como sculptors), still preserved on the northern front overlooking the square (Angelini 1955;1964), today dedicated to Reginaldo Giuliani (fig.2).Over the centuries, the Fontanone did not undergo any major changes, except for the re-plastering and the construction of a metal staircase in the 19 th century, aimed at facilitating access.In the centre of the northern wall, the mouths of the fountain are still visible, enclosed by two relief masks, probably added in the 17 th century.The famous painting attributed to Alvise Cima, known as Bergamo, perspective view of the late 16 th century or Bergamo, bird's eye view, considered to date the clearest view of the city in its medieval layout (later upset by the Venetian fortification built between 1561 and 1595) as well as the prototype of this genre in the city (Rossi 2012), shows the ancient building raised and covered by a gable roof; it was much probably used as a depot for weapons.Its southern façade is represented with three small doors while the northern front, although not depicted, was most likely open and punctuated by columns.Between the first and second half of the 18 th century, following the collection of ancient finds from archaeological excavations throughout the Bergamo province commissioned by the city council, the building was chosen as the site of the new city lapi- dary museum.The task of designing was given to Alessandro Pompei, an architect from Verona.The construction began in 1759 and ended in 1768, which is when the external bifurcated staircase was constructed.The following year, the tombstones were placed, under the direction of the historian Giovanbattista Rota, who was responsible for the first definition of the ancient history of the city of Bergamo (Rota 1804), presumably inside a loggia.Following a Napoleonic decree of December 25, 1810, the Ateneo di Scienze Lettere ed Arti di Bergamo was established.The new institution, born from the merger of the ancient Academies of the Excitati and the Arvali, was initially housed in the former refectory and in some adjoining rooms of the Rosate monastery.In 1818, the Imperial Royal Provincial Delegation arranged to assign the premises located above the Fontanone as its definitive headquarters to hold the meetings of its academic classes (Colmuto Zanella 2001).Thanks to the generosity of its members, it was then decided to renovate the place.A first hypothesis of adaptation was drawn up by Carlo Capitanio, architect in charge of the city's technical office, while never implemented.It was then followed by a second project by the architects and Ateneo's members Gian Francesco Lucchini and Giacomo Bianconi.The drawings, unfortunately now lost, involved closing the arcades of the portico with windows and creating two large rooms in the two opposite bays.The first one was intended to set up a large vestibule with a secondary entrance, while the second one was planned to serve as a library and offices.The architect Raffaele Dalpino, who was also an Ateneo's associate, conceived another arrangement in 1859.While maintaining the spatial distribution unchanged, through a wise use of architectural orders and rigorous respect for the modules adapted, in proportions, to the site and pre-existing structures, Dalpino proposed a new and refined configuration.Only apparently simple and symmetrical, the new project established a marked diversification of the two chambers, obtained by means of a different characterization of the furnishings, functions and arrangement of the tombstones (located only in the room on the left).A balanced synthesis of the two roles of museum and institution, which was retained until 1935 when the building became the headquarters of the Fascio Giovanile di Combattimento, the collections displaced, and the decor changed (Cassinelli 2001).After many years of disuse, the Palazzo dell'Ateneo -with its sober but elegant arched fronts framed by an order of smooth Corinthian pilasters (Licata 2015;Remo Dolci, Longaretti & Possenti 1982) -was refurbished in the 90s of the last century and used for social activities and conferencies.Following the nomination of Bergamo and Brescia as Italian Capital of Culture 2023, the University of Bergamo, together with the local Municipality and the Ateneo di Scienze Lettere ed Arti di Bergamo, started a research to improve knowledge of the building for the purpose of its valorisation.Investigations and diagnostics were undertaken to facilitate understanding of its stratigraphy and assess its conservation status.Specifically, integrated surveys were led combining several methods and approaches (direct, topographic survey and instrumental acquisitions with both active and passive electronic sensors).The lighting conditions of the environments (particularly of the tank) allowed some tests on the reconstruction of the clouds from different HDR processing to facilitate the appreciation of the different surfaces' characteristics and understand the succession of build actions (fig.3).The introduction of multi-image photogrammetry (SFM, Structure from Motion) and 3D laser scanners has encouraged the production of hyper-realistic 3D models (fig.4).This is an already widespread practice, but which nevertheless requires following a correct operating practice and a rigid workflow.The objective of this paper is to offer insightful advice on the delicate issue of producing high-quality digital surveys (Versaci, Cardaci, and Fauzia 2024).Today, mobile phones allow anyone to take pictures, but not everyone is a professional photographer.It is a false belief to think that purchasing a 3D laser scanner or photogrammetric software is sufficient to become surveyors or designers without having any, or enough, knowledge and skills (Cardaci, Versaci & Azzola 2023).

HDR IMAGES & TONE MAPPING CONVERSION
Electromagnetic radiation that has a wavelength between 400 and 700 nanometres (nm) is sensitive to the human eye.The radiation that hits the retina is what creates the visual experience.The relationship between light on the retina from all parts of the image is what influences what we see.The brain governs this process, which is perceptual and subjective.If the eye behaved like photographic film, human vision would be much simpler.Unfortunately, the human retina does not have the same unique and objective response to light as the film does.On the other hand, human visual receptors can record a brightness change greater than 10-12 log units, while camera sensors only 5-6 log units (Artusi et al. 2017).Digital cameras are incapable of fully capturing what humans sees with their eyes.The world is an intricate subject whose dynamic range, from dark to light, far exceeds the ability of the digital camera to capture it.The dynamic range is the ratio between the system's maximum and minimum light responses.The traditional camera's ability to capture light with a single exposure is limited and only suitable for the average light values of the scene.The measurement is done using the term Exposure Value (EV).A scene that can display a dynamic range of X:1 (e.g., 100:1, 3.000:1, 12.000:1, etc.) indicates that the maximum light coming from a segment of a scene is X times (100, 3.000, 12.000, etc.) more than the minimum.Therefore, considering the real-world luminance range and the limitations of cameras to keep scene details, the difference among human visual perception and photographs is a big problem in computer graphics.A partial solution is to use complex image processing techniques to transform high dynamic range (HDR) scene radiances into pleasing LDR images.Advanced HDR imaging technologies are today widely used in various fields.Generated pseudo HDR contents can accurately replicate the authentic appearances of real scenes, ensuring faithful reproduction for viewers.Standard desktop displays only play LDR content.HDR images, which are intended for use on normal RGB screens, need to be processed using tone mapping methods.Consequently, significant research efforts have been dedicated to devising tone reproduction algorithms.The objective of these algorithms is to render high dynamic range images effectively on lower dynamic range displays.A challenge in high dynamic range imaging is displaying radiance maps on conventional mediums like LCD panels, that typically lack in a loss of details in dark and bright regions.• High dynamic range (HDR): >15 EV.Tone Mapping is the process of transforming an HDR image into an LDR image by altering its colour and pixel size.All this is done to make the underexposed pixels brighter and the overexposed pixels less white.A sharper image can be created by adding more details in areas that are too light or too dark.There are many algorithms that carry out this transformation that typically rely on finely tuning a set of parameters.Finding the optimal settings can  The replacement of the current HDR transformation software is inevitable, but it won't occur until a few years from now.

PROCESSING AND RESULTS
The study was conducted in three distinct phases, adhering to a specific protocol (figs.5-7).These phases were: the acquisition phase, the reproduction phase, and the interpretation and comparison of the models phase.Both traditional topographic networks and a 3D laser scanning campaign were employed in the surveying of Fontanone Visconteo.On both the vaults and walls were placed more than twenty targets.With the help of rigid triangulation and trilateration, the coordinates were determined with high precision and less than 1 mm uncertainty.The 3D laser scanner point cloud also provided coordinate values for the points.The difference between the two measurements on the targets provided the accuracy of the system (average error is 1.8 mm and maximum error is 4.2 mm).To take the photographs, the camera was positioned on a topographic tripod.The shots were taken using a remote control to eliminate vibrations.From every station, three images of the same scene were taken: one overexposed, one normalized, and one underexposed.The photographs were saved in the camera's internal memory in both RAW (-1.5 EV; 0; 1.5 EV) and JPG formats.The processing of RAW photographs resulted in images with HDR mapping in 8-bit TIF format.Five different textures were obtained in order to test the reliability of the models as the   textures varied (HDR Smooth, HDR Creative, HDR Interior, HDR Realistic and HDR Natural developed with the HDRsoft® Photomatix Pro 7 software).The photogrammetric models were reconstructed with the Agisoft® Metashape software following the usual practice: alignment of the cameras (sparse cloud), insertion of target coordinates, optimization of the cameras, calculation of the depth maps, construction of the dense cloud, construction of the mesh model and surface mapping.The procedure was performed nine times: five times with fake HDR images, three times with RAW images, and one time with JPG images.To guarantee the same workflow for all series, the processing parameters were kept constant throughout all processing steps.The points were not filtered (Gómez-Gutiérrez et al. 2015;Mar-nerides, Hulusic & Debattista 2020;Santosi et al. 2019).In photogrammetric applications, the term HDR is often misunderstood.Many scientific works confuse HDR images with simple photographs with a greater emphasis on the light and dark parts.Raw sensor data (RAW information) or transformable images are used by normal Image-Based 3D Reconstruction software to make it visible on any device (operated automatically by the camera or through digital development).The sensor's RAW data typically has a colour depth of up to 12 bits, resulting in greater colour detail.Raw-Converted images, usually in TIF, JPG, or PNG formats, often have a color depth of only 8 bits because they are lighter to process and easier to reproduce (Kontogianni et al. 2015;Ntregka, Georgopoulos, Santana Quintero 2014;Suma et  al. 2016).It is essential to keep in mind that commercial displays are restricted to 8-bit depth.Directly processing HDR images is not possible with any widely accessible programs.What is commonly known as HDR is actually an 8-bit file that has been tone mapped (Cardaci et al. 2022).The experiments showed that the models produced by these fake HDRs are as accurate as those derived from processing traditional 8-bit images.Tone mapping treatment can result in a decrease in the reconstructive process by comparing the accuracy of the artifact to that obtained from JPG data.It is preferable to process uncompressed and non-destructive formats, such as TIF and PNG, instead of compressed and destructive JPG files.Using RAW images always yields the greatest accuracy and geometric precision, even if they take more computational time.The best results were found to be achieved with slightly overexposed RAW images (by 1/3 or 2/3 EV) compared to those exposed normally.Avoiding underexposed catches is important due to their increased noise and less dense cloud of points.Unfortunately, the model mapping is plagued by problems related to the general alteration of colours and large areas with black and white pixels.RAW models have a high level of geometric reliability, but they lack textures.It is common for them to fail to meet the engineer and architect's requirements when drafting the intervention and restoration project.Models with HDR mapping (obtained from TIF or PNG images after a conversion treatment) are a fair compromise between reliable metric accuracy and optimal graphic restitution for the recognition of wall stratigraphy, construction anomalies and decay.The best solution to the problem is, however, in the integration and disjointed use of the two types of images: RAW images (slightly overlapping) for the construction of the geometric model and TIF or PNG images with HDR mapping for the production of textures.The ideal procedure is to build a 3D artifact (low cloud, dense cloud and mesh model) directly from the raw data and then to replace the HDR mapping in TIF or PNG format (which is better suited to viewing) in the texture construction phase.The process does not involve errors in re-projection and/or overlapping of images because RAW and HDR files are obtained by the same shot, taken with the camera placed in the identical position.Even though it may increase work hours, this technique yields exceptional results in both geometrical and graphic rendering.
Tone mapping creates this by compressing the dynamic range of the radiance maps to fit within the display devices' range.Recent literature has provided numerous tone mapping methods to address this issue.The following HDR and LDR categorization has been proposed in 2015 by the Moving Picture Expert Group (MPEG) and now extensively used in the field (Han, Khan & Rahardja 2023): • Low dynamic range (LDR): ≤ EV; • Enhanced dynamic range (EDR): >8 EV and ≤15 EV;

Figure 4 .
Figure 4.The model obtained by 3D laser scanner.

Figure 5 .
Figure 5. Flowchart of the phases of experimentation and results.

Figure 6a .
Figure 6a.Summary table of characteristic data, errors and accuracy of RAW and HDR photogrammetric models.

Figure 6b .
Figure 6b.The diagrams indicate that the generated by fake HDR images (TIF -8bit) lack very high accuracy (similar to JPG -8bit); the reconstructive process is reduced by the tone mapping treatment.RAW data can always provide the highest accuracy and geometric precision.

Figure 7a .
Figure 7a.Comparing photogrammetric models with the 3D cloud laser scanner (hyposcopic view of the vault: RGB and Confidence Value mapping).