FLATTENING COMPLEX ARCHITECTURAL SURFACES: PHOTOGRAMMETRIC 3D MODELS CONVERTED INTO 2D MAPS

: The paper describes a workflow to flatten 3D photogrammetric models of undevelopable surfaces into unfragmented 2D texture maps. The aim is to create a texture map with reduced fragmentation compared to typical photogrammetric texture files associated with 3D models. Geometric reformatting of the mesh is required to achieve an unfragmented final texture image with enough visual quality to allow for its use in 2D editing software (e.g., Photoshop, Illustrator, etc.). With a lighter model and defragmented texture, graphic documentation of conditions, treatments, or other relevant information can be performed directly. The approach considers both simple and complex architectural surfaces, with particular attention to elements that cannot be developed without introducing distortions. In the case of historic buildings, such surfaces constitute the majority, especially in the case of decorative elements and irregular vaulted systems. The approach extends to full 3D models, particularly those where orthomosaics would result in stretched details. We will discuss two methods: the first is particularly suitable when the photogrammetric project (oriented images) is still available, and the second applies to generic 3D models without the availability of the original images.


INTRODUCTION 1.1 Orthophotos and undevelopable surfaces
Many projects in the field of digital recording generate digital orthomosaics, which combine metric and visible surface information (the surface's texture).Orthomosaics are products of the photogrammetric workflow, in which 2D images are processed to obtain 3D measurements.Orthomosaics provide an orthogonally projected 2D representation that is useful for various applications, such as condition mapping (e.g., materials, decay, treatment areas, etc.) or the production of line drawings.Although useful for several applications of heritage documentation, orthomosaics exhibit intrinsic limitations when the documented surface diverges from a regular flat surface (e.g., vaulted systems, capitals, and medallions, just to name a few).Metric quantities such as distances, angles, and areas measured in orthomosaics of 3D objects could not correspond to the real ones.In the case of such complex surfaces, the direct use of the 3D model and its texture is ideal.However, using 2D representations is often more practical for most applications, especially for users who are not experts in 3D modeling.Creating a digital orthomosaic without metric distortion is possible only when the object's surface can be developed without stretching or tearing.Developable surfaces can be thought of as those that can be formed by rolling a flat sheet of metal.As explained by Snezana (2010), "developable surfaces form a very small subset of all possible surfaces …"."There are three types of developable surfaces (excluding a fourth type, the planar surface): • surfaces in which generating lines are tangents of a space curve: this type of surface is spanned by a set of straight lines tangential to a space curve, which is called the edge of regression; • surfaces which can be described as a generalized 'cone' where all generating lines run through a fixed point, the apex or vertex of the surface; • surfaces which, in the same manner, can be described as a generalized cylinder, where all generating lines are parallel, swept by a set of mutually parallel lines".Cones and cylinders are typical surfaces that can be orthorectified without geometric distortions.Some open-source and commercial software have specific tools to unroll the model during orthorectification.However, most surfaces in the field of heritage documentation do not have such regular geometry.Even in the case of architectural elements that are theoretically developable (such as a barrel vault), the reconstructed surface reveals geometric deviations from the ideal shape.This means that for many heritage surfaces, orthomosaics are not useful for direct, undistorted mapping and measurement.Although 3D models with fine geometric details can be reconstructed in an almost fully automatic way, when the model must be textured and exported, the final texture file (i.e., image file) exhibits strong fragmentation, small texture islands, and a large number of cuts (Maggiordomo et al., 2021) as shown in Figure 1.The resulting 3D model is usually based on a triangular mesh (tri-mesh), which can capture complex and fine details with several million faces but is therefore data heavy and difficult to use in situ.At the end of 2023, the latest release of Reality Capture (version 1.3) introduced an experimental UV unwrapping process based on mosaicing, which results in fewer UV islands (https://www.capturingreality.com/News).

Overview of the proposed method
This paper proposes a different solution in which 3D models (of both developable and undevelopable surfaces) are converted into quadrilateral (quad) meshes and flattened, obtaining unfragmented 2D texture maps.Figures 1 and 2 depict the basic idea behind this work.As a test case, a complex umbrella vault in Castel Masegra (Sondrio, Italy) was reconstructed from a set of digital images into a textured 3D model (Figure 1).The model was generated in Agisoft Metashape and exported as .objwith an associated

Additional considerations on previous work
Methods and procedures described in this paper extensively use prior work developed by several authors.The first step is always creating a 3D model with texture, which can be achieved with different photogrammetric -structure from motion software.The image-based modeling pipeline usually includes image orientation, dense image matching (for point cloud -mesh generation), and texture generation.The technical literature provides numerous examples of applications in which photogrammetry generated detailed 3D models.The reader is referred to Means (2017) and Luhmann et al. (2019) for more details regarding heritage documentation.One of the fundamental steps in the proposed workflow is the conversion of the photogrammetrically generated tri-mesh into a quad mesh (Bommes et al., 2012).Such remeshing operation (also called retopology) is very delicate and must ensure a consistent reduction of the number of elements (quads in this case) without significantly losing the level of detail.The remeshing step allows for the creation of a cleaner and smoother geometry.The technical literature on this is substantial and several retopology tools are today available as free and opensource software or commercial applications.The last fundamental operation considered in this work is UV mapping (Sheffer et al, 2006), for which technical literature offers several solutions.The flat representation of the object's texture is obtained through an unfragmented texture map with visual quality sufficient to enable editing operations in image editing software.Quad meshes are particularly suitable for texture mapping (Bommer et al., 2013).Different methods for the creation of UV maps were tested.At the same time, distortions can be visualized, making the user aware of inevitable effects.

METHOD 1: RE-MESHING AND TEXTURE MAPPING WITH SPECIFIC UV MAPS
After generating a 3D model with texture using a photogrammetric procedure (Agisoft Metashape was used for the proposed example), a .objfile with a (fragmented) texture map can be exported, obtaining the results shown in Figure 4 (Case A, top, located close to Castel Sant'Angelo, Rome).The different "texture islands" come from the numerous images used in the photogrammetric workflow.Editing operations on the texture file are difficult due to the lack of continuity.
The original tri-mesh has about 712,000 faces and a single texture file with a resolution of 8192×8192 pixels was generated.After applying the proposed procedure, a new smoother mesh with only about 14,000 faces is obtained together with a new texture layout, which is much more regular.It can represent the entire texture with a graphic quality allowing further editing directly on the image file (Figure 4, case B, bottom).In the new unfragmented texture, the flattened lion's face appears wider than the actual 3D geometry.In fact, all the tiny faces representing the complex geometry were flattened to allow direct editing in 2D of small details, which are usually lost in the case of a traditional ortho-projection.
The workflow is shown in Figure 5. First, the original tri-mesh is converted into a smoother quad mesh.The example in this paragraph was carried out with ZRemesher, available in Zbrush.
Excellent results were also obtained with other retopology tools.
For instance, Quad Remesher is a plugin for several commercial and open-source software.A free tool is Instant Meshes (Jakob et al., 2015).
The different experiments have also shown that a preliminary edge refinement of the photogrammetric model improves the visual quality.The boundary line of a photogrammetric model can be rather jagged and irregular, resulting in unsatisfactory results during the remeshing phase.The boundary should be simplified, reducing the number of triangles with an interactive approach and visually evaluating the final quality.Modifications of the initial mesh are not a problem since the texture is unnecessary.The texture will be created in a second step.The results after the tri-to quad-mesh conversion are shown in Figure 6.Small details are lost, and the final mesh is smoother.This smoothing issue will be addressed in the next sections.
It is critical to note that this step aims to create a new texture map with an unfragmented structure.The aim is to create a novel quad mesh with the same level of geometric detail as the initial photogrammetric mesh.The focus is on the texture rather than the geometry.Of course, remeshing must provide a model correctly representing the overall geometry, with limited geometric discrepancy.
For this reason, a mesh-to-mesh comparison is carried out in CloudCompare to understand the discrepancy between the available two models.Figure 7 indicates an overall error of -0.015 mm ±0.65 mm (the original model has a size of about 1200 mm), corresponding to a sufficient relative accuracy (the relative error is about 0.05%).In the case of unsatisfactory meshing results, the model can be re-meshed with a higher number of target quads.
Another fundamental consideration for correctly applying this method is the need for a set of oriented images.Therefore, the photogrammetric project will be used at the beginning of the project (to create the tri-mesh model) and after the remeshing and UV mapping step with the proposed method.
The authors have also worked on an alternative solution (Method 2, see the next section) for those projects in which only the 3D model with a texture file is available, without the opportunity to access the set of oriented images.If the discrepancy of the new quad mesh is sufficiently small, a new UV map can be generated using one of the algorithms developed and available in several packages.Figure 8 shows the distortion map generated by different procedures: angle-based and conformal mapping (Lévy et al., 2002;Sheffer et al., 2005) (in Blender), and the UV master plugin available in ZBrush.
Distortions are also visually quantified to identify stretched or compressed areas, using the same color scale for different results (Figure 8, stretched areas in blue, compressed areas in red).In this case, UV master provided the best results with smaller distortion.Overall.Such result is not limited to this example, the distortion map with UV master was always smaller than the

METHOD 2: VERTEX-BASED TEXTURE MAPPING
The second procedure is directly carried out without reusing the photogrammetric software for texture mapping at the end of the workflow.Photogrammetry is used only at the beginning of processing for the creation of a textured mesh.The refinement of edges is still necessary and must be carried out without breaking the UVs associated with the model.In general, this second method can be applied also to existing 3D models and all those cases in which the set of oriented images is no longer available.The method is still valid also for generic 3D models not generated with photogrammetry (e.g., those from manual modeling).
The example used in this section is a semicircular vault with frescoes (Sanctuary of Sassella, Sondrio).Figure 9 shows the result after photogrammetric processing (the fragmented texture map is shown on top -A).The results after applying method 2 are shown in the same figure (Bbottom).Method 2 is rather different if compared to Method 1.It is based on the direct application of color to the faces of the model.This also enables direct editing with 3D painting software.The workflow also has more steps, as shown in Figure 11.The first step is the decimation of the number of triangles of the photogrammetric mesh while preserving the UVs, so that a new smaller model with the same texture map can be processed.Decimation is not always required unless the mesh comprises several million triangles (more than 20-30 million triangles in the proposed experiments), which can lead to a high CPU cost.We define this new model as a low-poly-tri-mesh, indicating that the mesh is made up of triangles.
Then, a copy of the low-poly-tri-mesh is generated and subdivided into progressively smaller triangles.The aim is to increase the number of triangles without altering the overall shape of the surface, obtaining a number of faces similar to the resolution of the image texture file associated with the model.The RGB texture information is then transferred directly to the faces of the new high-poly-tri-mesh, making it possible to interact with the color of the 3D model directly, neglecting the original texture file, which is no longer used in further data processing.The number of new faces must be comparable to the resolution of the texture file.
Figure 10.Workflow of the alternative procedure: method 2.
The low-poly-tri-mesh is then converted into a quad mesh with fewer elements.The method is particularly suitable for smooth architectural surfaces (e.g., vaults), notwithstanding good results in terms of metric accuracy were also obtained for more complex surfaces, such as bas-reliefs and decorations.Metric accuracy is also evaluated by comparing the new low-poly-quad-mesh and the original photogrammetric model (Figure 10).The error was -0.64 mm ± 1.17 mm (Figure 11).The vault has a width of some meters, so metric accuracy is sufficient, but very small details are lost.A new texture map is then generated from the colored quad mesh.Different strategies are tested to minimize distortions, which are also evaluated by creating distortion maps, which just depend on the quad mesh and can be checked without creating the final texture file (Figure 12).A regular rainbow pattern projected on the model also allows the user to visualize the geometric distortion of different UV mapping algorithms.The next step involves progressively dividing the low-poly-quadmesh and creating a new high-poly-quad-mesh with a number of faces comparable to the triangles of the high-poly-tri-mesh.The color is then projected from the high-poly-tri-mesh to the highpoly-quad-mesh, and a new texture map is created using the UV from the low-poly-quad-mesh.

Conformal
UV master The main limitation of this second method is the need for an intermediate model with an increased number of faces so that the texture can be projected directly on the 3D model.Although this model can provide direct access to 3D painting methods, the intermediate mesh could require a very high number of faces for a high-resolution texture file, leading to a high processing cost.

ADDITIONAL REFINEMENTS
We consider some additional steps that could help refine the results.These refinements come at an extra cost and should be considered depending on the characteristics of the object and the requirements of the application.

Splitting the
In the case of complex 3D models, the definition of cuts is a crucial step.Figure 13 shows an example of a 360° object (a capital in Villa Quadrio, Sondrio) for which a cut in the texture is inevitable.The model was therefore cleaned and split into two separated sub-models, and the unfragmented texture map was arranged for the two sub-models into a single image.As can be seen, the texture map's occupancy (the percentage of image area with a usable image) is very high, much higher than the direct use of a single model for the entire capital (considering four consecutive faces).The disadvantage is an increased number of seams.
Figure 13.The remeshed model and the final unfragmented texture map after model splitting (2 parts).

Remeshing with a higher number of subdivisions
The remeshing step usually provides a smoother surface, in which small details are lost.Shown in Figure 14A is the original triangular mesh of 715k faces used to describe method 1. Figure 14B shows the quad mesh used to generate the texture, which is made up of 14k faces.Figure 14C shows remeshing with more faces (94k), which can preserve more details.
Figure 14.A: original photogrammetric tri-mesh (712k triangles), B: the quad mesh used in the example presented in Section 2 (14k faces), (C) the model remeshed with a higher number of target faces (94k), (D) the same model with the original tri-mesh "projected".
Lastly, Figure 15D shows the quad mesh in C with "projected" details from the original mesh, obtaining a new quad mesh that is locally deformed to capture irregular detail.This last option is very interesting because it recovers several small details without requiring the generation of a quad mesh with a similar number of faces to the original triangular mesh.

Level of automation and manual corrections
Figure 15 shows a ceiling dome in the Islamic Arts Museum, Malesia.The model was retrieved from Sketchfab (https://skfb.ly/QWzS)under CC attribution.Therefore, it was processed with Method 2 because the original images were unavailable.
Figure 15.A complex undevelopable surface was flattened with the proposed approach in a (almost) fully automated way.
After refining the photogrammetric model, the flat representation was generated fully automatically, obtaining a visualization of the distortion map.However, some additional red areas (very distorted) are also visible close to the edges, especially in the bottom-left corner.They are very localized and small, and they would suggest that the mesh would require additional (manual) editing.The overall consideration related to this example is the good visual quality of the flattened map, especially considering the high complexity of the surface.

Working with irregular edges
The example in Figure 16 (the vault in the Church of St. George, Annone di Brianza, Italy) shows the problems when edges are not refined.The original triangular mesh was rapidly cut to isolate only the area with the vault.Cutting small triangles results in very jagged mesh edges, which are also reflected in more irregular quads during the remeshing phase.This also leads to a wrong texture, with some white areas along the edges.The visual effect is quite limited, but the overall suggestion is always to refine the edges of the photogrammetric mesh and turn them into a more regular boundary.

ADDITIONAL APPLICATION: DIRECT 3D PAINTING USING TABLETS
The remeshed model and its new texture can also be used in the latest version of Procreate®, which allows users to paint on an iPad.The model from photogrammetry was not correctly imported, especially the texture.
Once inside Procreate, a new layer can be added and the user can digitally paint on the 3D model.This is very useful during the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-2/W4-2024 10th Intl.Workshop 3D-ARCH "3D Virtual Reconstruction and Visualization of Complex Architectures", 21-23 February 2024, Siena, Italy fieldwork because the user can access complex 3D models and interact with them using digital painting operations.

CONCLUSIONS
The paper has presented two methods to flatten photogrammetric 3D models and create 2D texture maps.The visual quality of the resulting texture file (i.e., digital image file) allows the user to perform traditional operations, which are usually carried out with orthophotos or rectified images, such as condition mapping.The results can then be reprojected on the 3D model to obtain real measurements for complex and irregular surfaces.In other words, the method extends the use of traditional 2D representation to complex surfaces where orthophoto would provide distorted results.At the same time, the 3D model is not required in the mapping phase, so users not very expert in 3D modeling can use the resulting 2D map.
As mentioned, the paper has presented two methods.The choice of the method depends on the requirements of the application.Method 1 is a more generic solution for all those operations requiring a flat texture map, especially when a high-resolution texture will be generated from a set of oriented images, which must be available.Method 2 is more flexible since it allows the user to simultaneously modify the 3D model and its texture while creating the final texture file.This second method can also be used for textured models generated without digital photogrammetry or when the original images are unavailable.However, the second method is computationally more intensive because of the need for a very high-resolution quad mesh during the projection of the original texture on the faces of the model.

Figure 1 .
Figure 1.Original tri-mesh with the texture of the umbrella vault.The fragmented texture file is shown at the bottom.

Figure 2 .
Figure2.The new model with its new texture file (bottom).This is not an orthomosaic, it is an unrolled representation of the surface and the file can be used in 2D editing software.

Figure 3 .
Figure 3.The distortion pattern of the flat map was obtained with the proposed method for automatic flattening.Bottom: visualization of compressed (blue) and stretched areas (red).

Figure 4 .
Figure 4.The initial photogrammetric (triangular mesh) and its fragmented texture file (A), and the re-meshed model with an unfragmented texture (B).

▼Figure 6 .
Figure 6.The original photogrammetric tri-mesh (top) converted into a new quad mesh (bottom).

Figure 7 .
Figure 7.Comparison between initial (triangular) and final (quad) meshes (units are given in mm).

Figure 8 .
Figure 8. UV creation with different methods and distortion visualization indicating stretched (blue) and compressed areas (red).The same colorscale was used for the figures.The last method (UV master) seems to give better results regarding distortion.This result was not only achieved in this example but also in the other examples shown in the paper.

Figure 9 .
Figure 9.The initial photogrammetric (triangular mesh) and its fragmented texture file (A), and the re-meshed model with an unfragmented texture (B).

Figure 11 .
Figure 11.Comparison between the original triangular mesh and the remeshed model (in millimeters).

Figure 12 .
Figure 12.Comparison between different methods for UV mapping applied to the remeshed model.The distortion maps (on the left) have the same colorscale.

Figure 16 .
Figure 16.(A) The original model and texture from the photogrammetric model.(B) The final quad mesh and its unfragmented texture map after remeshing, model splitting, and UV mapping (without edge refinement).(C) Example of unedited edges.

Figure 17 (
top) shows the umbrella vault presented in Figures 1-3 and the mapping of cracks with a pencil.The user can manually rotate the model with the fingers.The new image layer can then be exported and still follows the same UV structure of the color texture map.The figure (bottom) also shows the layer structure with a flat texture and a new layer for tracing cracks.The actual version of the app (tested in December 2023) has a maximum number of two levels, which is not enough.The authors would expect an extension of the level number in further app updates.

Figure 17 .
Figure 17.The remeshed model and the unfragmented texture file were used in Procreate for condition mapping during on-site work, enabling the operator to map cracks directly on the 3D model and obtain a 2D flat representation as an extra image layer.