PHOTOGRAMMETRIC MODEL OPTIMIZATION IN DIGITALIZATION OF ARCHITECTURAL HERITAGE: YEDIKULE FORTRESS

: The idea of "digitalization of architectural heritage" has recently gained prominence to represent architectural and historical assets. With all these potentials, this study aims to create optimized models that can be used in serious gaming environments by presenting a method of photogrammetry. As a case study, Yedikule Fortress and its surroundings, which have a multi - layered structure that includes many cultural aspects such as Byzantine, Ottoman, and Republican periods in the historical process, have been studied within the scope of digitizing the architectural heritage to create an optimized model for gaming environments. The study was methodologically constructed in three phases: Photogrammetry, polygon modeling, and low poly/high poly baking process. The fortress and its surroundings are modeled using a high - detail point cloud and a high - poly mesh using aerial photogrammetry. The high - poly model was taken as a reference and transferred into a low - poly model as a mesh map, texture, and light characteristics. This allowed the high poly model to operate more efficiently and effectively in game engines. As a result, the study created a detailed and optimized model for the game engines to produce serious games specific to light and texture data, to be used on devices that support mixed reality (MR) technologies.


INTRODUCTION
Digitalization of architectural heritage is one of the leading areas of increasing importance in conserving historical heritage in recent years. By increasing research in this area, people can better understand their historical heritage and become more aware. The studies to be carried out in gamification, which plays an important role in the awareness /teaching of historical areas, will also guide this study. This paper, the first stage of a research project on digital heritage experience by gamification, defines an ideal method to create a photogrammetric optimized model to be implemented in Istanbul Yedikule fortifications.
Yedikule Fortress has hosted many functions in the past, such as the entrance gate of the city of Istanbul, defense walls, barley warehouse, prison, art house, and museum in the historical timeline. It has become important to document these historical values, which are important in transferring and reflecting cultural heritage in the digital environment and increasing their readability with developing technologies. This project has been undertaken in a graduate course called "Digitalization in Architectural Heritage in ITU (Istanbul Technical University) and also supported by a Tubitak (Turkish Scientific Research Institute) project. This study explains the transfer of cultural heritage to the digital environment., considered within the project's scope. Therefore, the method and the optimization process are explained further.
Within the Yedikule Historic District, the walls are documented using drone photography. After that, the photographs are uploaded to Agisoft Metashape software to transfer the threedimensional solid model and texture. Blender and Adobe Substance 3D Painter are used for the optimization phase of the high polygon solid model obtained from photogrammetry. In the digitalization of cultural heritage and gamification, it makes a novel contribution in terms of experiencing AR-MR environments and comparing them in terms of spatial perception.

Literature Review on Photogrammetric Modeling and Gaming
Serious games and gamification have been frequently studied within the cultural heritage field for the last decade. In this field, studies on tangible heritage, such as 3D reconstruction-based studies, are widely used photogrammetric methods. Serious games are primarily designed to help players learn through experiences. They offer an enjoyable way for users to achieve learning objectives, including familiarity with cultural heritage topics and increasing interest and engagement with them (Mariotti, 2020). The use of game-based learning, a component of serious games, is increasingly being studied in various subjects and for all levels of education (Vaz de Carvalho et al., 2013).
In Rome, a research project developed 3D reconstruction methods to create immersive VR cultural applications and serious games to present the historical background of the Forum of Augustus. Since the work was created to design a serious game in the VR environment, experts from various disciplines worked. It is stated that historical architecture and user navigation areas are governed by rules that constrain unlimited imagination and scenarios (Ferdani et al., 2020). Another study mentions modeling of Girifalco medieval castle covered with vegetation with UAV photogrammetry. Even though filtering was applied, gaps occurred in the model, incredibly close to the ground in areas with dense vegetation. Researchers have stated that UAV photogrammetry has certain limits and have suggested the usefulness of properly integrating UAV photogrammetry with terrestrial surveys (Masiero et al., 2019). Similar work was studied at the Schwartzenbourg castle ruins, a medieval castle overgrown with vegetation, to create a 3D model of the castle before the remains posed a danger to humans. The study employed terrestrial laser scanning (TLS) surveys and additional aerial photogrammetry surveys. The research stated that laser scanning and photogrammetry complement each other perfectly (Koehl et al., 2017). Kontogianni et al. (2017) have introduced a composite Serious Game that showcases the Stoa of Attalos, a significant structure in the Ancient Athens Agora. The game incorporates 3D models generated mainly through automated image-based modeling techniques. Another image-based modeling study uses aerial photogrammetry by drone and terrestrial photogrammetry with digital camera methods to document cultural heritage sites in the Chellah Archaeological Site of Morocco (Simou et al., 2022).
Fernández-Palacios points out that a visualization pipeline was created to facilitate the utilization and access of vast and intricate 3D heritage content through virtual Reality (VR) techniques. In this study, data were collected using laser scanning and photogrammetry together, ensuring the accessibility of normally inaccessible digital archaeological sites (Fernández-Palacios et al., 2017). In another study, structure from motion (SfM) photogrammetry was used to visualize the archaeological site of Bagan, located in the center of Myanmar, in a virtual reality (VR) environment. The optimized model the researchers created, combined with mesh maps (texture maps, normal maps, etc.), looks like the original high-detail model (Dhanda et al., 2019). This optimization also aligns with our study and shows that it is possible to optimize our study for environments such as VR, AR (Augmented Reality), and MR (Mixed Reality).
Although Turkey has a rich cultural heritage, studies have yet to be done on digitalization and the game. One of the studies in Turkey, The Digital Teos project, was carried out in the ancient city of Teos in Turkey. Within the project's scope, the researchers revived the architectural structures in Teos with solid modeling and photogrammetric modeling. These two modeling methods were compared to transferring historical and cultural information and establishing the accuracy of their relations with the context. It is stated that the UAV photogrammetry used is less costly, shorter, and easier to obtain than laser scanning (Varinlioglu, 2020). In visualization, the material and texture data obtained from the photogrammetric model on the plain 3d models were transferred with UV mapping.
Another project in Turkey was realized in Çatalhöyük, which is on the UNESCO world heritage list. In the study titled "3D-Digging at Çatalhöyük" within the project's scope, comprehensive research was carried out to create a digital copy of the archaeological excavation process using techniques such as laser scanning, photogrammetry, computer vision, and image modeling. Integrating different 3D data recording and visualization technologies into fieldwork and implementing them in virtual reality systems is challenging (Forte et al., 2012). "Dig@IT," another application studied at Çatalhöyük, is a tool that operates on multiple platforms and provides an entirely immersive, 3D experiences for visualizing, interpreting, and curating archaeological data (Lercari et al., 2018). The studies carried out are inspiring in terms of transferring and experiencing the archaeological excavation process in the virtual environment. Using different technologies in the data collection phase helped define a method to create a game environment.
As a review, some studies show that the models created by photogrammetry are gamified by direct optimization. In contrast, in some studies, it is aimed only at documentation or display in different environments. At the same time, it is seen that methods such as laser scanning are used in addition to photogrammetry.

METHODOLOGY AND CASE STUDY
Creating optimized models suitable for the game environment is an important issue that increases the game's fluency, functionality, and entertainment. At this stage of the study, the methods, processes, and tools used in creating optimized models for the representation of Yedikule Fortress, one of the cultural assets within the borders of Turkey's Istanbul province, in the serious game environment are expressed.
Certain strategies have been developed depending on the limitations of the tools used in the study. In line with these strategies, the study, Yedikule Fortress, was discussed in three parts in aerial photography, and a photographing strategy was defined to guide the drone movements ( Figure 2). With the photogrammetry application, chunks were created to reduce the application's processor load in data transfer to the digital environment. These chunks were used as a base in the photo shooting strategy, and then the whole model of Yedikule Fortress was obtained. The process of creating an optimized model, discussed in detail in the continuation of the study, is explained through Part A of the strategic plan, which is discussed in three parts ( Figure 1). All the processes discussed in the optimization process were applied to all specified parts, and the final model was created.
The optimization process is methodologically addressed in three main stages, namely "Transferring Data to Digital Environment with Aerial Photogrammetry, The Baking Phase of Mesh Optimization, The Baking Phase of Mesh Maps and Diffuse Maps." (Figure 2).

Figure 1.
(1) 3 separate parts will be photographed and modeled separately; (2) Each part will be photographed every 2 m. looking horizontally to the wall, (3) Each part will be photographed every 2 m. looking vertically down; (4) Photographed looking down (h=20 m.) every 10 m.

Figure 2.
Stages of the game-ready model making.

Transferring Historical Data to Digital with Aerial Photogrammetry
Photogrammetry application is a system that creates solid models of scanned objects using point clouds of photographs obtained by providing optimum environments, such as the correct light and ideal weather conditions for the element to be digitized. Terrestrial photogrammetry application, commonly used for transferring smaller-size objects to digital media, cannot be used effectively in large-sized buildings discussed in the study. The aerial photogrammetry method was used in the study depending on the limitations of terrestrial photogrammetry. The data obtained from aerial photography of Yedikule Fortress were handled in the Agisoft Metashape photogrammetry application. In the photogrammetry application, the data were handled in 7 stages. These are; import photos, mask photos, align photos, density cloud, build mesh, build texture, export mesh, and diffuse map.
Fatih Municipality, who supported the project, took part in drone photography. DJI Mavic 2 Enterprise Dual model drone was used for aerial photography. The processing of the EXIF (exchangeable image file) data from the location data on the photographs of the vehicle used in the drone shootings has an important place in the alignment of the photographs in the study. In order to ensure that the light and shadows on the shooting surfaces are in the same tone, the shots were taken in cloudy weather at noon. Exposure intervals have been determined to overlap by 60% to align the photographs in the photogrammetry application without any problems. In the following parts of the study, the stages of the Agisoft Metashape program in creating the digital model of Yedikule Fortress are elaborated.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy In transferring Yedikule Fortress to digital, 872 photos collected from drone shots were first transferred to the Agisoft Metashape photogrammetry software. As mentioned above, Part A of Yedikule Fortress was strategically divided into three parts to express the optimized model creation process. The application places the images according to their angles by using the location information added to the photos while shooting in line with the algorithm it contains (Figure 3). In the photographs transferred to the photogrammetry application, many structural and urban elements can be seen outside the Yedikule Fortress and its surroundings, which will be modeled. This situation causes confusion and pollution in the point clouds that will occur in the later stages of the photogrammetry process. In order to prevent this, each photograph discussed in the study was masked with the masking method ( Figure 4). Align photos operation is required in associating the photos transferred to the photogrammetry application. The alignment process was applied to associate the photos transferred to the photogrammetry application with each other, and the first point clouds were created with the aligned photos ( Figure 5a).
The alignment step obtains a low-density point cloud from the photographs. A dense cloud stage is used to increase this point cloud, which may be insufficient to create a solid model. After the low-density point cloud was created, it was turned into a higher-density point cloud by increasing the junction points. The level of detail of the solid model to be made with this process is increased (Figure 5b). After the mesh model with a high polygon count, the model's texture was finally created. The created texture (diffuse map) was examined with the UV Unwrap Map and prepared for the next optimization step (Figure 6b). At the last stage, the high polygon mesh model and diffuse map data of the A Part of Yedikule Fortress, which were transferred to the digital environment with the Agisoft Metashape software, were exported. The resulting models and texture data were exported in .fbx format suitable for the Blender 3D program to be used for mesh optimization.
The aerial photogrammetry method used in digitalizing Yedikule Fortress and its surroundings has created a model with a high polygon count, not optimized for a serious game environment. In the next stages of the study, the processes performed in the "The Baking Phase of Mesh Optimization, The Baking Phase of Mesh Maps and Diffuse Maps'' of the nonoptimized model and the production stages of the optimized models suitable for the serious game environment are explained.

The Baking Phase of Mesh Optimization
The mesh model created by the photogrammetry method has very high detail and surface count. Therefore, it needs to be optimized more for use in game engines. The topology data of the model created with high-poly photogrammetry should be transferred to a low-poly model. It is a crucial step in the 3D modeling process since high-poly models typically are not good for animation and contain too many polygons, which would overload the game engine's processing capability (Kuusela, 2022). The low-poly model created should be overlapped with the high-poly model as both faces (Figure 7). The model with a low polygon number was prepared in the Blender3D program regarding the high-poly model. In Blender3D software, the photogrammetry (high-poly) model in .fbx format was first transferred with the "import" command. At this stage, the Photogrammetry model was used as a reference for low-poly modeling. The low-poly model was prepared in modules for Yedikule Fortress units (towers, walls, golden gate).
We can explain the main reasons for using this method in two articles; -Preserving the resolution value of the Diffuse map, which creates the surface color of low-poly models.
-Increasing the level of flexibility for serious games that can be made.
During the preparation of the low-poly model, quad surfaces suitable for the surface geometry of the high-poly model were created. Then the shrinkwrap command under the modify tab was applied. The shrinkwrap command ensured that the lowpoly model we prepared was coated on the high-poly model in a way that would be flush. This way, the low-poly model was prepared to suit the UV mapping and baking stages.
A UV map of the 3D model is required before the texturing of the 3D object can start. A 2D representation of a 3D model's surface is called a UV map, and it is considered the ideal wrapping paper for a 3D object. A 2D texture may be applied to a 3D object by wrapping it in a UV map, which functions as a 2D representation of the 3D item (Denham, 2022). The UV map of the high polygon number model created by photogrammetry is scattered and irregular (Figure 8a). This situation also prevents any manipulation process that should be done on the model texture. The UV map of the low polygon number model is opened following the geometry form. In this way, scattered texture maps can be brought together in an orderly position (Figure 8b). The low-poly model prepared in Blender3D was mapped using the "Smart UV Project" command. The margin value between the UV map's surface islands is 0.01 (corresponding to 16 pixels for a 2048*2048 resolution texture). In this way, the color values of different UV islands do not mix.

The Baking Phase of Mesh and Diffuse Maps
After these stages, first of all, the diffuse map, which contains the color data of the high poly model, is baked with the Blender3D software. Baking transfers texture information as a bitmap from one 3D model to another 3D model. Generally, this process is applied from the high-poly model to the low-poly model.
The low poly mesh can look just like the high poly mesh, thanks to the mesh mappings. The texture (or albedo diffuse map), the normal map, the ambient occlusion map, the roughness map, and the metallic map are the five basic mesh maps used in PBR. The mesh's complete color information is contained in the texture map. Surface normal directions are encoded using RGB values in normal maps. When a normal map is baked from a high-poly model to a low-poly one, the high-poly model's detail is simulated by giving the low-poly model the appearance of depth when lighted. When things are lit by diffusely, ambient occlusion simulates the deep shadows of such objects. The roughness map identifies the surface imperfections that lead to diffuse light scattering. Finally, the metallic map indicates which portion of the substance is made of metal (Dhanda et al., 2019).
Mesh maps need to be baked after baking the Diffuse map. Adobe Substance 3D Painter program was used for the baking process of mesh maps.
In the Substance 3D Painter program, the low-poly model prepared in the Blender3D program and exported in .fbx format was first opened. The diffuse map prepared in Blender3D with the "Import resources" command under the "File" menu is transferred into the program and overlaid on the low-poly model.

CONCLUSION
The study's main goal was to propose a workflow and point out the important aspects of the application in a case study of the Yedikule Fortress. One of the critical starting points was that serious games reach more people, and the performance of game engines on various hardware is an important parameter. However, photogrammetry presented in this study has demonstrated the effectiveness of modeling and the combined use of other techniques to promote architectural heritage in the digital environment. This methodology also allows the experience of the building in different aspects belonging to the digital environments or even restoring/rebuilding in the future, even if it is demolished after any disaster, in terms of preservation of architectural heritage.
Furthermore, this study contributes to the digital process of 3D models for other fields rather than architecture. For instance, it preserves the data of large-sized artifacts. Also, bringing heritage assets to the game industry without a performance loss during the game opens up new possibilities for the serious gaming industry to widen its audience.
In terms of its contribution to the literature, this study aims to show this potential to create an architectural heritage model ready for a serious gaming environment by adding a new optimization-oriented study in addition to other studies in the literature.
In this process, there were four main parts to make the model usable in the game environment: -data collection, -the creation of the model, -optimization of the model, -the texture of the model.
The quantitative results of these processes are explained as follows. In the whole model created with photogrammetry, approximately 12 million polygon models were created. The optimized model consists of 154 thousand polygons. The file size is 452 MB in the non-optimized model and 7 MB in the optimized model. There is a 78-times polygon difference, and the size difference is 65 times.
In the Blender viewport environment, the whole model of Yedikule Fortress is running at 25 FPS ( Figure 10a) as high poly due to a non-optimized photogrammetry application. In the same hardware and software conditions, the same model performs up to 200 FPS after being optimized, as described in the methodology section ( Figure 10b). As observed above, no discernible change in the architectural heritage data was perceived by the serious game player, even at close sight. On the other hand, as we can observe with the FPS parameter, there is an increase in 3D navigation and in-game performance. Representing the architectural heritage in a serious The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy gaming environment will not yield successful results because 25 FPS does not provide a smooth experience to the players. However, 200 FPS obtained with this method in Yedikule Fortress will provide conditions that will contribute to the protection and promotion of the architectural heritage by providing a smooth game performance regardless of the data size and scale of the architectural heritage.
However, the validity of this situation has the potential to change with the "Nanite" technology that has recently come with the Unreal Engine 5 game engine. In this technology, components such as the number of polygons of the model do not affect the game performance. Since this technology has yet to become widespread in the field and there needs to be more academic study, it has been excluded from the scope of this article and is expressed as a foresight.
Another parameter affecting the model's performance that this study focuses on is considering the architectural heritage as modular. The flight/scan path determined during the drone scan was designed by dividing the fortress into modules. These modules saved time and performance in organizing photos and mesh creation stages while allowing the team to work on different modules. Separating and combining the whole model into modules has also provided the flexibility to use modules in the game by separating and combining them for different digital environment needs (Figure 1).
The results of the model creation process laid out several methods to collect, analyze and process the heritage data in the digital environment to achieve interoperability among different digital environments such as Extended Reality (XR). XR is an emerging technology used in architecture and heritage preservation, gaining popularity recently. The study counts significantly increase in these fields, applying spatial mapping and data visualization. One of the most important advantages of the process was that the study focused on creating highperformance models for any digital environment, even for the most performance-sensitive devices like mobile devices are the most widespread devices in the world (Figure 11). With all these potentials, this study proposed an optimized model that can be used in serious gaming environments by presenting a photogrammetry method in combination with optimization. More effective solutions can be developed by continuing this study in the future. The professionals in the field can improve their understanding of architectural heritage in different aspects or discover new layers of information. However, these findings are also important for attracting more people outside the field of study.