PROCESS OF CREATING AND COMPARING 4D MODELS OF RHENISH CASTLES IN DIFFERENT MODELING ENVIRONMENTS

: Alsace is renowned for its rich castle heritage. With more than 500 listed castles, it is one of Europe's most densely fortified areas (Châteaux forts Alsace, 2013). Some of them date back to the 11th century, and most of them are in ruins. It was against this backdrop that these studies were carried out as part of the Châteaux Rhénans - Burgen am Oberrhein project, funded by the European Interreg VI programme. More specifically, within the animation and promotion strands carried out by INSA Strasbourg and Tourismus Südliche Weinstraße. These actions focus on the digitisation and digital promotion of the Rhine castral heritage. Two projects are described and compared in this paper. The Oedenbourg castle is a little-known site. Tucked away behind its famous neighbour (Haut-Koenigsbourg castle, Orschwiller, Alsace, France), this ruined remains from the 15th century houses a Gothic dwelling dating from the 13th century. The Oedenbourg, with its architectural elements from different historical periods, was now the subject of new excavations to shed light on its history. The ruins of Wasenbourg castle, which dominates the commune of Niederbronn-les-Bains in Alsace (France), are the second site to be studied. Historians agree that it was built in the 13th century, but the precise date is around 1272-1275. In 1592, the castle is mentioned as being in ruins. This paper presents the different processes undertaken for the digitisation, 3D modelling and 4D restitution of the two sites, using two different modellers, namely Autodesk Maya for the Oedenbourg and Blender for the Wasenbourg.


INTRODUCTION 1.1 The Oedenbourg ruins
The Oedenbourg castle is a little-known ruin.It is located in Orschwiller (Bas-Rhin, France), just a few hundred metres from its famous neighbour, Haut-Koenigsbourg castle.Listed as a historic monument in 1993, it was transferred (as part of the Haut-Koenigsbourg domain) to the Bas-Rhin departmental council in 2007.The Oedenbourg dates to before the 15th century, when it was already mentioned as an Oden burg, meaning "abandoned castle".Travelling back more than five centuries, these ruins (Figure 1) describe a 13th-century Gothic dwelling.The Oedenbourg site has been the subject of architectural and archaeological work on numerous occasions.In the early 20th century, it was restored by Bodo Ebhardt at the same time as Haut-Koenigsbourg castle.Several elements of the dwelling, such as the large Gothic Bay window, were restored.Between 1991 and 2001, the site was excavated by several teams of archaeologists and architects, uncovering several hypotheses about the site's history.In the summer of 2023, the first part of a new three-year programme of archaeological excavations was opened.Conducted as part of the Interreg VI project by Jacky Koch, archaeologist, and head of historical periods at Archéologie Alsace, these excavations have already produced results, with the discovery of new elements.These investigations and hypotheses therefore point to three main historical phases, the last of which is still visible today in the form of the ruins of a Gothic dwelling.Preceded by two historical phases, this dwelling is merely a reminder of what the Oedenbourg site may have been like in the 13th century.The site is thought to have been a fortified enclosure dating from around the 10th-11th centuries.The purpose of this early phase (here phase A) remains unknown, as it consisted of a series of perimeter walls running from west to east along the mountain ridge and protecting a two-storey platform.At the beginning of the 13th century, the fortified enclosure was enhanced by a master tower, which served as a high point and as a demonstration of strength over the neighbouring valleys.It was not until one or two generations later that the dwelling, still visible today, was built.

The Wasenbourg ruins
The Wasenbourg castle was built in the 13th century on a site occupied since ancient times, on the Reisberg hill at an altitude of 432 metres.It overlooks the town of Niederbronn-les-bains The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-2/W4-2024 10th Intl.Workshop 3D-ARCH "3D Virtual Reconstruction and Visualization of Complex Architectures", 21-23 February 2024, Siena, Italy (Alsace, France).Its dwelling and courtyard were protected by a large shield wall (3.70 m thick, 22 m high), which may have replaced the tower and included an internal staircase, making it a defensive fortress closed off from the outside world.Salesse (2018) offers a monograph that, among other things, provided a historical chronology of Wasenbourg Castle, based on the many references dedicated to the subject.However, many historical events are not guaranteed and remain hypothetical, due to a lack of precise information.The castle is mentioned as being in ruins in 1592 (Figure 2).

OBJECTIVES OF THE STUDIES
Initially, the studies will involve surveying all the Oedenbourg and Wasenbourg sites using topographic and photogrammetric methods, as well as processing the data.The final deliverables should be 3D models of the castles.These models must be usable for the project's next actions, which will involve thinking about 4D reconstruction methods for the 3D models produced, i.e. reconstructions for all the periods in which they were inhabited.This article will follow the progress made during these projects to respond to the issues raised.It will begin by outlining the state of knowledge surrounding the subject of this project, tackling questions concerning the interest in and means of digitally safeguarding heritage, as well as the enhancement of the historical reconstruction work on the archaeological base.The acquisition of data in the field and its processing to make it usable for 3D modelling of sites in their current state will then be explained.The restitution of past states will be based on archaeological data and hypotheses, as well as modelling and texturing methods using Autodesk Maya compared with Blender to draw out interesting perspectives for future projects of the same kind.

Digital heritage preservation
The integration of digital technology in heritage preservation is becoming a matter of course.In terms of both preservation and dissemination, digitised heritage itself becomes a historical relic, freezing the state in its own time.Indeed, while an original relic deteriorates over time, a digital twin that is correctly stored and created does not lose quality.The emblematic example is the cathedral of Notre-Dame de Paris, which was surveyed before it burnt down and on which numerous restoration experiments are being carried out (Arte, 2023).Moreover, processing a digital copy would cost 60% less than processing the original artefact or site (Belhi et al., 2017).Digital technology therefore makes perfect sense in this field, enabling computerised backup while respecting the physical integrity of the site (Shao et al., 2019;Tejedor et al., 2022).However, digital technology is subject to obsolescence over time.In other words, as technologies evolve, data will probably, if not undoubtedly, become obsolete.Firstly, in terms of hardware, as in the case of cassettes and CD-ROMs, then in terms of software for reading and using the data, and finally in terms of ownership of the data, thereby restricting access to it.This notion of obsolescence can already be felt with historical archive data, which is sometimes difficult to interpret and therefore requires meticulous exploitation.There are many types of historical data, which Stefani (2010) classifies into two categories: metric data and visual data.Metric data enables quantitative analysis by describing mathematical quantities, while visual data enables qualitative analysis of the site, providing information about its appearance.Combining these two types of data produces more complete and exhaustive results.This is also known as iconographic data.Historical/archival data is highly uncertain, as it is sometimes contradictory, incomplete, or heterogeneous.In addition, the complexity of highlighting the result that could be extracted from a document in relation to its type of iconography needs to be considered.It is therefore on the iconographic research phase that the purpose of a historical reconstruction partly rests.Using the multitude of historical data described above, it is then possible to reconstruct the previous state of a site, enabling it to be preserved and made accessible to as many people as possible.Archaeological reconstruction is a set of interpretations and hypotheses based on historical documents, testimonies and remains.From 'conservationist' ideals to those of 'total reconstruction', several models of restitution have emerged, both physical (tangible) and digital.Dell 'Unto et al (2013) propose the following models of restitution: -The "restitution of the monument in situ", which favours the reconstruction of the monument on its original site using the same techniques and materials.
-The "restitution of volumes" based on the remains and without any desire to remain faithful to the original state of the site.
-The "transfer model", which consists of reproducing/moving a site within a special structure.
-The "conservation model", which aims to crystallise the monument to prevent any further deterioration.
-The "virtual model", which represents a site in its past, present or future state in digital form.

Acquisition techniques
According to Stefani (2010), the 3D model serves as a "descriptor of physical and metric quality", a "tool for extracting measurements" and a "documentary source".Indeed, 3D, for three dimensions, allows the expression of volumes, shapes, and textures.Through models, virtual or augmented reality, 3D becomes a versatile tool, representing a support for scientific study, as well as a support for tourism (Ferrari and Medici, 2017).The acquisition of raw data is the first stage in the process of reconstructing a building/site ("as captured").The major problem with acquisitions is respect for the site, which is often fragile and damaged.
In the field of heritage, acquisitions are often multi-sensor.Combining several types of data means that the weaknesses of some techniques can be overcome by the strengths of others.In the heritage field, scanned and photographed data will often be combined to produce geometrically and aesthetically exhaustive results.While methods based on laser scanning provide accurate, highdensity results that capture geometric detail, photogrammetry and photography will produce better-quality texture data (Rodríguez Gonzálvez et al., 2017;Guidi et al., 2009), which is also referred to as multi-resolution data, i.e. the coupling of data at different scales to cover different objectives and provide targeted renderings of a problem.

Recording and recreating history: 3D modelling of a heritage site
Virtual reconstruction of a historic site can involve modelling a 3D model.Based on data (iconographic or acquired), the model can be of different types: meshed, based on geometric primitives and others.It needs to be optimised to make it easier to handle and exchange, and then evaluated historically.Depending on its purpose, the 3D model may also be textured.In addition to the method used to model the site in 3D, the type of 3D representation also plays an important role in the viewer's understanding and immersion.This type of representation depends on the purpose of the model, the audience to be reached and the data available.Stefani (2010) lists: -A rendering that considers the current appearance of the site, recreating the ancient site in its current environment, which helps the viewer to become immersed in the model.
-A rendering that describes the topographical context, past and present.Inserting the model simultaneously into its past and present reality would then allow conclusions to be drawn about the tenacity of certain hypotheses.
-The morphological approach based on what already exists.By deduction and comparison, and by relying on historical documentation, this approach enables volumes to be reconstructed.

Optimising a 3D asset model
Optimising a 3D model is essential to its overall smooth operation.It makes for smoother handling/rendering and intersoftware exchanges.Surand (2019) offers several tips for optimising a digital model: -The polygon budget (poly count) is the number of polygons set for a 3D model.The more polygons a model contains, the heavier it is and therefore the slower the processing/rendering calculations will be.One trick is therefore to reduce the number of polygons for objects that are far from the camera.Following on from the previous point, it is possible to work in "Low poly".A "Low poly" version of a model is obtained by decimating the number of polygons in the "High poly" model (a highly detailed version).
-Scene optimisation can also involve optimising texture files, by limiting the number of images, using texture atlases, and varying the quality of texture images according to the distance of the object from the viewer.

Evaluation of a 3D heritage model
In the field of engineering, it is essential to control one's results, particularly through statistical calculations of errors (Landes et al., 2019).These assessments are made based on totally objective metric data.However, in the context of heritage, the data to be evaluated does not necessarily meet the conditions set out above.The evaluation of heritage restitution is more concerned with the evaluation of reconstruction uncertainties.De Luca (2006), Landes et al (2019), Stefani (2010) and others all state that uncertainty arises from the strong notion of interpretation of historical data.In addition to its definition, uncertainty is often represented graphically, making it easier for everyone to understand.Uncertainty tends to be represented by an error bar, an error sphere, a colour code in the study by Skeels et al. (2010) or a representation code (non-photorealistic rendering), based on different colours and line styles depending on the uncertainty of the element represented (Strothotte et al., 1999).The notion of "Level of Uncertainty" (LoU) can also be used to evaluate an archaeological 3D model.These levels are characterised by their own definition and by a representation colour (Landes et al., 2019).

Texturing for realistic rendering
3D modelling approaches allow a so-called white model to be represented, i.e. without the application of textures or colours.To obtain a more realistic rendering, we would like to carry out texturing, which, according to Yuksel et al. (2019), consists of applying a 2D texture to a 3D surface.The principle is that the faces of the 3D model are unfolded to form a 2D surface, like a sewing pattern (unwrapping).We can deduce the u and v coordinates of the faces to apply the texture to them.This is why this method is generally called UV mapping, which has been widely used for several years.So, textures can be applied to 3D models using UV mapping.There are several categories of UV maps that can be applied simultaneously to a 3D model (Sanseverino et al., 2022): The "Diffuse map" corresponds to the main colours of the object, representing materials (e.g.wood, stone, etc.) or solid shades, containing in addition information on light reflection.The "Albedo map" corresponds to the raw colour but without lighting effects.It is possible to combine the "Albedo map" with the "Specular map", which provides a check on the amount of light reflected by the object.It is also possible to use the "Metallic map" to define the metallic appearance of an object, which also corresponds to light reflection management.These maps are generally in shades of grey.The "Normal map" is one of the most interesting maps to use, along with the "Diffuse map".It can be used to simulate relief by highlighting the normals for each fragment of the model, making it more realistic.It corresponds to the evolution of the "Bump map", which has the same function but represents the real height in relation to the surface of the 3D model according to the shades of grey of the pixels.The "Normal map" uses 3 colours (RGB) to represent the deformation in 3 different directions.It is similar in principle to the "Displacement map", but the latter modifies the model by adding polygons, whereas the "Normal map" only allows you to visually cheat on the relief.Some programs also use the "Height map", which stores information about the object's height.Although similar, it is used in conjunction with the "Normal map" to increase the level of detail at all viewing angles.The "Ambient Occlusion map" is a grey-scale representation of the shadows to be added to the model, particularly in concave areas.To be more precise, it represents the extent to which each point is exposed to ambient light.There are many other UV maps that can be used to alter certain texture parameters: "Roughness map", "Opacity map", "Refraction map", "Emissive map", etc.

Data acquisition for the creation of 3D models of the current situation
Both projects were based on the same techniques for acquiring terrain data.

4.1.1
The uav photogrammetric method in the form of vertical and oblique images were used to create the model processed in Agisoft Metashape.High-resolution images were also used to acquire textures.

Additional terrestrial laser scanning (Faro Focus X330
), including sphere georeferencing, was used to correct the models and close the gaps.The general processing chain is illustrated in Figure 3.As a result of the various tests carried out on different software packages, the final processing chain adopted for the rest of the project will be that illustrated in Figure 6.

4.2.1
The Oedenbourg processed Maya.All the renderings of the Oedenbourg were produced using Autodesk Maya modelling and rendering software.This software is one of the best known and most widely used in the animation and audiovisual industries.It offers a wide range of functions, from modelling to texturing and animation.Many additional tools can be added via add-ons.All the modelling was carried out using geometric primitives in a quad structure.As the historical phases of the site have almost all disappeared, we couldn't base our work solely on acquired and mesh data.In addition, the use of geometric primitives greatly reduces the weight of the final model and makes it easier to edit geometrically.First, it was important to look again at the topological organisation of basic primitives (cube, sphere, torus, cone, etc.).A primitive is made up of faces, vertices, and edges.Modifying a geometric primitive involves editing its three components.This may involve moving them in space, changing their size, adding, or removing them, etc. Understanding this concept is essential to understand the functionalities of Maya's various tools.It was also a step towards automating modelling by scripting in MEL or Python.Secondly, it was essential to structure the objects correctly in Maya, to optimise the modelling chain and make the work easier.
A final important point to emphasise in this modelling phase was respect for the topology of the objects created, which guides the behaviour of the modelling tools.4.2.1.1.Structure: The historical model of the Oedenbourg was structured into three classes: A (surrounding wall -10th-11th century), B (main tower -early 13th century), C (Gothic dwelling -late 13th century).To visualise the temporal phases, we used Maya's Layer function, which allows to make the elements of the layer visible or not, and editable or not.For example, to visualise phase B (comprising the primitive enclosure wall and the main tower), Layers A and B has to be activated.The groups were structured within each class (A, B or C).As our modelling was essentially architectural, the decompositions and groupings of objects were defined according to their architectural affiliation.The shell has been separated from the fittings and openings.These classes were themselves divided by architectural elements such as openings, doors, walls, roofs, etc. 4.2.1.2.The semantics were directly inspired by the structure of the scene.3D objects are named X_objet_i with X its temporal class (A, B or C), object its description and i its number if it was a duplicated element.In general, the traceability of modelled 3D objects has been codified by adding to the object nomenclature the designation of the ensemble to which it belongs (dwelling, tower, etc.), or its geographical position.4.2.1.3.The representation chosen within the 3D scene itself is the wireframe shaded representation.This representation makes it possible to visualise the configuration/topology of the elements and thus to understand the use and choice of modelling/editing tools.Modelling was mainly carried out in 3D but can also be carried out in 2D in the case of architectural modelling based on technical drawings.In this configuration, the representation mode is often wired for greater visibility.At the end of the workflow, textured wireframe shading was also used to visualise the approximate overall textured effect of all the elements combined.This made it possible to detect textural inconsistencies in both colourimetry and placement.4.2.1.4.Topology describes the relationships and interactions between the different entities (edges, faces, vertices) forming a 3D model.Topology is very important within a 3D model.Correctly constructed, it facilitates editing, UV mapping and the creation/rendering of 3D model animations.A 3D object can be modified while maintaining a coherent topology by decomposition using Edge Loops.Optimisation of a 3D model is required throughout the modelling process, from design to editing and export.Maya offers cleanup, retopologize and remesh tools.
As an example, we find the different parts of the latrine modelling, part on the heights of the north perimeter wall (Figures 7).

4.2.2
The Wasenbourg processed in Blender.The entire historical reconstruction of Wasenbourg was processed in Blender, which offers UV mapping, animation, and physical simulation tools, and can even act as a game engine.Its community of users provides numerous add-ons.Blender also offers two modes of interaction with the elements of a scene: object mode, where the action will affect the entire object, and edit mode, which will only affect the selected part of the object.A 3D object is made up of vertices, edges, and faces.In edit mode, it is possible to select from these 3 components and modify them.In addition to the classic tools for modifying initial geometric primitives, such as the subdivide, extrude, and bevel tools, Blender offers "Modifiers", which are pre-built algorithms that can be applied to objects.The boolean modifier allows an object to remove a volume from its topology that corresponds to another 3D object.This was useful for creating holes in walls to accommodate windows, for example.Using the same principle as Maya, the "Remesh" modifier transforms the topology of a 3D object into a triangulated mesh.Another interesting tool in Blender is the geometric node which allows to modify the geometry of an object using node-based operations.It can be accessed by adding a geometric node modifier.This system has been used to manage textures in the "Shading" interface mode, which proposes modifying the geometry of an object or the parameters of a material through parameter blocks linked by Nodes, forming a schema.An initial material is made up of the Principled BSDF block containing all the main texture parameters, linked at the output to the Material Output block which displays the input elements in the scene.Many parameters, such as base colour, metallic effect, or specular effect, can be modified directly in the BSDF using a scale between 0 and 1.It is possible to import these values by providing input UV maps associated with the various parameters.This has the advantage of offering a parameter on local areas rather than a fixed value on the entire texture.UV maps are created from the textured model in its current state or from photographs acquired in the field.The diffuse map is extracted from it as well as other UV maps by modifying the colours and shading effects ("Specular map, "Metalness map", "Roughness map", "Height map", "Ambient Occlusion map").There is also an add-on to generate a "Normal map" from the base image.The texture of the castle walls as well as the windows were thus generated.The main difficulty of this method was to make the image connectable, repeatable on the model (seamless texture) without observing distinct cuts.It was decided to use a seamless texture from the Oedenburg Castle which corresponds to a wall with bosses like those of the Wasenbourg Castle but without the clamp holes.The holes were then redrawn on the image and the texture applied to all the walls of the model that have stones with bosses.For the parts where the stone has clamp holes but no bosses, we exploited the texture of the current state.Finally, for the windows, we used a window image from another medieval castle.

Maya or Blender:
Both software programs offer similar modelling tools but are organized differently.They both rely on feature-rich interfaces with tool redundancies and customizable bars.The key to both environments is the use of keyboard shortcuts to access the various features that cascade into each other.Data structuring is possible in both environments, the construction history of different objects can be preserved, and transformations/modifiers can be applied to make them evolve.Objects can be broken down and named for better understanding and tracking of the modelling.Representation in different modes is useful for visualizing different components and their nests in a single object.Topology and retopology tools are powerful to maintain a homogeneous structure and above all to allow efficient and realistic texturing.Finally, heavier but effective rendering tools (Arnold on Maya, a tool integrated into Blender) made it possible to calculate very realistic renderings or even interesting animations, especially in the case of the project of historical restitution through time.It's worth noting that Blender is free open-source software, while Maya is a licensed product.Customization is done in Maya via MEL (Maya Embedded Language), Blender offers the use of Python scripts.While Maya's user interface may seem more standard, Blender's user interface is also easy to navigate with more simplified surface and texturing tools in Blender than in Maya.Support communities are important for both environments but seem more active for Blender.Finally, both environments offer import and export formats allowing interoperability with many other platforms, and particularly the STL format for 3D printing of the generated models, which will be a strong desire in the Rhine castle projects.

Results and evaluations:
This project focused on the production of 3D restitution models of historic sites (the ruins of Oedenbourg and Wasenbourg) based on archaeological data and knowledge accumulated over time.3D models of the current state of the ruins were made in Maya or Blender.4D renderings have been proposed for different periods.For example, based on hypotheses and archaeological analyses accumulated over several decades, the site of Oedenbourg has been reconstructed in its three most important historical phases, namely phase A: the fortified enclosure (10th-11th century), phase B: the main tower (early 13th century), phase C: the Gothic dwelling (late 13th century) (Figures 10).
Assessment and uncertainty terms were used to produce colorized models according to the criteria in Figure 8 (methods of (Landes et al., 2019) and (Hermon et al., 2006) combined).The models are now ready for use in virtual or mixed reality environments, to be physically transformed into touch models.The models are now ready for use in virtual or mixed reality environments, to be transformed into tactile models, and will serve as a knowledge base for archaeologists and historians, who can use them to create unprecedented visualizations and simulations to confront their hypotheses with realities on the ground, transcribed in digital twins.

CONCLUSION AND OUTLOOK
These two parallel studies made it possible to initiate the project on the modelling of Rhine castles.It offers complete data acquisition processes for modelling the ruins of castles in their current state.With the additional knowledge of historians and archaeologists, these 3D models served as a basis for historical restitutions at different periods in time.All models can be integrated into realistic renderings using animation software.These ruins and restitutions will also make it possible to create tactile bronze models for the visually impaired or people with reduced mobility.All the stages of the search for knowledge up to the restitution in the form of models have made it possible to complete, sometimes to confirm, sometimes to cast doubt on previous work and archaeological research and in any case make it possible to complete the documentation on the remains of the Rhine castles that are still accessible.period 2023-2025 for the knowledge and enhancement of Rhine castle heritage.

Figure 1 :
Figure 1: Ruins of the Gothic dwelling at Oedenbourg

Figure 2 :
Figure 2: The Wasenbourg ruins with their imposing shield wall.

Figure 3 :
Figure 3: Photogrammetric and lasergrammetric processing chain to obtain a point cloudThe models obtained in the form of point clouds were then meshed and textured using the processing chain shown in Figure4to produce the 3D models illustrated in Figures 5.

Figure
Figure 4: General processing chain: meshing and texturing.
Figure 4: General processing chain: meshing and texturing.

Figures 7 :
Figures 7: (a) Image of the ruins of the latrines in the dwelling (north face) -(b) Modelling of the latrines under Maya -(c) UV mapping of the 3D model of the latrines under Maya -(d) Final textured rendering of the latrines under Arnold (for Maya)

Figure 8 :
Figure 8: Colour codes for uncertainty representation Figures 9 show illustrations of these assessments at the two sites studied.

Figure 9a :
Figure 9a: Graphical representations of the evaluation criteria for the restitutions of the Oedenbourg site by evaluation by element.

Figure 9b :
Figure 9b: Wasenbourg in the thirteenth century 4.3.2Rendering and animation: A final consisted of the creation of realistic scenes and animations to enhance the project and to render it visually from different points of view, sometimes unprecedented.The preferred rendering environment was outside of the two modelling platforms described.This was Lumion, which offers interesting rendering and animation features.

Figures 10 :
Figures 10: Final 4D rendering of the historical phases of the Oedenbourg site in Lumion 4.3.3Virtual reality export: an extension of the project was to export it to a virtual reality cube.This device allows visualization in immersion mode in a cave environment.The device uses the Unity platform for the various renderings and offers interactions related to the movement of the main actor, captured by cameras, and by joysticks allowing to move or use tools available from a virtual tablet.An overview of the environment is shown in Figure 11.

Figure 11 :
Figure 11: The Wasenburg in the VR Cube