INTEGRATION OF HBIM, XR AND BEACONS FOR CULTURAL MEDIATION OF HISTORICAL HERITAGE : THE CASE OF “AL-QUARAOUIYINE MOSQUE” IN FES

: The mediation of cultural heritage is a significant concern in the field of history and culture valorization. Technologies such as HBIM (Historic Building Information Modeling), XR (Extended Reality), and Beacons offer highly promising solutions that have demonstrated their value in various heritage-related applications, namely 3D reconstruction, documentation, restoration, preservation, and mediation. The primary objective of this paper is to propose an approach that combines HBIM () and XR (), integrating Beacon sensors to create a rich cultural mediation experience. Our approach is based on an HBIM model of the site, enriched with multi-source semantic information stored in a NoSQL database. It also leverages Beacon sensors to deliver historical content related to the study site (Mosque Al-Quaraouiyine) based on the user's position, thereby providing a cultural mediation experience within an XR environment. This study resulted in the creation of an HBIM model of the study site, the integration of a MongoDB database for storing semantic data, the utilization of the Unity SDK to retrieve this data, and the deployment of Beacons for user position detection within the XR environment. These advancements led to the display of contextual historical and multimedia information through interactive panels, offering a rich and interactive experience. The added value of this project lies in its innovative combination of several modern technologies to provide an educational and immersive experience to users, while also paving the way for future


INTRODUCTION
Cultural heritage serves as a testament to history, identity, and cultural diversity, playing a crucial role in transmitting these aspects to diverse audiences.The evolution of cultural mediation, transitioning from traditional tours to digital solutions, is underscored in the context of the transformative potential offered by Historic Building Information Modeling (HBIM) and Extended Reality (XR) technologies (Ferretti et al., 2022).Notably, the incorporation of beacons on historical sites further enhances the cultural mediation environment (Barsocchi et al., 2021).The convergence of HBIM, XR, and beacon technologies presents a promising avenue for creating engaging and educational cultural experiences, thereby strengthening the connection between visitors and heritage sites.This study explores these intersections and their implications for cultural heritage preservation and dissemination.
In this research, an investigation about the integration of HBIM, XR and beacons in cultural mediation, supported by a case study of "Al-Qarawiyyine Mosquee" is presented.The question addressed here is: how the 3D relevant technologies of HBIM, XR and beacons can respond to the need of cultural mediation of built heritage?The objective of this paper is to develop an innovative solution based on HBIM-XR-Beacon integration for cultural mediation.
The paper structure is as follows: Section 2 briefly introduces the concepts and gives an overview of related work in this research area.Section 3 presents the methodology we adopted for developing the solution.The case study, results, limitations and the conclusion are set out in Section 4.

RELATED WORK
In the landscape of heritage building, Heritage Building Information Modeling (HBIM) stands as a pivotal method, offering a sophisticated approach to the management of historical sites.As highlighted by (Khan et al., 2022) and (Arfaoui, 2018), HBIM creates a cohesive environment by consolidating diverse information into a unified BIM database.This comprehensive database includes 3D geometry, architectural details and historical information associated to the site, fostering in-depth documentation that allows efficient management, preservation, restauration, and cultural mediation of heritage sites.The significance of HBIM is underscored by studies such as those conducted by (Santagati et al., 2021) and (Khan et al., 2022), showcasing its ability to eliminate superficiality in documentation.By providing a deeper understanding of the architectural and historical context, HBIM becomes a cornerstone in the preservation and management of historical buildings.The integration of Extended Reality (XR), a transformative technology encapsulating Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) ( (Yang et al., 2020) augments HBIM capabilites by extending reality with added information such as historic documentation and external resources.For instance, the work by (Ferretti et al., 2022) exemplifies how XR enhances visualization, simulates real-life scenarios, and fosters collaboration among project stakeholders.This seamless integration ensures effective interoperability with BIM, unlocking new possibilities in the realm of historical BIM.The integration of NoSQL databases, specifically MongoDB, with Building Information Modeling (BIM) models introduces a dynamic dimension to construction and project management.
MongoDB functions as a flexible repository for data storage and analysis, enabling real-time visualization within the BIM framework ( (Kanna et al., 2022).This integration not only facilitates data structuring but also extends support for modeling environmental parameters, such as temperature and humidity.Consequently, it enhances applications such as thermal comfort monitoring within the construction domain, specifically for historic site preservation.The introduction of Beacon sensors marks a significant advancement in enriching physical environments with semantic information.(Bilbao-Jayo et al., 2021) delve into predicting user behavior in indoor spaces using beacon signaling data.(Spachos and Plataniotis, 2020) propose a system leveraging beacon technology to provide users with cultural content related to observed works of art.In cultural heritage sites, Bluetooth Low Energy (BLE) technology, as explored by Barsocchi et al. (2021), offers a promising avenue for enhancing visitor experiences.Their cross-platform mobile application, utilizing BLE for proximity detection, aims to deliver tailored digital content related to artworks.The meticulous testing and evaluation of this approach underscore its potential to significantly enhance visitor engagement and interaction within cultural exhibits (Barsocchi et al. (2021).The culmination of HBIM, Beacons and XR technological advancements is exemplified in the work of (Ferreira et al., 2018), where HBIM, beacon sensors, and XR converge to enable real-time location and guidance within complex structures.Beacon sensors emit Bluetooth Low Energy signals, allowing mobile devices to precisely interpret user positions.In conjunction with this location data, BIM provides detailed information about the physical context of the building, including room layouts, levels, and structural features.The convergence of HBIM, beacon sensors, and XR represents a paradigm shift in cultural mediation, promising visitors interactive and immersive experiences.This integration, supported by NoSQL databases like MongoDB, not only manages semantic data effectively in XR environments but also personalizes the visitor experience through precise indoor guidance and contextual content delivery.In this paper, our objective is to implement these technologies within the distinct context of the "Al-Qaraouiyine Mosque" in Fes.

METHODOLOGY
The methodology proposed in this work establishes a connection between the HBIM model, the MongoDB database and the beacons through an XR environment (Figure 1), ensuring the creation of an immersive experience that allows users to explore the historic monument and access its culturally valuable information.The importance of this methodology lies in its ability to merge diverse technological domains to create a rich cultural mediation experience.The project followed a three-stage methodological process to reach its conclusion.Firstly, data acquisition was initiated, involving the collection of information on the site.Next, a complete HBIM model of the site was produced in Revit, integrating the acquired data.For semantic management, MongoDB was employed as a NoSQL database, covering both the HBIM model and additional multimedia data.The third step was to extract the geometric and textural information in the XR development environment.The precise selection of semantic information to be extracted to Unity was orchestrated using Dynamo and the Unity SDK.Dynamo nodes were created to select all elements in the Revit model, extract the specified information, and a Python script was attached to transform this data into an exportable JSON file.Finally, Beacon sensors were deployed to detect user positions and broadcast semantic content appropriate to each position on the historic site.This global approach enabled the development of an interactive XR environment, offering contextual information panels and multimedia data linked to each object present on the site.

CASE STUDY
We conducted our research in the "Sahn Al Quaraouiyine", a distinctive section of the Al Quaraouiyine Mosque in Fes 2), initially constructed in 859 under the guidance of Fátima al Fihriya.Beyond its historical significance, the Al Quaraouiyine Mosque holds a prominent position in Fes, both architecturally and spiritually, standing as one of the city's most imposing structures.The courtyard, located on the western side of the mosque, is a captivating feature adorned with meticulously crafted blue and white tiles.Noteworthy are the three marble fountains within this space; The central fountain, an original Almohad piece dating back to the 12th century, is complemented by two additional fountains, positioned beneath symmetrical patios added during the Saadian reign under Sultan Abdallah ibn al-Shaikh.
The Al-Quaraouiyine Mosque (fig.3), as it stands today, is the result of initiatives, extensions and experiences accumulated by Moroccans (sultans and benefactors) over the Ages.This unique monument is a living testimony to the long and rich history of Fez in particular, and Morocco in general.In the context of this project, we focus on twelve specific historical elements of the site.The goal is to visualize the semantic information associated with these elements in an extended reality (XR) environment, employing a carefully adopted methodology.Our approach seeks to provide a comprehensive understanding of the historical and cultural significance encapsulated within the Al Quaraouiyine Mosque's architecture and its Sahn by adopting 3D innovative technologies namely, HBIM, XR, and beacons.

Acquisition and Data Collection
3D geometric data for the study site were acquired using an RTC360 terrestrial laser scanner to build the HBIM.A total of 21 stations were deployed.The data were imported into the Cyclone pre-processing software for noise cleaning, and registration.Subsequently, we collected semantic information for integration into the HBIM model from site managers in the form of PDF documents, as well as historical site multimedia files.In addition, numerous photos and videos during various visits to the mosque were collected.The information gathered testifies to the rich history of the cultural heritage we wish to highlight.Figure 4 shows the raw point cloud obtained after the 3D scanning mission.

HBIM modelling
This phase involves using point cloud data to create an accurate 3D model of Sahn Al Quaraouiyine.The modeling of structural elements is crucial to establishing the fundamental structure of the courtyard.The main objective is to reproduce the geometry of these elements in detail, including their shape, dimensions and layout.Each element is carefully modeled to faithfully reflect its physical reality, in order to create an accurate and complete digital representation of the structure, which will serve as a solid foundation for the subsequent stages of the project.It is then important to model the architectural features, ornamental details, specific building materials and all the architectural historical elements that make up the site, using parametric families.Each ornament, sculpture, decorative motif, stone or building material is reproduced to produce a complete HBIM model that will be both an accurate archive of architectural history and a means of virtually preserving the site's rich cultural and heritage assets.The location of beacons on the site is also modelled.This detailed modeling also enables researchers, art historians, architects and curators to study and preserve these historic elements in greater depth, while offering the public an immersive experience that transcends time and space.To optimize visual quality, SIMLAB COMPOSER is used to incorporate realistic textures and adjust lighting parameters.The model is transferred from Revit to SIMLAB via the DWF exchange format, resulting in a more authentic and visually refined representation.The resulting model is shown in Figure 5.

Extraction of Geometry and Textures
Once all modifications been completed in SIMLAB COMPOSER, the model is ready to be exported in a format more widely compatible with XR environments, typically FBX format.This export preserves all the essential elements of the model, including geometry and textures.The choice of FBX format is significant because it is a widely supported file format in the 3D modeling and video game creation industry.It is compatible with many applications, including Unity, which is commonly used to create interactive 3D applications.Once the model has been imported into Unity, it becomes a key component of the XR development environment.

Extraction of sematic data and Database Structuring
In this step, we need to extract the semantic information associated with each geometric element, store it in the database and then transfer it to the XR environment.To do this, we used Dynamo, the visual programming language integrated into Revit software, and the JSON (JavaScript Object Notation) format compatible with MongoDB.Dynamo is the preferred choice for extracting semantic information from the HBIM model in Revit, thanks to its native integration with Revit, its visual programming interface and its flexibility to be customized according to project needs, in particular the precise choice of the type of semantic information we want to extract (fig.6).The result is a file containing detailed information on each component of the model, recognized by an identifier that will later be used to create the link between this information and the geometric object itself.This file is imported directly into the database.In order to enrich the XR experience and enhance the value of cultural heritage, it is important to add, in addition to contextual information, multimedia information that will enable the user to fully discover the site in question and immerse themselves in its history through photos illustrating the historical phases of the monument, videos of guides recounting the experience of the historic site and narrating its rich past, and PDF files of documents, old stories and archives about the site.
We can insert the multimedia files into MongoDB Compass using Gridfs (fig.7).This is a file storage system for MongoDB, designed specifically to store and manage large files, such as multimedia files.It is particularly useful when you need to store large files that exceed the 16 MB limit for each document.GridFS automatically divides files into smaller chunks and stores them in the "fs.files" and "fs.chunks" collections.The "fs.files" collection contains the file metadata and the "fs.chunks" collection contains the actual chunks in the file.This is achieved using a python script.

Figure 7. Integration of multimedia files in MongoDB
MongoDB, as a document-oriented NoSQL database, offers a unique framework for storing, organizing and querying information, and has the ability to efficiently manage a wide variety of data types, which is essential for enriching the XR experience, especially as the data we've used to enhance this XR experience is diverse and comes from many different sources.We used this database to store all the semantic data extracted from the HBIM model under the collection named {Semantic}, and we also managed to store multimedia elements including photos, videos and PDF files as chunks (fig.8).

Development of the XR Environment
The initial step in developing the XR environment (fig.9) is to import the HBIM model into Unity, using the FBX file of the historic obtained previously.This file is placed in the project, and the position, rotation and scale of the HBIM model are set to match the desired location and orientation in the environment.We have implemented mechanisms for interacting with the model, such as moving within the model, selecting objects, displaying the identifier of the selected object, etc., which can be performed using the controllers or simply with the user's hands detected by the headset.Then we proceed with the following steps: 1. Connecting Unity to MongoDB using a C# script: This step involves developing a C# script to establish a connection between MongoDB and Unity.This allows Unity to query the database and extract semantic informations associated with each element of the HBIM model.2. Importing semantic information from the database: Once the connection is established, the XR application can access the MongoDB database to perform queries and extract semantic information.3. Developing a C# script for linking geometric elements and semantic information: This script plays a central role in the application, associating each element of the HBIM model with its equivalent in the MongoDB database using a unique identifier.Thus, when a user interacts with an element of the model in the XR environment, the script retrieves the corresponding semantic information and displays it to the user.4. Developing the user interface for information visualization: This part of the application involves creating a user-friendly interface within the XR environment.This interface should allow users to easily access historical and cultural information about the objects they explore.Information panels are developed to present this data.5. Visualizing the result with the Oculus Quest 2 headset: Once the development of our application is complete, we proceed to visualize the result using the Oculus Quest 2 headset and interact with the objects.The application can be enhanced by using beacons (BLE beacons) for precise localization.These devices are placed in significant locations on the historical site, each beacon is associated with a unique identifier, and as the user approaches an object or site of interest, the XR application can detect the corresponding beacon and automatically display multimedia information, such as videos, images, or audio commentary, at the appropriate location.In AR, VR, and MR, integrating beacons into development environments such as Unity is a complex process that requires appropriate application programming interfaces (SDKs) to ensure direct and consistent communication between beacons and software.However, to date, there is no officially recognized and widely adopted SDK for directly integrating tags with Unity.This means that there is no standardized protocol enabling Unity to interact with beacons natively, making it difficult to exploit them physically directly in this type of application.To overcome this limitation, we had to create a simulation within the Unity environment to represent the role of c) The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-2/W4-2024 10th Intl.Workshop 3D-ARCH "3D Virtual Reconstruction and Visualization of Complex Architectures", 21-23 February 2024, Siena, Italy beacons in VR.This simulation was developed using specific modeling and programming methods to reproduce the expected behavior of beacons in a VR context.This includes the simulation of beacon position detection, their unique identification, and any other key functionality they offer.

CONCLUSION
This project investigates the use of XR and beacon technologies to create immersive and informative experiences based on HBIM building models.The use of VR has demonstrated its potential to create immersive environments that captivate and engage users for education and cultural preservation.It has also shown that integrating HBIM technology with NoSQL databases such as MongoDB and XR environments can revolutionize how we preserve, document, and share historical heritage.Interaction with virtual objects and navigation in 3D scenarios has contributed to a deeper and more memorable learning experience.It has also allowed for customizing content to meet the individual needs and preferences of users at their own pace and according to their interests.Beacon tags have also played a key role in enabling location-based experiences.They have been used to trigger contextual content when users approach certain areas, reinforcing the interactive dimension of virtual environments.Several perspectives for future projects can be proposed, including the following:

•
Extending this application to cell phones to open up opportunities for universal accessibility, enabling a wider audience to enjoy historical and contextual information without the need for a specific VR headset.However, this requires significant adaptations to the user interface, optimization of performance for different devices, efficient management of power consumption, and consideration of the loss of immersion inherent in the use of touch screens.Despite these challenges, this expansion would considerably broaden the scope of the application, making it more accessible and cost-effective for a diverse audience.
• Implement an automation system for ongoing updates.If new discoveries are made about a historic site, the virtual content can be updated automatically.In this way, we can ensure that users have access to the most up-to-date and accurate information.This could be achieved by connecting to real-time historical data sources, using APIs to retrieve up-to-date information, or even integrating user contributions to continually enrich the historical database.However, this also requires rigorous management of data quality, verification of sources and protection against incorrect or inaccurate information.Automating this process would ensure that the application remains a valuable and up-to-date resource for users.

•
The future implementation of BLE beacons in real environments.

Figure 3 .
Figure 3. Plan of the Al Quaraouiyine mosque and adjacent monuments, indicating the location of Sahn Al Quaraouiyine

Figure 9 .
Figure 9. Examples of interaction providing information: a) modeled beacons that broadcast an audio recording.b) broadcast of a historical video in the virtual environment.c) display of a contextual information panel related to an element.