SURVEY TOOL ALIGNMENT FOR REAL-TIME MIXED REALITY INFORMATION MODEL INTERACTION IN HERITAGE RECORDING

: While heritage documentation today is largely based on three-dimensional models as key element, the easy access to tools for creating such models as well as quick and easier to use workflows lead to a stark increase in data creation while being followed by less structured and standardized methodology. Increasingly information models are applied to structure all available data regarding an object and reference it within a digital twin of the physical object. Nevertheless, various approaches reviewed all show a process of information model creating as post-processing task, undoubtedly leading to parallel systems of captured data, and disseminated data. The following research proposes to encounter the mentioned issues by developing a strongly interwoven integrative building documentation workflow with including GNSS-RTK based survey and augmented reality tools, to start compiling coordinate-based information models right on site during the first acquirement of data. As much data as possible should be included into this process, covering geometry, visual observations, and semantic information. Making data available right after inclusion into the information model should then allow to give real-time feedback on all data available (old and new) right on site or remotely within a web-platform, virtual reality interface or else. Detaching the work of curating and information model from a manual processing step, but rather being able to link information through their position right at the time of origin, would allow models to be created, enhanced, and expanded collaboratively further giving them a fourth dimension of time and collective knowledge.


INTRODUCTION
Today methods for heritage documentation are seeing threedimensional digital models as a key element. Software and devices for creating these models are getting ever more accurate while being fast in acquiring their models and intuitive in use. While this allows various actors and organisations to create super-realistic digital representations of our physical (built) environment, the data produced becomes of overwhelming amount and highly fragmented. Especially since the possibility of processing images into high-quality digital models becomes ever more accessible as well as customary to be used even by many non-professionals for means of game development and other applications. While an increase of coverage in digital counterparts to physical structures could be considered valuable in some context, the quality of this data can often be considered questionable, not because a lack of precision or resolution, but due to absence of transparency of its genesis and/or dissemination of results as well as missing interoperability, standardized methodology and information longevity (Santana Quintero et al., 2020, p. 6).
The following paper therefore aims to propose such a standardized methodology that in itself enshrines interoperability as one of its highest values and therefore ensures interoperability within a greater context. Further, through mapping out data streams between hardware and software, it suggests a viable way to disclose the genesis of digital data within a complex and interlinked documentation campaign as well as the following processing phase. All this is done in a greater context of investigating the ways and possibilities of introducing augmented reality as means of information input into the process. Nevertheless, there current work tries to outline how this could work and what opportunities this could provide but will focus on the basis of survey behind the mixed reality interface as well as the process of compiling geometric and positional data for a to be described coordinate-based information model.

Problem statement
With the aim of better enabling data already available for the preservation and research of monuments and sites and counter before mentioned fragmentation of data, various research efforts are setting up platforms based on digital models to carry more information than just geometry and colour, using the digital representation of the physical as container for semantic observations and references to other datasets (Bianchini and Potestà, 2021;Brumana et al., 2020). Since the first proposal of using BIM (Building Information Modelling) for a heritage context (Brookes, 2017), often referred to as HBIM or Heritage/Historic BIM, many of these research efforts are working on providing a best practice for specific cases of converting survey data into BIM models and try to define frameworks as well as discuss the implications of applying BIM for a heritage context. Not to mix in terminology, this paper will use the term BIM only for information models using standardized IFC classes as defined in ISO 16739 as components. All other model-based assemblies with semantic information with simply be referred to as "information model." While all of these research efforts are considered of great value in their goal of shedding light onto the process of compiling such models and are well worth being discussed, it is the believe of the author that maybe the BIM framework as we know it today is not entirely set out to fulfil all the hopes that are put into its application in the field of heritage conservation. In fact, there is believe that some of the basic structure and general rules of BIM are somewhat inadequate. Further, there is an understanding that the love of the heritage documentation sector with BIM could be considered one-sided. Considering the buildingSMART strategic roadmap (buildingSMART, 2023a) to the development of BIM systems, there is no trace to be found of the implementation of surveying processes for existing structures or components tailored to support heritage documentation.
First, the basic principle of BIM models being composed of individual objects, each being a dynamic block, can already bring about some issues in the process of translating survey into BIM. While in a new built structure the boundaries and materialistic composition of each element is still up for determination and therefore known, the digitalisation of pre-existing substance always comes with some uncertainty. Especially in amorphous structures semantic segmentation, the task of splitting the built volume up into elements, comes with some interpretive leeway. Simply said -where does the wall end and where does the vault start? Further, the BIM framework does not know incomplete objects or uncertainty. Therefore, within the process of creating a BIM model from pre-existing structures, there is even further reconstructive interpretation involved. As a result of this necessary interpretation, it is believed that funnelling information into a BIM framework reduces data transparency as especially these interpretive steps are seldomly documented and/or comprehensibly.
While using a BIM framework can be considered beneficial in terms of data longevity and accessibility to a larger range of users due to its popular use, the believe is that it might not be the best way to ensure long-term storage of an as-is state of a physical structure. Through translating a documentation into a BIM model, we are leaving the domain of the measurable and move into another domain of interpretation. As a later transformation of the measured information into an interpretive model might still be possible but not vice versa, it is believed to be of immense importance to consider preserving the raw survey model with the same meticulousness as the BIM model. Especially in cases, where the original structure might not be available or accessible anymore, or at least not in the same state, the mere objectivity of the measured model, representing the as-is state of a defined area of our physical environment at a defined moment of time with measurable accuracy, should at least be held up to the same importance as the refined BIM model.
Resulting from this comes an understanding that it must be of equal importance of creating frameworks for the organisation of raw survey data in a way that its genesis, interrelations, accuracy, and limitations remain known and comprehensible to a later user. Ideally a supplemental relation of BIM and survey model could be achieved, each developed to its necessary refinement, neither trying to cover all needs by itself. With a latest memorandum of understanding between buildingSMART International and Geospatial World (buildingSMART, 2023b), there is some hope for a stronger link between BIM modelling and geodetic information -as is playing a large role in heritage documentation. This could lead to a more interlinked implementation of measured survey data and refined BIM data and could mean survey and interpretation could go more hand in hand in the future.

Research proposition
To encounter these issues, it is believed to be crucial to introduce standardised workflows and structures of dissemination as early as possible into the process. Ideally it is believed that all geometric data, all semantic information and all visual representations and observations, such as sketches, plans, photos, and observations should be introduced into the process on-site when collected. Using georeferencing and positioning of each piece of data collected as a link between all datasets, the proposed approach aims to refine an enrich every piece of data on site using appropriate metadata so it can be included into every type of database and/or information model without (much) further processing. And even when further processing is needed -i.e., refinement of positioning, registration of laser scans, processing of photogrammetric models -initial data should already be at a level to be represented within a real-time update of the surveying process as early as possible.
To achieve this, an approach of strongly interlinked application of surveying devices, introduction of real-time registration and (preliminary) processing of laser scans and photogrammetry as well as wide-range integration of GNSS-RTK positioning on various sensors is proposed and discussed. This so-called integrative documentation workflow should further be supplemented with intuitive mixed-reality devices able to introduce semantic information, references, observations into this process independent of geometric data -in short: every piece of data is assigned its position on site. While there are some advantages to base the collection of semantic information on already available geometry (Abergel V. et al., 2021), this research aims to develop a process independent of geometry to allow parallel acquirement.
It is believed that such an approach would not only be able to deliver data to be processed more easily into BIM or GIS systems, but also extends possibilities of collaborative work through real-time visualisation of data on-site and remote -i.e., VR (virtual reality) or web-application based consultation of a remote researcher during on-site work. Detaching the work of curating an information model from a manual processing step as much as possible, but rather being able to link information through their position right at the time of origin, would allow models to be created, enhanced and expanded collaboratively (Manuel and Abergel, 2022) further giving them a fourth dimension of time and collective knowledge, laying the groundwork for an "n-Dimensional analysis and memorisation ecosystem" ("ERC n-Dame_Heritage," 2021).
The research is based on following four basic principles:

Principle 1: Position
A coordinate-based information model could provide organisational layout not only for the surveying campaign, but as a storage environment referencing all available information to each other based on their geolocated position and orientation. Therefore, a strong focus is given on applying a position or positional relation to every piece of information as early in the process as possible.

Principle 2: Individuality
Detaching even parameters traditionally intricately linked with geometry like materiality, state of preservation, chronology as well as other descriptive information and observations from geometry allows them to prevail even when their "host geometry" is updated or reprocessed. Further, this could mean information could more easily be transferred to and compared with other geometric datasets.

Principle 3: Real-time
Processing as much information as possible in real-time while on site allows comprehensive feedback on the documentation process and precludes the necessity of a manual post-processing of individual models and datasets into a collective information model. Further, making pre-processed data available already on site allows a compression of the research process and could promise a more complete understanding through availability of more comprehensive information earlier in the process.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy

Principle 4: Update
All processes involved should allow an update sequence as much as possible. While it is unavoidable some parts of the procedure demand the marking of a certain milestone of processes that need to be finished before moving on, i.e., the registration of laser scans before further processing into 2D plans, it should always be the guiding light to achieve process sequences that allow change within an earlier step and will follow through a string of updates -as automated as possible -to update these changes down into the last step of the process, i.e., the update of a raster image in the background of 2D plans after changing the colouring of the scan.

Research goals
The goal of this research is to propose an integrative documentation workflow that serves the compilation of a coordinate-based information model that stipulates the integration of mixed reality tools for information input, while adhering and promoting all four beforementioned basic principles along the way. While the whole workflow is laid out as a theoretical framework, it is adapted and adjusted to become increasingly of a practical best-practice. To do this the individual components within the workflow are not just seen as general stat of the art but incorporate hardware and software available to the author. Based on these components, processes and data streams are investigated to functionality and practicality and are continuously included and redeveloped to serve the bigger context. Therefore, this research is not to be seen as a proposition for a new standard, but a case study workflow that promotes the possibilities and tries to discover obstacles and challenges along the way. Deviations of what is decided within the case study compared to what would be ideal will be discussed within the introduction of the individual components.
In the first part of the results the workflow in its whole will be presented and elucidated with its core principles. Further some of the key components will be elaborated and specific decisions in the workflow layout discussed. After this the possibilities for information model interaction using mixed reality technology that are envisioned to be built upon the integrative documentation workflow will be presented and outlined. As this is an ongoing research, components in this part represent an outline of what is deemed to be possible and should help highlight the advantages of the before presented workflow.

Literature
As the topic at hand covers a wide range of aspects, there was not literature available at hand that could deliver comparably comprehensively engagement with a documentation workflow or the general concept of a coordinate base information model. Research dealing with individual components are cited where used but are not represented here as a collective basis of knowledge. While there is a white range of papers available on Scan-to-BIM workflows, most of them focus on a specific case study and/or methodology of creating BIM elements. And while most of them are presenting impressive approaches and results, there are not considered particularly helpful in the endeavour presented here. Nevertheless (Bianchini and Potestà, 2021) and (Attenni, 2018;Attenni et al., 2022) was of particular help grasping the boundaries and implications of applying BIM models for heritage documentation.

Methodology
To gain an understanding of the larger picture of data streams and components involved in the building documentation workflow currently applied within research and teaching at the department of building history and building archaeology, they were mapped out using elements of a procedural diagram. While the first generation focused on nodes with processes and connectors with data streams, the here presented third generation of the diagram already tries to incorporate decision nodes. Based on the information mapped out in the diagram, processes and streams were optimized through iterative refinement towards incorporating all the principles introduced earlier in this paper.
Step by step, it was possible to introduce new procedures as well as even acquire and introduce new hardware and software that were able to simplify parts of the whole or even render some parts obsolete.
Through an increasingly standardized workflow it was then possible to refine the level of definition of consecutive processing steps. Whenever possible it was attempted to even automate processes through the introduction of command line scripts. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy

INTEGRATIVE WORKFLOW
The procedural diagram of the integrative documentation workflow (Figure 1) aims to visualize a variety of things. It consists of a number of nodes, which represent a range of different devices and software packages used to acquire and process data. Most nodes are supplemented with a descriptive text, explaining the operations taking place. Second the nodes are connected representing a stream of data as well as direction from one node to another, further mapping out their relationship with each other. Along the connectors the type of data that is exchanged is demarked.
On the vertical axis, from top to bottom, stages of the documentation and processing procedure are shown with the top being the preparation phase and the bottom being the publication phase. In this, the top half of the diagram -demarked prepare, collect and live-process -represents the actions that take place on site, and the bottom half everything that happens within a postprocessing phase.
Especially within the on-site part of the diagram a more refined elaboration of processing steps including decision nodes was introduced in preparation to the introduction of automated processes. This was forgone in the post-processing part, at is believed that within this realm, processing procedures can be allowed to be less standardized, do not have to follow the principle of real-time processing and can be and more tailored to the specific needs of the object in focus.
One aspect that can be read from the diagram, given the vertical time axis and the procedural succession through the connectors, is the sequence of consecutive steps, and what processes need to be finished or be delivering data for the next step to start. Further, the layout visualises an organisational pattern for a centralised data management within the process as well as the necessary interfaces and points of exchange that need to be implemented for the process to work to its full potential. Within that it is believed beneficial to read the diagram not as a set of containers of data, which hold a certain information and exchange it with each other but interpret connectors as streams of data. Through input of data by a device one end of the process it runs through various nodes, interacting or merging with other sets, being transformed by operators, and exiting as final output on the other side. Much like a node editor in graphic programming or parametric design it is envisioned that through laying out the streams of data through the various nodes in combination with (semi-) automated processing steps, this can be understood as the basis for an interlinked processing environment with centralised storage that is key to any interaction with real-time feedback. Additionally, through interlinking processes and data with each other with a mandatory shared geodetic coordinate reference framework, output data will show much larger coherency and interoperability -in itself and with other sets.

Coordinate reference
To be able to interlink data streams as proposed it is obviously necessary to introduce a shared coordinate reference. While it could be deemed ideal to try to use a most absolute system in every part of the process, some issues arise that entail the necessity of accepting more than this one general system. Individual national coordinate systems with their fixed and demarked reference points throughout their area of recognition used to be standard and most absolute system within the field of archaeology and building documentation -for everyone that was not working only with a locally delimited coordinate system.
Since the introduction and broad use of GNSS (Global Navigation Satellite System) Equipment this certainly shifted towards the application of global geocentric coordinate systems. The usage of the World Geodetic System 1984 (EPSG:4326) is widely implemented in GNSS devices, such as phones, drones as well as survey grade equipment. While this could be considered a most absolute system to work with, the polar coordinates with longitude, latitude and altitude are experienced to be impractical as well as incompatible with customary practice and software packages, mostly using cartesian coordinates. Therefore, it makes sense to apply UTM (Universal Transverse Mercator) coordinate systems, projecting 6° meridian strips into a cartesian coordinate system (i.e., EPSG:25833 for most parts of Europe). A good overview of available coordinate reference systems, their transformations as well as areas of use can be found through the open-source web service EPSG.io (Klokan Technologies GmbH, n.d.).
Through still covering a large surface area within their reference frame, coordinate values withing UTM projection can have up to seven figures plus decimal. Unfortunately, some software packages -especially CAD and BIM software have issues with managing the placement of data this far from the origin, resulting in display errors or even software crashes. Therefore, it is usually necessary to introduce yet another coordinate reference system, a locally levelled project coordinate system with its origin within or close to the object of interest.
To follow through with the basic principle Position from chapter 1.3, there are certainly a wider range of coordinate systems involved, as each positioning -although in a simplified way described as such -actually involves position and orientation, therefore having its own object coordinate system. As each individual laserscan, photo, 2D vector section, … is defined by its own point of origin and orientation, it is understood as part of the integrative documentation workflow to reference as many of these individual object systems to the common coordinate reference frame using a rigid body transformation (see Milosavljevic, 2004) described by a 4x4 transformation matrix.

Positioning
A principal component in the integrative documentation workflow to achieve positioning of individual objects and observations is the wide range implementation of GNSS position. Triangulated through a signal of time and current position from at least four satellites, a geoposition of the receiver is calculated. As only relying on this signal will only be able to produce positional accuracies of several meters, the introduction of correction data into the process is necessary. Correction data can be applied in real time (RTK -real-time kinematic) or in a postprocess phase (PPK -post-processes kinematic). As one of the main principles for this workflow is defined as the demand to design processes as real-time as possible the obvious choice (if possible) here should be RTK.
Correction data is generated by base stations on the earth surface that are constantly comparing their known position with GNSS positioning, therefore calculating the deviation or residual between the two. Using an interpolation of the known residuals of neighbouring stations, the correction data is calculated for a specific receiver by a provider. The information is then transmitted via internet connection using a RTCM (Radio Technical Commission for Maritime Services) format via an NTRIP (Networked Transport of RTCM via Internet Protocol) stream to be received in real-time or for a given time frame for post-processing. Figure 1 shows the exemplary calculation of residuals for a base station in Austria. While this correction data can be obtained by a (often paid) provider, correction data can also be calculated on-site by a sperate GNSS-Antenna serving as base station as well as a known position. The transmission of correction data can be done in a remarkably comparable way within a local network, although setup might be more demanding and prone to errors. Applying correction data to GNSS positioning can lead to accuracies of up to 0.02m.

Laserscanning
The application of 3D laserscanning in the process of building documentation is quite common and universally used today. In recent years devices have become ever more independent in the automatic positioning and so-called registration of scans that besides choosing the right settings for the right job, their operation consists of carrying the device from one position to another. At our department, a laserscanner of the Austrian manufacturer RIEGL Laser Measurement Systems with the typification VZ 400i is at use that promotes a wide range of processes implemented within the integrative documentation workflow. On one hand, the scanner is equipped with a GNSS RTK antenna that stores RTK precision positioning with every scan with satellite reception. Through fixing a state-of-the-art mirrorless Sony ILCE-7RM4 camera with a calibrated 10mm wide angle lens it is possible to not only RGB colour the resulting pointcloud but further include images taken at every scanposition (6 images per position) in raw format, fully include them into the photo processing workflow with colorgrading and whitebalancing. This leads to a very coherent colouring scheme throughout different means of documentation and particularly good, combined utilization.
Another important aspect is the onboard registration within the device. The newest generation of scanners from RIEGL are equipped with a secondary CPU, which enables on-board registration, georeferencing and analysis executed parallel to the surveying process. While the onboard algorithms can register scans with each other in the same speed as scans are captured (plus a certain delay) in most cases, there is no way to manually support the process in case of failure.
Through the implementation of accessing scanner functions using onboard python scripts as well as a range of options for network connectivity this model, as well as the hopefully soon to be applied VZ 600i, promises an even deeper integration into an interactive survey workflow. Future research is required. Aside from this, the scanner is of quick scan speed and accuracy and manages to capture images parallel to the scanning progressnevertheless these factors can be read from a product sheet and will not be further elaborated within this paper.

Photography
Taking photographs plays a vital role within building documentation for obvious reasons. Since the wide-ranging introduction of processing photogrammetric models from images, this might have even increased. Nevertheless, the collections of images taken on site are often the ones in most danger to just become that -a collection of images. Sorting images into specific categories, by area, by use or other can help, but finding "that one image" still often remains a task of skipping through a lot of others you are not looking for.
With the implementation of a REDcatch 3D ImageVector Multi into the documentation workflow, it was possible for our department to assign GNSS RTK positioning with 0.02m +1ppm accuracy and IMU (inertial measurement unit) orientation with 0.2° roll pitch yaw freedom to individual images. This can be especially helpful for images processed into photogrammetric models, as position and orientation are already predefined and supports the alignment, scale and geopositioning of the model. In practice the antenna is placed onto the hot shoe (usually for attaching an external flash) of the camera. Through receiving GNSS positioning via the antenna plus correction data via internet connection a precise location of the camera is calculated and continuously written to a log file. When the trigger of the camera is pressed and an image is taken, an additional line, demarking the time of the trigger, is written to the log file and a later be associated with the image to assign the position.
Despite having such devices available it is impractical to use it for every picture. On the one hand, the device obviously comes at a certain cost that can be considered significant, secondly, it is only practical to be applied when taking a range of pictures and going through the initiation phase just for a handful of pictures is too time consuming and third, it only works with GNSS reception. Resulting from this is an aim to supplement the GNSS supported image capturing and positioning with a photogrammetric image alignment process with position estimation. The outline of this procedure will be presented within the next chapter.

Process automatization
To streamline processes within the documentation workflow, work is being done on implementing automated processes for task that are quite common, standardized and sometimes do not need any user input or intervention. A great tool to do this is CLI (command line interface) scripting within Windows OS. Deploying small scripts can pick up processing tasks semiautomated or even automated.
CLI scripts are dependent on the software they are accessing and follow a certain syntax. Simplified it can be said that CLI can access some of the commands that are usually entered within a GUI (graphic user interface). While some software has a deep integration of CLI and allows all commands to be accessed through this, other packages only offer a limited collection. Sometimes -as is the case with commands within the RIEGL laserscanning software RiScan -it is not intended to be used via CLI, but observing the processes of the software and going through the log file as well es configuration files, the syntax of individual small programs deployed within the larger software package can be understood and called from a command line as well. The operations within a script can either be applied to a specific file, all files of a certain folder, files manually dragged and dropped onto the script by a user. Stored within a .bat file, CLI scripts can be applied to files that are dragged and dropped onto the script as well as run as standalone instance.
One software package deployed within the integrative building documentation workflow is remarkably well accessible through CLI and appears promising to be used for the full automatization. Command Line Scripting within the photogrammetric software RealityCapture can on the one hand be used to apply standardized processing settings and workflows for photogrammetric models to images without the need of an operator standing by. Even further a process is being evaluated that is checking a specific folder in certain interval of time for new images and then runs these images through photogrammetric alignment. In combination with FTP (file transfer protocol) upload of images directly from the camera, as is implemented within the newer mirrorless Sony ILCE7 cameras, this allows a real time preprocessing the positional relation of images to each other and exporting this approximated position to a separate file next to the image. This has already been assessed within a testing environment and proved to be working.
Combining this with GNSS RTK positioning of images as explained in the previous chapter 3.4, as well as possibly with the continuous FTP upload of laserscans with scan-images could allow the assembly of a preliminary real-time documentation model within a centralized processor. In doing so there would further be a centralized data management and processing instance -as mapped out in the workflow diagram -that is able to provide information about the current quantitative and qualitative progress of the documentation effort. While this is in the realm of possibilities, singular components of this process currently do not yet allow this to be laid out in that way as for example the log file of GNSS antenna is not readable while it is still being written, therefore cannot be accessed in real-time (yet). Research efforts are being taken to enable these new possibilities (compare Messaoudi et al., 2018;Pamart et al., 2019aPamart et al., , 2019b.

Processor
To achieve beforementioned automatization of processes a centralized data management and processing instance needs to be introduced. To reduce the amount of data transfer necessary in real-time, it is considered beneficial that some processing is done on-board the individual devices, i.e., the registration of laserscan.
Nevertheless, there should be a central unit able to interlink data from individual devices and provide real-time feedback to the operators. Both installing this device locally on site or at a remote destination have their advantages and disadvantages that need to be considered before setting up the system.
Having a local device removes the constrain of needing to have a fast and stable internet connection, as is often not available at cultural heritage sites, especially remote ones. Further, having the central unit at hand gives the opportunity to intervene into processes much more easily and maybe even further reduce the demand for wireless data transmission through periodically plugging in the device and transferring data via cable -likely with faster transfer speed. Installing a device remotely, so to say "at the office" could reduce the necessity to have additional hardware and the need to carry yet another device to the site of investigation. Further, transferring data as quickly as possible could also be seen as form of real-time backup, understandably beneficial. Nevertheless, even when relying on future network protocols speeding up network traffic, the substantial amounts of data produced at a surveying campaign would certainly put a strain on any network transmission.
Either being local or remote, with automated processes deployed at a centralized data management and processing instance, information could be provided to mixed reality interfaces giving qualitative and quantitative real-time feedback on the surveying process and could allow research to already start gaining a fuller understanding of the site with the data captured in time. At last, the centralized data management could be able to index data on capture and create comprehensive metadata and paradata on the surveying process just in time, enshrining a better understanding of genesis and composition of the digital assets right at their point of origin.

INFORMATION MODEL INTERACTION
The following chapter aims to outline and discuss how mixed reality devices could introduce a new perspective of interacting with a digital model on site. Previous chapters focussed on presenting the so-called integrative documentation workflow that is, following four guiding principles presented in chapters 1.3 to 1.6, considered the basis for mixed-reality interaction.
An augmented reality tablet with GNSS RTK positioning, as is already imprinted within the layout of workflow is considered one of the main elements in focus for providing interaction with the digital model. Nevertheless, the next chapters will further try to introduce various other possible interface and how the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy application of the integrative workflow is designed to foster such. While all the presented aspects are under investigation, they will only be presented to limited extend. The complexity of the matter as well as the unresolved and ongoing development of this component calls for presenting it at a later time.

RTK AR tablet interface
To interactively engage with a digital model, while being at the physical parent object on-site, augmented reality technology is believed to be the most promising interface at hand. In this, the physical world -as seen through glasses or the live-view of a handheld tablet with camera -is augmented with digital elements. This can be any piece of information, models, analytical layers, basically any digital data. While augmented reality glasses might be more intuitive in usage, as the change of perspective is defined by the movement of the head, an augmented reality tablet I considered more beneficial for the proposed application, as it appears to be more capable in not just reviewing but contributing information.
Cornerstone of any mixed reality interface is the so-called "tracking." Through the combination of a series of sensors, devices are able to detect movement and/or their position within a space or in reference to a certain physical object, to then visualize digital content that was beforehand put into positional relation with the tracking "target." To be independent of any predefined object of reference, a tracking hierarchy was determined that is considered to go hand in hand very well with surveying technology. Key element in this is -again -GNSS RTK positioning. With attaching a GNSS antenna, much like the one used on top of the camera, it is possible to define the geoposition of the tablet device in up 0.02m accuracy. This position is used to define to position of the "scene" or to use previous terminology the project coordinate system. Within this scene a combined SLAM (simultaneous localisation and mapping) algorithm using visual information from the camera and gyro-sensors and magnetic sensors inside the device is used to track the movement of the tablet. Once the tracking is established digital objects can be visualised and inserted into the scene, as is considered the basic principle of interaction with the digital model.
While only relying on GNSS tracking would limit to areas with satellite reception (no indoor spaces), the application of a visual SLAM algorithm could extend the area of possible application into indoor spaces as well. Aside from the forementioned tracking, there are options to possibly increase accuracy and stability. Applying so-called area targets, a conversion of laserscans to track physical substance or image targets, predefined 2D reference patterns establishing a coordinate reference, could be beneficial, but would enforce a timely succession of capturing and interaction as laserscans need to be capture or image targets need to be defined in position before use within the tracking algorithm. Nevertheless, it should be considered to maybe introduce these additional tracking targets into the process along the way to achieve a more rigid tracking environment.

Geometry capture and visualisation
A key advantage in introducing augmented reality into the workflow is the possibility of giving the surveying team comprehensive feedback on the already acquired, otherwise quite intangible, digital materials. While on the one hand this could of quantitative nature -how many photos / scans and where -it is imagined that through the introduction of automated processes this could further lead to qualitative feedback. In this a collection of laserscans could be analysed to visualize coverage or point density along the object. Further the alignment of images could give feedback on how well the processing of the photogrammetric model might work out and if some areas need to be densified.
Of course, there is no need to visualize the digital representation of the object that one has right in front of her/him but providing feedback of what is already collected and what qualities can be expected can certainly be helpful in understanding the progress of the campaign and reduce the risk of insufficiencies.

Information input and Interactive mapping
Besides giving feedback on the surveying progress, the AR device could further be used to input sematic information into the model in accordance with the four main principles. Once tracking is established the type of input can be various, from plain text notes, photos, small sketches, voice input, reference to URL, DOI or other types of identifiers, … Images taken with the device could be used to trace observations with area extension on the surface of the object. In the workflow diagram this is included as a so-called mapping task.
The documentation procedure would differ here between frontend (= user interface) and backend (= calculations in the background). On the frontend the user could take an image, use a polygonal demarcation on the tablet screen to define an area of consistent characteristics, i.e., an area with the same building material, selects a certain predefined or new layer definition for it and transmits his input for processing. On the backend the image is taken, matched to the collection of images and scans to then possibly show demarcations already in vicinity or within the visible area. After this the polygonal input of the user could be projected onto the measured surface of the object using the depth map calculated for the picture and store the area with a certain attribute to the central storage.

Autonomous survey
Beforementioned interactive documentation tools would already revolutionize the documentation process as we know it as there I presently now research known that implemented such procedures. Nevertheless, once reconfiguring a survey workflow into a centralized and interlinked data environment there are still more implementations conceivable. While currently still under investigation by the Research Unit Virtual and Augmented Reality with Prof. Hannes Kaufmann at TU Wien in Vienna, Austria, continuous progress is made on the deployment of an autonomous robotic platform for surveying purposes. As this is not natively available without human guidance or predefined paths, the robotic dog SPOT is currently being deployed on autonomous exploration using a 3D SLAM algorithm. (Mittermair, 2022). This should subsequently lead to the ability of autonomously exploring areas and use the RIEGL laserscanner mounted on top of the robotic platform to acquire survey-grad scans that are later used to refine mapped surrounding within the navigation system of SPOT.

Browser-based interaction
Another implementation that appears promising within the prospect of a completed integrative documentation workflow is the real-time interaction through a web-based platform. It is envisioned that surveyed data is possible to be streamed from a webserver to access the last current data and be able to contribute and annotate or just observe the progress of the research from a remote destination.

Mixed reality lab
A more immersive experience of remote interaction with the information model could be considered the access through a socalled mixed reality lab or similar institution. As a combined effort and initiative by a range of faculties and departments at TU Wien, a new mixed reality lab is being set up within the next years. This new laboratory will be set up with a 8k LED cave, multi-character tracking throughout most of the area of the room, various head-mounted AR and VR displays, holographic screens as well as a novel 192-speaker 3D audio system, specifically developed for the TU Wien Mixed Reality Lab. Aside from this there is a 8k stereo projection LED wall available, called DAVIS (Data Visualisation Space), and available for teaching purposes and accessible to all departments.
Such installations could be considered the next step of an immersive platform to remotely participating in a surveying and researching process. Through being able to stream data from centralized storage into a Mixed Reality Lab or Data Visualisation Space could give the opportunity to invite researchers not currently on site to participate within the scientific discourse at the same time of the surveying campaign and include their contribution right away into the coordinatebased surveying model.

CONCLUSION
The integrative documentation workflow laid out as a whole and within chapters about individual components introduced is believed to suggest a novel and more comprehensive approach into the surveying process of a cultural heritage site. Mapping the workflow within the procedural diagram is believed to not only be helpful in the development and refinement of such a workflow but could be understood as an organisational chart documenting the methodological approach of deploying different surveying devices and processing resulting data through different operations. In this it is not only a mean of researching the process but could suggest a procedure to document descriptive paradata "documentation of the evaluative, analytical, deductive, interpretative, and creative decisions made in the course of computer-based visualisation (Hugh Denard and King's College London, 2009, p. 8 f)." into a computer-generated graphic that represents the as-is state of the object at a certain time, with a certain range of accuracy and verity as closely and objectively as possible (compare Denard, 2012, p. 67).
While there are still processes to be solved and simplified it is believed that the process as a whole could work and given a certain time and research will be able to be applied. Some factors within the entire process such as accuracies through the application of GNSS RTK positioning, augmented reality visual tracking and in general the practicality of AR devices in the surveying process can still be up for discussion but are deemed to be worth investigated.
Hopefully within the closer future there will be a chance to not only evaluate single components for functionality, but to deploy larger segments or even the workflow as a whole to gain insights on advantages and disadvantages of certain decisions and solutions. Reducing the number of "moving parts" that could lead to failure of the process, as well as solving or bypassing "bottlenecks" within, like data transmission rates, GNSS reception, RTK availability will help streamline the process. Standardisation of exchange formats, even for abstract geometries like boxes, planes and polylines and a wider range of implementation of rigid body transformation and CLI scripting within software packages could further lead to greater interoperability between different hardware and software and further simplify the process.
All aside thinking of the new possibilities of interaction between physical and digital environment and how close to being able to grasp these solutions we have become in recent years is an exciting outlook onto new things that can be developed and implemented within future research and how transformative the technologies might be.