A SEMI-AUTOMATED METHODOLOGY FOR 3D IMAGING OF LARGE VAULTED CEILING PAINTINGS

: 3D imaging of large painted and vaulted historic ceilings offers many challenges, including those due to complex small and large-scale geometry, environmental conditions and limited access. In this paper we introduce CHAPI (Cultural Heritage Automated Photogrammetric Imaging), a low cost solution to these problems which provides an efficient, semi-automated method of capturing large vaulted ceiling paintings in high detail, with a consistent photogrammetric network and in limited time. We will present and examine two different case studies from the Plafond3D project, the baroque ceiling paintings of the Schloss Rheinsberg Spiegelsaal (mirror room) and the Ansbach Residence Festsaal (great hall/ballroom). The paper will examine the particular challenges of capturing large painted areas with accurate colour and geometric reproduction, and suggest how the photogrammetric recording and reconstruction process of ceilings can be optimised for large rooms. This paper also gives technical specifications and Open Access Code for the CHAPI build.


INTRODUCTION
Today, photogrammetric surveys are a common tool in the documentation of heritage sites. It is a particularly useful technique when capturing large painted surfaces as high detail and colour fidelity can be achieved through careful photography (Bernal et al, 2021), and even more so in cases such as those of baroque, vaulted ceilings with stucco-work where both the large and small-scale 3D geometry is as important as the 2D texture (Hoppe et al, 2020). However, despite its utility and increasing ubiquity, there are challenges inherent in imaging large scale painted heritage, including (but not limited to) lighting (availability of natural or artificial illumination), access (both in terms of the physical situation of the object in question, and the often limited amount of time available to complete the imaging), and level of detail requiredthe size of the smallest features that need to be recorded compared to the size of the overall object.
In this paper we introduce CHAPI (Cultural Heritage Automated Photogrammetric Imager), a low cost solution which addresses these challenges, and provides a method for capturing large ceiling paintings in high detail, with a consistent photogrammetric network and in limited time. The device was originally designed to answer the particular challenges of one difficult environment, but has since proved invaluable in other situations. We believe it can be adapted for, and used in, a variety of use-cases beyond those of its initial conception.
We will present and examine two case studies from the Plafond3D project, ceiling paintings from Schloss Rheinsberg and the Ansbach Residence.

THE PLAFOND 3D PROJECT
The three year international and interdisciplinary Plafond 3D project, co-funded by two national research associations, the Deutsche Forschungsgemeinschaft (DFG) and the Agence Nationale de Recherche (ANR), aims to study the "historical, cultural, formal, and technical phenomenon" (Plafond-3D, 2022) of 17th and 18th century French and German painted, vaulted ceilings. Over the course of the project, six sites (three in France, three in Germany) will be digitised in 3D. The 3D model outputs will document the sites and be used in art historical, restoration-conservation and architectural research, and to disseminate the objects of cultural heritage to both researchers and the general public (Pattee 2023).

Residenz Ansbach, Bavaria
The Ansbach Residence, constructed during the 16 th and 17 th centuries was originally the seat of the Margrave of Brandenburg-Ansbach and is currently the administrative seat of the government of Middle Franconia. The vaulted Festsaal (Great Hall), approximately 250m 2 , is decorated with a fresco by the celebrated Italian artist Carlo Carlone (1686-1775). The ceiling is currently undergoing major restoration work by the Bavarian Administration of Palaces, Gardens and Lakes, and thus a scaffolding platform covers the entire area of the hall, with an approximate 2m clearance to the ceiling (Figure 1). The scaffolding enabled the imaging of the fresco in high resolution from close range and the digital models have already begun to generate new insights into the working practices of one of the most important fresco workshops in the eighteenth-century.

Schloss Rheinsberg
The Rheinsberg Palace is part of the Prussian Palaces and Gardens Foundation Berlin-Brandenburg (SPSG) and was the home of crown prince Frederick (later Frederick II, also Frederick the Great) from 1736 until his accession in 1740. It is decorated with the first known ceiling frescos by French court artist Antoine Pesne (1683-1757), including the Spiegelsaal (mirror room), approx. 130m2 ( Figure 2). As many of Pesne's later frescos for Frederick II were destroyed by bombing during WW2, this style of ceiling painting can be studied in its earliest phase only in Rheinsberg.

CHALLENGES
Although both ceilings presented their own particular challenges for photogrammetry, the CHAPI system was developed specifically for use in the Ansbach project, with the understanding it could be re-used in subsequent imaging campaigns.
The situation of the Festsaal ceiling in the Ansbach Residence presented both opportunities and challenges. The presence of the scaffolding platform covering the entire area of the room entailed an average clearance to the ceiling of just two metres, and an almost complete absence of natural light. Due to the ongoing restoration work, we had limited access to the scaffolding, meaning the entire ceiling had to be imaged in what was effectively a single day. During an initial visit in June 2021, at which time the platform covered approximately half of the room, imaging of small areas in detail was conducted, and tests were carried out to determine the optimal method for photographing the entire ceiling. As it was impossible to consistently illuminate the entire area, it was decided a ring flash would be the best option for artificial illumination, due to its consistency, mobility and the minimisation of shadows. The required outputs for the Plafond 3D project required a scaled, textured, three dimensional model of the ceiling from which orthophotos with a resolution of one millimetre or better could be generated. With these requirements, and due to the limited throw of the flash and the short distance between the camera and ceiling, the images taken had a relatively small field of view (Table 1), and thus, to get a usable network with sufficient overlap (80%), it was estimated that imaging the ceiling alone would require more than 1500 pictures, with a controlled spacing (≈30cm vertically and ≈50cm horizontally). Thus, the particular requirements of the Ansbach project directly led to the development of our imaging system. Rheinsberg's Spiegelsaal provided a more conventional environment for photogrammetry, with a ceiling height of approximately 5 m and abundant (if inconsistent) natural light. However, the room is, as the name implies, decorated with many mirrors and was lit on two sides (east and west) by eight full length windows. The light varied considerably throughout the day and, due to changing cloud cover, even between individual images. These types of issues are, however, common to many photogrammetry projects, and do not require special solutions. However, access to the room was again limitedthe palace is open to the public and the Spiegelsaal was specifically closed for one day to accommodate our imaging. The requirements for the project  were similar, with the need for high resolution imaging entailing a large number of images taken with a consistent overlap. In this case, the use of CHAPI again ensured the necessary number of images could be taken in a controlled fashion within the limited time available.

METHODS AND DEVELOPMENT OF CHAPI
With the particular challenges of Ansbach in mind, we required: a) a stable, mobile platform on which a camera could be mounted in multiple orientations b) a method of automatically measuring distance with a resolution sufficient for a dense photogrammetric network c) a method of automatically triggering a camera tethered to a laptop d) a low-cost solution that was also easy to assemble/disassemble for transport.

Construction
Our solution is CHAPI, a device constructed on a fourwheeled, collapsible walking frame (Figure 2). The frame was chosen as many of the requirements for an old person's walker also match our requirements: it is lightweight (8.9kg) but sturdy (max. payload 130kg), and the wide wheel base provides both stability and a smooth ride. The tires are solid Polyurethane rubber and are suitable for both indoor and outdoor use, while two brakes operated with handlebar mounted grips provide further stability for longer-exposure photographs. In addition, the handlebar height is easily adjustable and the device can be easily folded into a small profile for transportation. Our model (Meyra, 2023) was donated, but they can be obtained second-hand for around 50-100€.
Standard aluminium construction profiles (item24, 2023) provide a flexible, configurable structure on which to mount the camera (we used 40x40mm I-Type profiles). They are attached to the body of the walking frame with quick release sliders ( Figure 3) and 90deg brackets. The profiles can be attached using a single allen key in multiple threedimensional configurations in a matter of minutes. The camera can be attached directly to the profiles using a cheese plate, or via a standard tripod head to allow more flexibility.
Distance is measured using six small rare earth magnets attached to one of the rear wheels. A Hall sensor module attached to the walker's frame sends a signal every time a magnet passes. The wheel circumference is 60cm, allowing distances to be measured with a resolution of 10cm.
A wooden board is attached to the walker's seat via four bolts, providing space for the control box and laptop.
The whole device can be quickly assembled/disassembled for easy transportation, and the whole set-up (including the electronics, but not the laptop and camera) cost under 300€.

Electronics
The device is controlled by an ESP32 microcontroller (6€) on a 30pin ESP32 development board running ESP Home (ESPHome, 2023), and powered via a micro USB port connected to a power bank or laptop. Code is flashed to the controller via the board's built-in wifi. The electronics are mounted within a laser-cut wooden box created using boxes.py (boxes.py, 2023). (Figure 5) Figure 3. CHAPI, and (below) disassembled for transport.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy

Inputs:
The controller has multiple inputs; three buttons, a switch, a rotary dial and a socket for attaching the Hall sensor. One of the buttons is used to trigger the camera, the other to switch between manual and automatic modes. Two of the buttons have associated audio jacks, allowing for remote triggering (for example, the camera can also be triggered by a button attached to the handlebars which is connected to the controller via a standard 3.5mm audio plug). The rotary dial is used to set the required distance measurement, and the third button resets the current distance measured to zero. The switch toggles between two modessetting the required distance and live operation. The device can also be controlled and monitored via a web interface.

Outputs:
The controller outputs via a variety of modules: an LED display, a passive buzzer and two relays. The LED display allows you to set the required distance, and displays the current distance travelled. The sound board provides alerts on boot-up, errors and most importantly, when the required distance has been travelled. One relay is currently unused, the other is connected to a Raspberry Pi Micro which is attached via USB to a laptop. Our camera is tethered to a laptop running Nikon Capture One software. The Raspberry Pi mimics a USB keyboard, and when triggered, sends the key-code for the software's shutter release keyboard short-cut.

Code:
The code for operating the device is written for ESPHome (esphome, 2023) using simple YAML (yaml, 2023). All code and more detailed documentation for constructing CHAPI and the electronics is available on GitHub, Gitlab and Zenodo (Repositories 2023).

OPERATION
The required spacing between images (in multiples of 10 cms) is set via the rotary dial and shown on the LED display. Once set, the toggle switch sets the display to show the current distance travelled.
The device can be operated in two modes, manual and automatic (selected with one of the buttons). In the former, the user is alerted by an audible tone when the programmed distance is achieved, at which point they can manually trigger the camera via the buttons on the control box or handlebar. When exposure times are sufficiently short, automode can be used, with the camera triggering automatically every x cm. For example, in Ansbach, the use of the flash and therefore fast shutter speeds (1/200) enabled the use of auto mode, so CHAPI was pushed forwards at a steady pace and a picture captured every 30cm. There was no observable degradation in image quality using this method. In Rheinsberg, where only natural light was available, shutter speeds in the order of half-to-one second were necessary, thus semi-automatic mode was used. This method was still considerably easier and quicker, and resulted in a more consistent network than, for instance, manually moving a   Light (for navigation) was provided by an LED lamp.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy tripod a set distance each time. Strings taped to the floor, separated by the distance required for an 80% image overlap, provide a guide for the operator.
In both Ansbach and Rheinsberg, an initial set of images orthogonal to the ceiling were taken. In Ansbach further rings of images at 45° and 90° were obtained (also using CHAPI) in order to model the vaulting. A further set of images (not using CHAPI) were taken with a 12mm wide angle lens and using the available light (several fluorescent tubes and four Scangrip LED lights). These images were used in the SfM processing, but not for texturing. In Rheinsberg CHAPI was again used to capture additional rings of images at 30° and 60° with a 50mm lens in order to model the vaulting, as well as additional images to ensure that areas of the ceiling potentially obscured by the four chandeliers had sufficient coverage. Again, a set of images were taken with a tripod and 12mm lens to image the rest of the room and improve overall geometry.

PRELIMINARY IMAGING RESULTS
The requirements of the Plafond 3D project include a digitised model that can aid in the definition of materials and methods used for the production of the ceiling paintings. As such, the desired output was an accurate, scaled geometrical 3D model of the ceilings (figure 8) with a high resolution recording of the texture which could be exported as an orthophoto and used as a resource for both art historical research and for conservation/restoration purposes. To answer the art historical questions, it was requested that features in the order of 1 mm could be detected in both the 3D model and orthophoto imagery. Therefore, a Ground Sampling Distance (GSD) of at most .5 mm and ideally .25 mm would be required (Verhoeven, 2016).
In Ansbach, using CHAPI and a Nikon D850 in cropped sensor mode with an 18mm lens, it was possible to capture approximately 1900 images in just over five hours, of which 1400 were orthogonal images of the ceiling itself. The GSD of the ceiling imagery was ≈0.35mm. In Rheinsberg, around 1500 images were taken with a 105 mm full frame lens in four hours (900 of the ceiling), achieving a GSD of ≈0.2mm. These results are summarised in Table 1.
The images were processed in Agisoft Metashape 1.8 alongside scans from a Leica BLK terrestrial laser scanner (Leica Geosystems, 2023) to provide scale and improve overall geometry (Bruno et al, 2022). The geometrical and textural resolution in both models in fact exceeds the project requirements, and is sufficient to model the stucco and architectural features of both ceilings, identify cracks and other damages, and in some cases see artist's marks and even individual brush strokes (figure 9). (Separate photogrammetric campaigns were conducted on small areas in Ansbach to more fully reveal these features.)  Processing is ongoing, but an orthophoto of the Ansbach ceiling painting generated in Agisoft Metashape photogrammetric software has a resolution of 50765 x 32990 pixels providing a physical resolution of approximately 0.35mm/pixel or ≈72dpi. The full resolution 3D model has also revealed potential new insights into the artist's methods.

CONCLUSION AND FUTURE WORK
We have shown that CHAPI can improve the results of photogrammetric campaigns of large ceiling paintings by both improving the consistency and density of a photogrammetric network by increasing the number of images possible in a certain amount of time. We found CHAPI to be an invaluable tool in our baroque ceiling projects, without which it is doubtful whether we could have achieved the required result in the limited time available. But beyond this particular project we believe the concept can also be used and adapted for a multitude of room and building-scale photogrammetry projects where large amounts of images must be taken at a consistent spacing. We are releasing both the design and software as an open source project and invite users to test, adapt and configure it for their own particular use-cases.
Our own future work includes increasing CHAPI's linear resolution, adding the ability to measure distance in both forward and reverse directions, and replacing the internal electronics with a custom circuit board, in order to facilitate future expansion. We aim to provide more options for attaching the profiles to the walking frame in order to provide increased flexibility for the camera mounting, allow the mounting of multiple cameras and to investigate alternative lighting solutions.