AR EXPLORATION OF THE PERSPECTIVE/ARCHITECTURAL SPACE PAINTED BY PERUGINO

: The perspective study of one of the panels from the cycle of San Bernardino attributed to Perugino has revealed a contracted space in depth, represented three-dimensionally by the objective model . In contrast, the ideal model eliminates the contraction and maintains the proportions dictated by the architectural interpretation. The contribution presents an advancement in the perspective study aimed at developing an augmented reality (AR) application. It is designed to be experienced at the museum housing the artwork, serving as an in-depth exploration that explains to visitors the nature of the perspective space and the architectural space represented. The case study of the painting by Perugino becomes an opportunity to define and experiment with a workflow aimed at creating AR applications focused on the visualisation of architectural models derived from the perspective restitution of pictorial works. The description of the adopted method focuses on the following aspects: the use of the pictorial image as an AR marker and the relationship between the perspective picture plane and the 3D model; the optimisation of 3D models for AR visualisation purposes; the issue of the landscape; scene lighting; and the user interface for multilayer interaction.


INTRODUCTION
In 2023, the quincentenary of the death of Pietro Vannucci, known as Perugino, was commemorated.In the context of the celebrations and cultural fervour surrounding the painter's figure, perspective studies were initiated on the painting cycle depicting the Stories of San Bernardino, housed at the Galleria Nazionale dell'Umbria (GNU) in Perugia (Fig. 1).These are eight panel paintings illustrating some miracles of the saint, six of which are characterised by a predominant architectural context in the composition.At the beginning of the last century, two of these panels were attributed to the hand of Perugino, who was also considered the coordinator of the group of artists who executed the cycle in 1473 (Venturi, 1913).The panel San Bernardino heals the daughter of Giovanni Petrazio da Rieti from an ulcer (Fig. 2) is one of the two attributed to Perugino, and perspective studies were conducted on this particular piece (Baglioni and Menconero, 2023b).From the point of view of the history of representation, Perugino's work is very interesting because it falls within a period of change when perspective was evolving from an empirical workshop practice to a true science based on repeatable and measurable theoretical principles (Gay, 2014).
The study of the perspective setting of Perugino's work helps formulate hypotheses about painted architecture, whether perspective is merely an evocation of those spaces achieved through artistic practice or if it is the result of a conscious scientific projective process (Baglioni et al., 2016).
A brief summary of the results of the perspective study is provided in paragraph 3 because it is closely related to the content proposed in the augmented reality (AR) application.
The present contribution has two main objectives: 1) on the one hand, to advance research on Perugino's work aimed at disseminating the results of the perspective study so that they are accessible to a non-specialist audience; 2) on the other hand, to define a workflow aimed at creating AR applications focused on the visualisation of architectural models derived from the perspective restitution of pictorial works.
The proposed AR app is designed to be experienced at the museum hosting the artwork as an in-depth exploration explaining the nature of the represented perspective and architectural space.In particular, the first term (perspective space) refers to the 3D space that observes the perspective principles, while in the second case (architectural space), the architecture dictates the rules.

RELATED WORKS ON AR IN PAINTING
Studies focused on AR are numerous and cover various aspects, from technological to applicative ones.A recent and in-depth state of the art on AR in architecture is presented by Russo (2021), who explores various parts of the AR built domain: workflow design to digital content creation, system creation to platform definition, and user experience evaluation.AR has strong potential in the field of enhancing cultural heritage thanks to its ability to overlay virtual elements onto the real world (Fanini et al., 2023).
The case of painted architecture, or painting in general (Zhang and Wang, 2023), is emblematic because the painted surface can define itself as the marker for activating 'augmented' content (image-based tracking system) much more simply and effectively than other more complex types (3D markers or markerless).the Covid-19 pandemic restrictions, including some based on AR technology.Finally, the Master of Campo di Giove4 by Altair4 Multimedia was an AR experience at the eponymous temporary exhibition at the Museo Nazionale d'Abruzzo.In this case, AR was used for scientific purposes, showing visitors various multispectral diagnostic images of some displayed panels by framing them with their own devices.After a brief examination of examples of AR applications in museums, the related works examined in this paragraph focus on AR applied to the context of pictorial art, distinguishing between 1) studies where technology allows 'augmenting' various information about the painting or visualising artworks in different contexts; and 2) investigations oriented to the specific case of perspective pictorial works, the perspective restitution and visualisation of the represented space, as in the case of the present study on the painted architecture by Perugino.The first group includes works in which AR becomes a means to inform viewers about the content of depictions in the form of pictures, graphics, texts, videos, and audio, activated on-site with their devices by scanning the work directly or an associated tag (Bernardello et al., 2020;Pierdicca et al., 2015).Also, within the first group is a study investigating the potential interaction between AR and social networks by creating Instagram filters, allowing the visualisation of pictures and texts, and sharing photos or videos of the experience through the social network (Bernardello et al., 2022).The second group includes studies on AR applied to the visualisation of architecture represented in perspective works, be they canvases, inlays, frescoes, or engravings.All works of this kind, including the one presented here, are based on an initial perspective analysis that allows for extracting the fundamental elements through which to reconstruct the represented space in three dimensions.The first examined research (Sdegno et al. 2015) presents the results of the restitution of the architectural space painted by Paolo Veronese in the large canvas The Fest in the House of Levi (1573) conceived for the refectory of the Basilica SS.Giovanni and Paolo in Venice, currently preserved at the Gallerie dell'Accademia.The AR typology developed is activated with a QR-code marker, and particular attention was given to the graphic rendering of the architectural scene and characters.The second study (Carlevaris et al., 2021) concerns reflections arising from the application of AR to the perspective architecture of a wooden inlay, a work by Damiano Zambelli, in the choir of the Basilica of San Domenico in Bologna (around 1530).In particular, AR is activated by the inlay itself (imagebased tracking system), and considerations include: 1) creating the window effect, i.e., the 3D model must be perceived beyond the image/marker; 2) ensuring that beyond the 'window' on the 'augmented' space, elements of the real space are no longer visible; 3) defining a frame that delimits the real space from the virtual space so that the perspective depth effect activated with AR is more effective.The last research examined (Fasolo et al., 2022) concerns the apse of the church of Sant'Ignazio in Rome, a work by Andrea Pozzo.This apse appears in a perspective engraving inside Pozzo's treatise (Perspectiva pictorum et architectorum, 1693-1700), which is used as a marker to activate the display of a portion of the 3D architectural model of the church, showing where the perspective picture plane is located relative to the architecture.Also visible are some graphic elements derived from the study of the perspective setting of the engraving.
Beyond the originality of the research object, the main advancement of our study compared to the state of the art regarding AR technology applied to paintings is proposing a multilayer application where, from a single image/marker, it is possible to switch through different 3D architectural models and other 'augmented' content useful for understanding the results of the perspective study of the artwork.

PERSPECTIVE STUDY SUMMARY AND RESULTS
Perspective as a representation method presupposes a projection operation from a centre of projection (point of view) towards the object and the sectioning of visual rays onto the picture plane (the plane where the image is formed).These assumptions create a biunivocal relationship between the represented object and its perspective image.Knowing the perspective image makes it possible to trace the shape of the object that generated it through perspective restitution, known as the "inverse problem of perspective" (Paris, 2000).In the case of the panel painted by Perugino, it was possible to derive the 3D model of the architecture from the drawn structures.The first operation conducted was the 'diplomatic' drawing of the painting, in the philological sense of the term (Migliari, 2016), following the lines drawn by the artist without forcing their convergence at specific vanishing points (Fig. 3).Through the analysis of the perspective setting, especially the vanishing points of the horizontal lines perpendicular to the picture plane, the main elements necessary for the subsequent restitution were identified: the principal point P and the horizon line h.The measurement of the principal distance (MP = PO), and therefore the position of the centre of projection O of the perspective, was derived through oblique lines at 45° (e.g., s') identified in the alternate vertices of the well octagonal base.The ground line g, which indicates the horizontal plane of the floor, was arbitrarily positioned at the lower edge of the depiction, corresponding to the beginning of the decorative frame (Fig. 4).From the perspective theory, we know that the position of the ground line does not affect the perspective but only its dimension.For AR applications, the measurements are not influential, while it is appropriate to have the floor of the 3D scene corresponding to the decorative frame.The perspective restitution of the architecture was based on the position of the picture plane, the horizon, the ground line, and the identified centre of projection.The three-dimensional model thus obtained can be defined as objective because it is congruent with the perspective space, as demonstrated by the view from the centre of projection and the congruence with the perspective of the painting (Fig. 5).However, the restitution reveals that the architecture is subject to contraction, progressively shortening the intercolumniation so much that it alters the round arches, presenting a space that is not regular and isotropic as one would expect.
To this objective model, the ideal model was added (Fig. 5), conceived according to an architectural interpretation in which the intercolumniation and arches are regular.Clearly, the view of this model from the centre of projection presents a different perspective than that of the panel.
The interpretative hypothesis suggests that the artist altered the perspective to emphasise the landscape portion framed by the monumental arch.

Objectives
The painting from the cycle of San Bernardino conceals the dual nature of architectural and perspective space, unveiled through the analysis of perspective.This contradiction is not perceptible through direct observation of the panel, which seems to maintain perfect spatial coherence.
The objectives of the AR application are: 1) to show users the relationship between the painted architecture and the represented three-dimensional space, illustrating the mechanisms of the 'perspective machine'; 2) to reveal the dual nature of the perspective/architectural space (hence the objective and ideal 3D models) hidden within Perugino's painting.
To achieve these objectives, the multilayered AR app has been designed so that from the marker/image of the painting, users can access various contents through buttons on the user interface.The display of the 3D models, both objective and ideal, occurs beyond the painted surface, taking advantage of the window effect created, in part, by the decorative frame on the edge of the panel.In the scene, the protagonists are the architecture and the landscape: the characters have been removed as they were not essential to narrate the results of the perspective study.The architectural models have neutral textures without materials to avoid confusing shapes and allow for a greater focus on spatial aspects.In contrast, the landscape is clearly visible because, along with the architecture, it is a fundamental element for hypothesising the reasons behind the spatial incoherence.

Method
In describing the adopted method for creating the AR app, we focus on some critical aspects: 1.The use of the painted image as an AR marker and the relationship between the perspective picture plane and the 3D model; 2. Optimisation of the 3D models for AR visualisation; 3. The issue of the landscape; 4. Scene lighting; 5. User interface for multilayer interaction.

Image-based marker:
One of the advantages of applying AR to perspective paintings concerns the effective use of the painting as a marker to activate 'augmented' content.In particular, visualising the reconstructed perspective space beyond the image is very interesting, as if the latter were a window onto the painted space.To achieve this effect, the 3D model, the picture plane, and the centre of projection must be reciprocally in a projective relationship.Once the position of the centre of projection in relation to the picture plane is established, the position of the 3D model can vary concerning its size.Specifically, being a conical projection system (point-source projection centre) and needing to ensure the biunivocal relationship between the object and its perspective representation, the model will increase in size as it moves away from the centre of projection.To critically define the size of the 3D model, the height of a human figure is often considered, and the model is scaled accordingly.In our case, however, the scale of the model was defined based on the position of the scene's floor, which was intended to coincide with the lower edge of the representation near the decorative frame.
Transferring the elements of perspective into the realm of AR, the architecture from the perspective restitution becomes the 'augmented' object, the picture plane (painted image) becomes the marker, and the centre of projection marks the position that the AR-activating device must have to ensure that the perspective of the painting and the view on the model coincide (Fig. 6).The device, therefore, replace the eye (point of view or centre of projection) in the projective relationship.Another important aspect that ensures the effectiveness of the window effect concerns the need to display the 'augmented' 3D model only within the boundaries of the painting's fame (Fig. 7).This effect is achieved through the Depth Mask Shader, which prevents any object behind the masked region from being rendered5 .Thus, this shader must be applied to simple planes that are coplanar with the image/marker and surround it sufficiently to 'hide' the underlying 3D architectural model.

3D model optimisation:
The objective and ideal 3D models originate from NURBS modelling as a consequence of perspective restitution6 .The architectural elements were modelled by synthesising the architectural order and decorations into polyhedral geometric elements, focusing attention on the spatial relationship of structures and the perspective layout.
The transition from a NURBS modelling environment to a polygonal mesh visualisation presupposes the tessellation of the model to transform the description of shapes from parametric and continuous terms to lists of coordinates.
In the realm of AR, incorrect tessellation can cause issues with model visualisation due to surface normal inversions (Russo et al., 2019) or poor rendering performance due to the high number of polygons.Although the tessellation operation can be automatically performed by software, it is important to manage the parameters according to the desired result.In the case of models for AR visualisation, the goal is to find the right amount of polygons that ensure a correct display of the model without burdening the rendering calculation.In particular, the number of polygons should ensure a smooth representation of curves, especially along the apparent contour.To achieve this result, tessellation density parameters can be adjusted in areas with curvature while diluting it in flat areas.Additionally, the proportions of tessellation polygons can be managed to avoid them being too thin and elongated, which could lead to issues during rendering calculation (Fig. 8).

Landscape generation:
The 3D models feature a neutral material, without textures, to avoid confusing the observers, allowing them to focus on volumes and spatial aspects.The matter is different regarding the landscape because it plays a significant role in interpreting the artist's perspective choices.
The set goal was twofold: on the one hand, to find a way to display the landscape during the AR experience, and on the other to 'close' the space beyond the architectural models with a scenic backdrop.The proposed strategy involves creating a 'box' composed of four contiguous planes (top, front, left, and right) onto which the texture of a natural landscape is projected, similar to that framed by the monumental arch in the painting (Fig. 9).
The main problem concerns the limited extent of the visible landscape portion in the painting and the need for a much larger texture.To overcome this issue, generative artificial intelligence (AI) was employed, helping fill the texture gaps with images based on the content and artistic style of the landscape visible in the painting.A generative AI image-to-image system called Generative Fill 7 was used, which operates in two ways: 1) through selection and textual description, it modifies the selected picture area consistently with the descriptive prompt; 2) if no prompt is associated with the selection, the system proposes a generation based on the content of the rest of the picture.This tool created a texture that, at an appropriate viewing distance, provided an idea of the landscape surrounding the architectural scene painted by Perugino.
To distinguish between the original parts and those generated by AI in the texture, a different opacity level was used between the two parts.Upon careful observation, which does not disturb the overall view, the transition between the original portion with 7 The software used is Adobe Firefly integrated in Adobe Photoshop 2024.
100% opacity and the rest with lower opacity is noticeable (Fig. 10).To make the AR experience accessible even on less powerful mobile devices, real-time rendering calculations must not be too demanding.Section 4.2.2 emphasised how optimising 3D models can impact the rendering speed, especially concerning the number of polygons in the mesh.Lighting calculation is another important, and sometimes resource-intensive, parameter for real-time rendering.An expedient to lighten and simplify the scene's lighting calculation is achieved through pre-rendering lighting so that the texture of 3D objects records the scene's lighting conditions.Regarding the studied painting, ethereal, shadow-free lighting was desired to simulate the nature of the light depicted by Perugino.To achieve this target, two approaches were combined for the AR scene lighting: 1) ambient occlusion (AO) calculation was pre-rendered and applied as a texture to the 3D models, and 2) a global illumination of scene was used without any form of direct light 8 .

Interactive user interface:
To enable the comparison between the display of the objective model and the ideal one, it was necessary to create a graphical interface for user interaction in the AR app.
Once launched, the app opens the mobile device's camera 9 , and overlay buttons allow users to activate and deactivate the respective contents.The interface was designed so that the five buttons are anchored in the bottom-right corner of the screen, making them easily accessible to the right hand even if holding the device (Fig. 11).The white colour of the text ensures readability in various lighting conditions, facilitating visibility in the dim lighting typical of the exhibition hall inside the gallery.
Figure 11.The user interface of the AR app in portrait and landscape mode, and a detail of the five buttons for interaction (author's elaboration).

Results
The developed AR application allows users to explore the space of the painting attributed to Perugino and understand the relationship between perspective and the represented architecture 10 .By framing the painting with their device, users can access a series of contents through five buttons on the graphical interface (Fig. 12): a. Info, to read a brief description of the scientific results of the perspective study, instructions for using the app, and credits (Fig. 13 8 Texturing of the models with AO pre-rendering was performed in Blender 2.79.The global illumination was set within Unity. 9The AR app was only developed for the Android operating system. 10The apk file to install the AR app on your Android device is available at this link: https://bit.ly/sBernardinoAR.You can watch a video showing the app's functionality at this link: https://youtu.be/UD8w0Ysr5Z4.An additional advantage of adding buttons is ability to completely deactivate all contents and view the painting through the mobile device without additional overlaid layers.This seemingly simple option allows for a more accurate comparison between the painting/AR marker and 'augmented' contents, as the operation always occurs within the device screen, avoiding the discontinuity arising from observing the painting outside the screen and the 3D models through the screen.

CONCLUSIONS
For several years, museums have been incorporating technological content to enhance their artworks, focusing on emotional engagement through captivating storytelling and offering insights derived from scientific studies.The presented AR application fits into the latter context, aiming to explain to users the nature of the perspective space represented in the painting of the San Bernardino cycle and its contradictions (Fig. 14).The choice to enhance scientific knowledge through augmented reality, preferring it to written verbal language, aligns with the theory that communication through iconic signs in museums is preferable, in line with the type of stimulus provided by artworks.This approach avoids causing visitors discomfort by shifting into another cognitive mode (from the sensory-motor mode of linguistic signs based on verbal language to the analytical-reconstructive mode of iconic signs based on visual perception) (Antinucci, 2014).
Although the developed AR app brings various benefits, it has raised some issues regarding its use in the field of perspective restitution of painted architecture.In the case of the San Bernardino painting, despite the very effective AR windows effect, the limited view from a narrow field of view does not allow for grasping architectural models from lateral observation, where the contraction of the objective model compared to the isotropy of the ideal model would be evident.On the contrary, the reduction effect of the landscape portion, transitioning from the objective model to the ideal model, is very clear.Another branch of disseminating perspective studies on this work by Perugino led to the creation of a physical model that helped understand the same 'perspective machine' (Baglioni and Menconero, 2023a).In that case, the physical nature of the models and the ability to observe and touch them from any point of view clarified what remains uncertain through AR.To overcome the problem, figures illustrating the formal characteristics of the objective and ideal models have been included in the poster activated by the app's Info button.
In any case, the combination of the real world and the virtual world, along with the engagement of interactive experience, makes AR more involving than traditional forms of communicating scientific content (such as panels and audio guides).Moreover, its application to perspective works is particularly significant in revealing the continuous transformation that links "space as it looks" and "space as it is" (Arnheim, 1977).

Figure 5 .
Figure 5.Comparison of the objective model and the ideal model: axonometric views and perspective views from the centre of projection (author's elaboration).

Figure 6 .
Figure 6.Relations between the perspective and AR systems (author's elaboration).

Figure 7 .
Figure 7.In transparency, the parts of the model do not have to be rendered during the AR experience to achieve the window effect (author's elaboration).

Figure 9 .Figure 10 .
Figure 9.The textured 'box' with the landscape pictures (author's elaboration) ); b.Calibration, as assistance to position the device at the viewpoint where the perspective of the painting and the objective model coincide; c.Line drawing, to see the outline of the painting and combine it with the observation of the objective or ideal model; d.Objective model, to observe the 3D model in combination with the outline and check how they coincide, exploring the contracted perspective space; e. Ideal model, to explore the regular architectural space and check how much the portion of landscape framed by the monumental arch narrows down.

Figure 12 .
Figure 12.Content accessible from the AR app.Letters refer to the list of features in the text (author's elaboration).

Figure 13 .
Figure 13.The poster accessible from the Info button of the app (author's elaboration).

Figure 14 .
Figure 14.AR app testing at the Galleria Nazionale dell'Umbria (photo by Matteo Flavio Mancini).