Design of an XR-based AI-augmented Content Framework for Frescoes (focusing on Raphael's School of Athens)

Abstract. Recent advancements in Artificial Intelligence (AI) have significantly enhanced the cognitive abilities of digital humans, making them increasingly human-like not only in appearance but also in behavior. Although AI has been actively applied across various industries, its integration into cultural heritage and museum-based eXtended Reality (XR) experiences remains relatively underexplored. This study aims to develop an interactive XR experience by recreating 57 philosophers, mathematicians, and scientists depicted in Raphael’s fresco (1510–1511) as AI-driven digital humans. To achieve this, we integrate technologies such as ChatGPT, domain-specific AI models, and AI Augmentation (AIA) to enable these historical figures to engage in intelligent and dynamic conversations with visitors. The ultimate objective of this research is to design a framework that combines static cultural heritage assets, such as paintings, with immersive XR technologies to establish an AI-based digital human platform. Through this system, users can move beyond passive observation to engage in personalized and interactive dialogues with historical figures, deepening their understanding and exploration of cultural heritage. By enhancing intuitive interaction and immersive presence, this framework paves the way for next-generation digital cultural heritage exhibitions.