Integrative AI for the Understanding of Ancient Javanese Architectures

Abstract. The use of digital techniques has seen an increasing amount of use in recent years for heritage documentation. The development of artificial intelligence (AI) also contributed to this rise, with many different applications to help facilitate the heritage recording process. A by-product of these developments is the increasing amount of available data, in tandem with the ever-increasing need for training data for AI purposes. This paper aims to re-use old datasets and repurpose them using modern methods. The objective is therefore to see if older datasets may be used to improve the quality of AI-based methods, while also investigating the use of new technologies such as Visual Language Models (VLM) to perform semantic queries and Gaussian splatting on these datasets. For this purpose, datasets from a previous documentation project involving Javanese “candi” architecture is used in this paper since this particular subject has not seen too many AI-based documentation research in the literature and is thus an interesting example to evaluate the generalisation of AI methods. Results show that old datasets can very well be used with modern techniques with promising results. In terms of semantic segmentation, machine learning yielded an overall accuracy of 0.89 while deep learning yielded 0.79. Several interesting inferences were also observed in the VLM query results, while Gaussian splatting showed very strong potential for visualisation-based applications to further enhance the reusability of these old datasets.