DIGITAL 3 D BOROBODUR : INTEGRATION OF 3 D SURVEYING AND MODELING TECHNIQUES

The Borobudur temple (Indonesia) is one of the greatest Buddhist monuments in the world, now listed as an UNESCO World Heritage Site. The present state of the temple is the result of restorations after being exposed to natural disasters several times. Today there is still a growing rate of deterioration of the building stones whose causes need further researches. Monitoring programs, supported at institutional level, have been effectively executed to observe the problem. The paper presents the latest efforts to digitally document the Borobudur Temple and its surrounding area in 3D with photogrammetric techniques. UAV and terrestrial images were acquired to completely digitize the temple, produce DEM, orthoimages and maps at 1:100 and 1:1000 scale. The results of the project are now employed by the local government organizations to manage the heritage area and plan new policies for the conservation and preservation of the UNESCO site. In order to help data management and policy makers, a web-based information system of the heritage area was also built to visualize and easily access all the data and achieved 3D results. Figure 1: The geographic location of the Borobudur UNESCO site (left) and some images of the temple with its stupa and bas-reliefs.


INTRODUCTION
Borobudur is the most outstanding temple in Indonesia.Located on the Borobudur plain, this temple sits majestically in an open area about forty kilometres north-west from Yogyakarta, Indonesia.It is located in the very central part of Java Island, Indonesia (Fig. 1).It was built around 824 AD by Samaratungga, a king of the Syailendra dynasty , a flourishing Buddhist kingdom.The Borobudur building (Fig. 1) needed about 125 years of construction and was utilized by religious activities until the 11 th century.About two centuries after the construction, Borobudur was not used by people because of the great disaster in Java Island and because of the civilization movement to East Java.Borobudur was forgotten until it was opened again by Raffless in 1814 (Durmacay, 1978), and then completely cleaned by Hartmann from Dutch Indies Government in 1835.Borobudur has been restored two times, the first restoration was done by Theodore van Erp on 1907-1911 while the second one was realized by UNESCO andthe Indonesian Government in 1973-1983.The temple was inscribed as an UNESCO World Heritage Site in 1991 (number 592) and represents one of the largest Buddhist temples in south-east Asia.
The structural stability problems of the Borobudur temple were solved after the second restoration.After that, the structural stability has been continuously monitored to control the temple deformation both horizontally and vertically.The main problem of the Borobudur temple is its deterioration rate.Borobudur and its stones are at the mercy of sun rays and heavy rains all year long, thus accelerating the deterioration of the structures and decorations.The Borobudur site is also very often affected by volcanic ash.In autumn 2010 the eruption of M ount M erapi, located approximately 30 km from the temple, covered the entire temple with ca 2.5 cm of ash, killing also the nearby vegetation and damaging the historic monument due to its acidity.The temple complex was closed for many days to clean up that ash-fall while the upper levels remained closed to the public until late September 2011.The Borobudur Conservation Agency, with the help of UNESCO, was able to rid the temple's stones of volcanic sediment, to plant trees to stabilise temperatures and finally to support the living conditions of local residents (M asanori, 2011).As one of the greatest UNESCO world heritage sites, its geometric documentation is important for monitoring its deterioration and preservation but also for spatial management and dissemination purposes.M onitoring the Borobudur temple requires great efforts due to the large and complex size of the building (Fig. 2).The length of each side is 121 meter and the height of the whole building is 35 meter.It consist of three part: (i) the temple-foot, the lowest part consisting in two levels of foot structure; (ii) the body of the temple, consisting in four levels of main wall and five levels of balustrades with the sculpture gallery (bas-relief).The total number of panels of bas-relief is 1460.The upper part of the building, i.e. the head, consists of three level terraces with 72 small stupas and one main stupa at the centre.On October 28th -November 1st, 2013 a workshop related to the application of aerial (UAV) and close-range photogrammetry for documenting cultural heritage sites was held in Borobudur.It was organized by the Remote Sensing and GIS Research Group of the Institute of Technology Bandung (ITB).The main aim of the workshop was to build an heritage information system, based on reality-based 3D surveying data, for the documentation and conservation of the Borobudur site.This paper steams from the outcomes of the workshop and surveying operations realized during the event.The planning of the data acquisitions, the achieved results and some final comments are afterward presented.

Related works
Since more than 20 years, sensor and data fusion is employed as very successful solution for the 3D documentation of large heritage sites.In a multi-resolution survey, we should be distinguished between (i) geometric modeling (3D shape acquisition, data registration and further processing) where multiple sensors and resolutions are combined to model features with the most adequate sampling step and (ii) appearance modeling (texturing, blending, simplification and rendering) where multiple images are used for realistic appearance and various LODs (Level of Details) are used to face 3D models complexity during the visualization.For the 3D documentation of large and complex sites, the state-of-the-art approach uses and integrates multiple sensors and technologies (photogrammetry, active sensors, topographic surveying, etc.) for the derivation of different geometric LODs of the scene under investigation, both in geometry and texture.Each LOD shows only the necessary information while each technique is used where best suited to exploit its intrinsic modeling advantages.Stumpfel et al., (2003) digitally reunified the Parthenon of Athens and its sculptures using photogrammetry and stripe projection systems.Gruen et al. (2005)

The digital 3D Borobudur project
Similar to other projects, a multi-sensor multi-resolution approach was selected in order to realized different 3D products for better management and conservation of the heritage site.Photogrammetry was chosen as the most suitable 3D recording technique for the surveying and modelling of the site.Indeed, due to its complexity, dimensions, number of occlusions, necessary LODs and required products, active sensors were not able to fulfil the project specifications.Different UAV platforms and digital cameras were employed.In order to plan the image acquisitions and provide a cost effective network configurations for each of the multi-LODs flights over Borobudur, a network design analysis was performed.Camera network design (Fraser, 1984;Kraus, 1997;M ikhail et al., 2001;Nocerino et al., 2014) is indeed a very important task to ensure a good quality of the acquired images and successive 3D model generation.

3D S URVEYING OPERATIONS AND PHOTOGRAMMETRIC PRODUCTS
Within the 3 days of the 2013 workshop, the UNESCO area was surveyed by a group of 20 people using terrestrial and aerial (UAV) photogrammetry.A fixed-wing UAV was used to capture the surrounding area of the Borobudur temple (ca 1 km x 1 km).A hexa-copter was flown at lower altitude to capture the temple (ca 120 m x 120 m) at high resolution.Finally terrestrial images were used to capture and digitally model the details of the temple (e.g.relief on the walls).
The flowchart which describes the process is shown in Figure 3.
As visible in the flowchart, different acquisitions were realized to capture different LODs of the heritage site.Using UAVs, the 3D reconstruction process of the temple can be completed at reasonable costs and time if compared to conventional mapping techniques.The employed UAVs were coupled with a control system for real-time data transmission and image orientation.

Ground Control Points
GCP measurement, performed with GNSS or total station instruments, is fundamental for scaling and georeferencing purposes but also to check the quality of the photogrammetric processing.

Fixed-wing UAV
A fixed-wing was flown 280 m above the ground to cover around 1 km 2 .The aim was to generate a DEM and an orthophoto of the surrounding area of the temple.A flight plan was made before the flight (Fig. 6 -left), with a 80-60% overlap and a planned GSD of 7 cm.The camera mounted on the platform was the Sony NEX-7 (23.4 x 15.6 mm sensor size with a pixel of 3.9 micron) coupled with a 20 mm nominal focal length.

Hexa-copter UAV
Several experiments were done before the data acquisition to design the best flight lines to cover the temple with the hexacopter platform (Fig. 5 -left).Finally five strips were designed to cover the area around the temple (Fig. 6 -right).Images were acquired in vertical as well as oblique mode, in order to capture the details of the vertical facades of the temple.For the vertical images, the UAV was flown at 50 m above the ground mounting a Sony Nex-5 camera with a 16 mm focal length.This flight aimed to map the temple in lower scale (1.5 cm GSD on the ground) than the fixed-wing.On the other hand the oblique images were acquired flying at 20 m, 30 m and 35 m above the ground (Figure 8) using again the same camera.

Terrestrial Photo
Terrestrial photogrammetry aimed to capture high details of the structure, such as reliefs in the vertical walls or other decorations.A mix of parallel axis and convergent images were acquired to ensure a reliable self-calibration, achieve a complete 3D reconstruction of the details and collect suitable stereo-pairs for successive interpretations and analyses by the experts.The employed camera was a Sony RX100.

Photogrammetric results
The photogrammetric processing was performed considering the single datasets and using the available GCPs for the georeferencing and the successive merging procedure.
Using the achieved products (DEM , orthos, dense point clouds, 3D models), the various LODs were created (Table 1) for the database creation, visualization purposes and conservation analyses.

DATABAS E AND VIS UALIZATION
The development of geo-spatial database starts with the design of data model which shows what data are to be contained in the database and how the items in the database will be related to each other.The database design process is usually divided into three main activities, namely: conceptual, logical and physical data modeling.The entity-relationship diagram shown in Fig. 9 represents the proposed conceptual model of geo-spatial database of the Borobudur site.The attributes are not shown in the diagram for readability purpose, however they have been identified and included in its corresponding logical model.

TOWARD 3D WEB-BAS ED MANAGEMENT AND ANALYS ES
As previously mentioned, the Borobudur temple is extremely vulnerable as it is exposed to heavy rains, earthquakes and volcanic activities.The site has been damaged by earthquakes, which are very strong in Java island, moving the ground surface as waves with imaginable consequences to the structural integrity of the temple.The ash of volcano M erapi, which erupted very frequently in the last few years (in average every 3-4 years), covers the temple detreating the ancient stones.On February 2014 the site was closed to visitors due to the ash coming from the Kelud volcano in East Java, located ca 200 kilometers east from Borobudur.Therefore the Borobudur temple represents an example of a dynamic and complex context of conservation and restoration, which needs efficient and cost effective solutions.Each stone composing the temple and its outstanding reliefs must be classified, indexed and catalogued and its location georeferenced.For these purposes, 3D digital technologies, if wisely and systematically applied, represent a powerful solution for sharing and efficiently managing information about large and complex heritage environments.With this perspective, in the last years, the problem of visualizing and managing complex 3D data has been extensively investigated by different researchers who have presented web based solutions for visualization purposes (Silla, 2012;Potree, 2014;Potenziani et al., 2015) as well as for analyses and management (Fassi et al., 2011(Fassi et al., , 2015;;De Luca, 2013) or in the form of navigable virtual reality environments based on unity game engine (Agugiaro et al., 2011;Hullo et al., 2015).The above mentioned systems, very often structured with different LODs, may be enriched with additional features and tools for analyzing the 3D shape of each single stone relief as a whole.Pires et al. (2015) proposed V-PTM system based on Reflectance Transformation Imaging (RTI) built on 3D mesh models instead of using real RTI images specifically acquired with generally bulky and expensive equipment.This feature may help archeologists to better interpret and study the figures carved in the reliefs by easily changing the light source direction.As shown in Fig. 11a, the characteristics of the Borobudur stones make the figure's readability very hard on color images while much more details are clearly highlighted by simply changing the illumination on the 3D model.Another important study that is carried out on the reliefs of Borobudur temple is the analysis of shape deformations with respect to the ones ideally expected.This is an area-based analysis able to show global warps.Figure 12

CONCLUS IONS
The paper presented a multi-sensor multi-resolution approach to geometrically document the very complex and large UNESCO site in Borobudur.Photogrammetry was employed at multiple scales to produce various geometric LOD and deliver 3D models, orthophotos, maps and a spatial database for conservation and preservation purposes.All these products had to follow the specifications and needs of the local government organizations in charge of the management and conservation of the heritage site.Topographic maps with absolute National Standard for Spatial Data Accuracy (NSSDA) errors of 0.25 m (accuracy), as good as a map in scale 1:1000, were achieved using GCPs and nonmetric small format digital cameras mounted on a fixed-wing UAV platform.This provides a relatively inexpensive and flexible solution for generating DEM for large scale topographical mapping requirements in Indonesia where clouds at low altitude often do not permit standard photogrammetric acquisitions.The produced map can be used for tourism and conservation purposes as well as for cadastral and landscape planning.Using the hexa-copter UAV data, a map 1:100 was realized, showing that UAV photogrammetry with small format nonmetric cameras could give satisfactory result also at big scale.This map can be used for monitoring of deterioration and preserving the site.For the sharing and visualization of the produced 3D data, the developed application enriched by Openwebglobe is avoiding lengthy loading time and poor visualization performance and it allows to easily access all the data.
used a multi-resolution image-based approach to document the entire valley of Bamiyan and produced a GIS of the UNESCO area.El-Hakim et al. (2008) integrated terrestrial photogrammetry and different laser scanners to produce a textured 3D model the Erechteion of Athens.Guidi et al. (2009) used a multi-resolution and multisensor approach to digitally reconstruct the entire Roman Forum in Pompeii.Remondino et al., (2009) merged drawings, images, range data and GPS measures for the detailed modeling of castles and their surrounding landscapes.Takase et al. (2009) used old maps, aerial and street photos, paintings, archaeological records and LiDAR data for the 3D reconstruction and visualization of the historic city of Kyoto in Japan.Kochi et al. (2012) integrated range-based and imagebased point clouds for architectural modelling.Serna et al. (2015) fused active sensors data with images for colouring purposes.

Figure 3 .
Figure 3. Workflow of the project: from acquisition to database generation and web sharing.BKPB (Borobudur Heritage Conservation) set up a control network made of 84 Ground Control Points (GCPs) in a local coordinate system (Fig. 4 -left), defining each point with permanent nails.During the UAV campaign, 6 new GCPs were measured using geodetic GNSS double frequency L1/L2 receivers with good distribution covering the AOI (Fig.4right).Implementing a rapid static differential positioning, every GCP must be measured not less than 30 minutes in

Figure 5 :
Figure 5: T he hexa-copter used for the low-altitude flight (left) and the GCPs measurement (right).

Figure 4 .
Figure 4. T he BPKP GCPs network on the temple (left) and the GCP's distribution for the UAV fixed-wing images (right).

Figure 7 :
Figure 7: DEM of the entire area produced from the UAV fixed-wing images.

Figure 8 :
Figure 8: 3D model of Borobudur temple built from the UAV hexacopter data.

Figure 10 .
Figure 10.Web application for the visualization of the developed spatial database of the Borobudur area.
shows a deformation (due to structural movements or inefficient restorations) of relief stones.The colour coded map displays a highly deformed surface due to movements / compressions of the structure.Repeated 3D modelling activities may keep track of local deformations of both reliefs and building structures as well.11.A carved figure in the reliefs of Borobudur temple: its deterioration and colours make the readability quite hard (a).Much more readable synthetic views of the same relief can be easily and automatically generated by changing the direction of light casted over a 3D mesh model and then given as input to obtain a V-PT M (b-f).The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-5/W7, 2015 25th International CIPA Symposium 2015, 31 August -04 September 2015, Taipei, Taiwan This contribution has been peer-reviewed.doi:10.5194/isprsarchives-XL-5-W7-417-2015

Figure 12 :
Figure 12: 3D surface model of a and computed shape deformation with respect to the ideal / flat surface.A typical colour pattern is visible (depressions in blue, swelling in orange/red) showing deformations of the (units are meters).