MODELLING AND INTERACTIVE WEB-BASED VISUALIZATION OF CULTURAL HERITAGE OBJECTS

Nowadays, there are rapid developments in the fields of photogrammetry, laser scanning, computer vision and robotics, together aiming to provide highly accurate 3D data that is useful for various applications. In recent years, various LiDAR and image-based techniques have been investigated for 3D modelling because of their opportunities for fast and accurate model generation. For cultural heritage preservation and the representation of objects that are important for tourism and their interactive visualization, 3D models are highly effective and intuitive for present-day users who have stringent requirements and high expectations. Depending on the complexity of the objects for the specific case, various technological methods can be applied. The selected objects in this particular research are located in Bulgaria – a country with thousands of years of history and cultural heritage dating back to ancient civilizations. \this motivates the preservation, visualisation and recreation of undoubtedly valuable historical and architectural objects and places, which has always been a serious challenge for specialists in the field of cultural heritage. In the present research, comparative analyses regarding principles and technological processes needed for 3D modelling and visualization are presented. The recent problems, efforts and developments in interactive representation of precious objects and places in Bulgaria are presented. Three technologies based on real projects are described: (1) image-based modelling using a nonmetric hand-held camera; (2) 3D visualization based on spherical panoramic images; (3) and 3D geometric and photorealistic modelling based on architectural CAD drawings. Their suitability for web-based visualization are demonstrated and compared. Moreover the possibilities for integration with additional information such as interactive maps, satellite imagery, sound, video and specific information for the objects are described. This comparative study discusses the advantages and disadvantages of these three approaches and their integration in multiple domains, such as web-based 3D city modelling, tourism and architectural 3D visualization. It was concluded that image-based modelling and panoramic visualisation are simple, fast and effective techniques suitable for simultaneous virtual representation of many objects. However, additional measurements or CAD information will be beneficial for obtaining higher accuracy.


INTRODUCTION
3D visualisation and proper documentation of cultural objects helps us to keep the history and the memory of historic buildings, archaeological sites and cultural landscapes.Moreover, it supports economic growth by stimulating cultural tourism.Therefore, preservation, visualisation and recreation of undoubtedly valuable historical and architectural objects and places has always been a serious challenge for specialists in the field of cultural heritage.Some of the problems that specialists face regarding 3D modelling of cultural heritage objects are the proper selection of technology for image acquisition and post-processing, automation of the image-matching procedure, the visual and metric quality of the final product and proper documentation.These problems and challenges provide the overarching motivation for the current research.Nowadays, an impressive evolution has been observed in the creation of 3D digital models of objects, buildings and cities and their virtual representation.Using low-cost solutions with up-to-date fit-forpurpose technologies was considered suitable for 3D restoration, documentation and visualisation of cultural heritage (Hanan, et al.2015, Sužiedelytė-Visockienė, et al. 2015).Rapid technological developments in the fields of photogrammetry, laser scanning, computer vision and robotics can be seen, and some of these techniques are nowadays also commonly used for the visualisation and archiving of cultural heritage objects.3D models have proved to be valuable for virtual tourism, digital documentation in case of damage and photorealistic representation, urban planning and 3D archives.Various acquisition techniques can be used for 3|D model creation.Active (Lidar) or passive (image-based) sensors are among them but sometimes additional information such as CAD models or data acquired by GNSS can be beneficial (Remondino and El-Hakim, 2006).Even Unmanned Aerial Vehicles (UAV) have been tested and found to be reliable image acquisition technique (Meyer, et al., 2015, Themistocleous, et al. 2015).The selection of the most suitable and effective technology is also strongly related to the specific requirements of each project, its budget and the experience of the users.
The main objective of the current study is to demonstrate various techniques for the creation of interactive web-based visualisation of 3D models and to compare them in terms of their visual and metric qualities.For this reason three technologies based real projects were tested and described in the paper: (1) image-based modelling using a non-metric handheld camera; (2) 3D visualization based on spherical panoramic images; (3) and 3D geometric and photorealistic modelling based on architectural CAD drawings.After a short review of the recent research in the field the techniques for obtaining 3D models, the three different tests performed in the selected case study areas in Bulgaria will be described.In a subsequent section, per each test the acquisition technique or input data, post-processing procedures, opportunities for virtual representation and results will be presented.Next the visual and metric qualities of the final models will be compared and effectiveness of the various applications from the research, conclusions and recommendations will be made for future improvements.

RELATED WORK
The creation of complete, detailed and accurate 3D reconstructions -especially for constructions with high level of complexity and detail remain challenging tasks for researchers.Image-based modelling has been explored for years by photogrammetrists.Currently, there are software packages that provide opportunities for manual, semi-automatic and even automatic 3D image-based model generation.Many efforts have also been made by the computer vision community on assessing the feasibility of full automation (Nister, 2004, Pollefeys, et al., 2004).Recently, many multi-image matching algorithms with a combinations of laser scanning data have been investigated and compared (Kersten, et al., 2012, Remondino, et al. 2014, Rinaudo, et al., 2010).More specifically, Guidi et al. (2015) tested software packages based on Structure from Motion (SfM) and automatic Image Matching (IM) techniques, achieving accuracies of less than 3.5mm.However, semi-automatic or even manual methods are still used, especially for complex structures where high accuracy for details is essential.Much recent research has focused on using laser scanning technology for the 3D modelling of cultural heritage objects with high accuracy and within a short time frame (Yastikli, 2007, Serna, et al. 2015).However, compared to laser scanning technology, a clear advantage of image-based modelling is the low cost of the equipment.This is one of the reasons that this method is preferable when there are budget limitations.As mentioned previously, scene reconstruction has been intensively explored over the past decade by photogrammetrists and computer vision scientists (Akbarzadehet et al. 2006, Förstner, W., 2002, Musialski et al., 2013).However, because of the limited field-of-view (FOV) available while using standard images and videos and aiming maximum object coverage, researchers started to investigate the use of spherical and (if possible) spherical stereo images.(Kim, et al., 2010, Kim, et al. 2013).In general, metric applications with high-resolution rotating cameras were a focus for many researchers in the field (Luhmann, et al., 2004, Schneider, et al., 2004, Schneider, et al., 2005).However, there remains only limited research focused on comparing various techniques for 3D modelling in terms of their metric reliability.
At a more local level in Bulgaria, scientists have also expended much to preserve national cultural heritage objects and to keep the inherited traditions from the past.Koeva (2015).
In summary, diverse approaches for 3D modelling have been investigated recently.Due to the usual complexity of the Cultural Heritage objects and specific requirements for each project, special skills and knowledge are required for the selection of proper and effective 3D modelling techniques.

TECHNIQUES FOR CREATING 3D MODELS OF CULTURAL HERITAGE OBJECTS
The three cases based on real projects discussed in this paper are located in Bulgaria which is a country with thousands of years of history and cultural heritage that spans several ancient civilizations.

Image-based 3D modelling using non-metric hand-held cameras
3D image-based modelling using hand-held cameras is a technique that has been used for various types of objects such as historical buildings, monuments, siluets, artefacts or natural heritage (Gruen, 2013).Manual or automatic image matching procedures can be applied depending of the requirements of the specific projects.The main processes for the task of 3D visualisation of buildings located in the central part of the capital city of Bulgaria, Sofia, will be explained in the following sections of this paper.

Image acquisition.
For the image acquisition stage of an image-based modelling approach, the correct sequence of capturing the images and the proper selection of the capture angle are essential.Moreover, overlap between images has to be considered and the specialist for this process should take the lighting conditions into account as well.The images should fully cover the object of interest, which also means that it would be beneficial to have more than the minimum required number of images.However, in practice not everything can be achieved perfectly for various reasons: (1) there might be occlusions, for example from vegetation or other objects in the urban parts of the cities; (2) the facades with dominant glass coverage can create reflections; (3) the facades can be in a full shadow; and (4) the access can be restricted etc. Especially in the central part of highly populated cities it can be challenging to fulfil all the requirements for the creation of a perfect 3D model.For example, considering the usually sunny weather in the selected study area of the city of Sofia, the suitable time for image acquisition had to be carefully planned in order to avoid variations in lighting and shadows.Traffic flows during this process also should be considered.Non-metric cameras are not calibrated and self-calibration procedure can be done in order to use them for 3D modelling.For the current test area, the camera used for acquiring the images was low-cost 20 megapixel CCD digital Canon IXUS 175 camera which was self-calibrated using Photomodeller software.

Image processing and 3D modelling.
The challenging part in the first test was to create a 3D model of buildings in the central part of Sofia where the traffic is intensive during daytime and the full access to the objects can be sometimes problematic.For that reason, careful planning for the proper time of image acquisition was needed.The distance between every image was carefully evaluated by considering the distance between the camera and the object and the required overlap.For full coverage, between 30-70 images per object were needed, depending of object's size.After the capturing procedure, the overlap and coverage were checked and, if necessary additional images were taken.Then the images were organized according to the software requirements and for the best final result additional work was done on the equalization of the lighting, brightness and contrast.
After a comparison among the available software packages that could be used for image-based modelling, AGK Visual Reality (VR) from Autodesk was selected.The main reason for this selection was that the geometric model of the objects could be accurately drawn in AutoCAD.Another reason was that usually most desktop applications have problems while working with more than one object, whereas the selected one rasterizes information for numerous 3D models combined with the additionally added elements of the infrastructure.For the selected study area, additional information from the Municipality's company GIS-Sofia was available, such as: footprints of the buildings and their administrative functions, geolocations, street networks, terrain information (points, contour lines and TIN), a QuickBird satellite image and an orthophoto created from digital aerial images.
The methodology used for geometric construction of the 3D models was according to the requirements of VR software.
(http://www.akgsoftware.at/index.php/visual-reality).After the extrusion of the buildings according to their height based on the given footprints, each of the façades was redigitized following the VR software requirements.In this way the geometric model constructed in AutoCad was automatically transferred into VR for the image matching procedure (Figure1).For each facade the corresponding image or part of the image had to be manually assigned according to the direction of its geometric creation.For the missing facades, an artificial texture from the available gallery of the software was selected or taken from the aerial images.Additional 3D models of exterior features such as people, cars and vegetation were automatically added from the gallery, to realistically represent the urban environment.

Virtual representation.
For the interactive visualisation part of the task the 3D models were exported in the *.wrl format and visualized using Internet Explorer.For this, the Blaxxun.exeplug-in for visualization and navigation of the 3D model was installed.

3.1.4.Results.
In the project, as a result the central part of the city of Sofia was modelled.One of the buildings -together with its surroundings is shown (Figure2) as an example for one of the created in the central part of Sofia objects.The results were evaluated for the visual and metric qualities of the models.The overall task of 3D models, namely to reproduce reality in a realistic way, was achieved and it was therefore concluded that the visual accuracy was sufficient.However, differences in colour homogeneity, reflective surfaces and shadows affect the overall visual qualities of the obtained 3D models.Regarding the metric accuracy of the final 3D models, for this test it varied from 1 to 3m.
The comparison was done based on vertical and horizontal tape measured distances on the objects in reality and the model using RMSE calculation.The main reason for mismatching was due to the quality of the images, occlusions in the urban area and missing parts of the facades.To improve this accuracy, three possible solutions are: to take pictures with a higher resolution camera, to be very precise in non-planar facades and to avoid occlusions.Nevertheless, as the aim was to represent numerous objects simultaneously, the achieved visual reliability with this first test was satisfactory.

3D visualization based on spherical panoramic images
Image-based modelling is proven technology for the photorealistic reconstruction of cultural heritage objects.Images that can be used for this task can be planar (tested in 3.1.)or 360 0 panoramic.The panoramic images for Spherical Photogrammetry with equirectangular projection are obtained by stitching together different photos, taken from the same point and rotating around a pivot (d'Annibale, 2011).
The aim of the second test was to prove the suitability of Spherical Photogrammetry and low-cost solutions for photorealistic visualisation not only of one but of many important touristic objects, at the same time, and to represent them with additional geospatial information in one interactive web-portal.
The selected historical objects are located in four municipalities in the central part of Bulgaria.

Image acquisition.
To obtain high quality panoramas, proper selection of acquisition instruments, according to the required image resolution, is needed and the theoretical concepts applied most should be clear (Fangi, et al., 2006(Fangi, et al., , 2008(Fangi, et al., , 2009(Fangi, et al., , 2011)).After investigation of the existing reflex cameras, the Nodal Ninja 5 R + D16 Rotator were selected.One of the reasons for this decision was because this camera has a special adjustment that allows it to be rotated on a tripod over the "pupil" to eliminate vertically any parallax (shift of the foreground objects relative to the background ones).The advantages of the D16 Rotator is that the user has 16 choices for the spacing of "click stops", as the head rotates (Figure 3).The photos were taken using f9-stop aperture in order to use the maximum advantages of the lenses.The exposure time when using "manual mode" was 1/250 -30 seconds, using the principle of zeitraffer photography, that is needed for the HDR software that performs image post-processing.To include additional information about the objects and integrating this into the web-portal, frame images were captured using the Canon EOS 5D Mark 2 with EF 24-70, f/2.8LUSM, EF 70-200, f/2.8LUSM, 15 mm, f/2.8L lenses, Extender EF 2x II and tripod Manfrotto 808RC4.
The image acquisition and later on web-based visualisation of the selected objects was organized into 6 thematic, touristic routes (museums, churches, nature etc.) because the aim was to integrate as many touristic objects from the region as possible.
For this reason, a hand-held GPS (Garmin) was also used for determining the geographic position of each of the objects.The digital camera was mounted on a tripod that was remotely controlled from a distance of at least 5m.As previously set, the camera rotated automatically and took images at regular intervals of 15 °.The distance between the tripod and the objects while capturing the panoramas was 2 to 8 m.

Image processing and 3D visualisation.
For postprocessing, a special "stitching software" recommended by the camera manufacturers was used for the seamless stitching of the images.Moreover, there was an opportunity for the automatic removal of the shadow created by the tripod.This is an advantage because no manual work is needed for removing any signs from the used equipment.Another advantage is that also brightness and contrast differences, which undoubtedly exist in every mosaicking process, were automatically equalized.This system is definitely easy to use and satisfies the requirements for a flexible and seamless stitching process.Therefore 50 objects were selected, captured, organized into thematic routes and visualised in various colours, over the map that was specially created for the project, with 3D terrain representation.
The software used to create the map was ArcGIS (Figure 4).

Virtual representation.
To integrate various types of information and visualisation into one web-portal, TourWiever7 software was selected.The general purpose of this application is to achieve simple visualisation tasks usually for one object and, in order to adjust it for this project, creativity and combinations with additional open source solutions were needed.The overall design of the web-portal and of each button and menu were independently performed using CorelDraw.The functionalities of the buttons, links and time intervals were set and adjusted in TourWeaver, but those procedures were very time consuming.Because of the huge amount of data that was needed to be integrated, panoramas were resampled at lower resolution before being imported and linked in the web-portal.This was also needed because visualisation at various scales is necessary for the users to have a coarse resolution even in the detailed parts of the panoramas.
For better orientation, two types of maps were used: one created especially for the project and one from Bing which was provided as an automatic option by TourWeaver.The link between the two maps and the panoramas in the main view was made via a radar function with an orientation to the North.By clicking on a selected touristic object the user can freely navigate in the panoramic view, obtain information about its position (GPS coordinates), achieve simple image visualisation in "Thumbnails" and obtain text information.Additionally, sound and video created to describe the region were also integrated into the web-portal (Figure5).To assess the metric accuracy of the models, taken from a distance less than 5m.from the objects, in the central part of the created models from panoramas, where no distortions were observed, was 0.1m.For those taken from a longer distance, the accuracy reached 0.3m.The calculations were done using RMSE calculation between measurements on the real objects and on the central parts of the panoramas.
The web-portal was translated from Bulgarian to the English and Russian languages and was published on the web pages of the four municipalities.3D physical models were also created using 3D printing technology.One representative, important object from each of the four municipalities was selected, 3D printed and fixed on a 3D map model (Figure 6).Before publishing the web-portal, five potential users were asked to test it and to give their observations, recommendations and difficulties in order to improve its functionality.Some of these users did not have any previous experience with such applications and their suggestions were very valuable in making the web-portal more user-friendly.
The final result can be accessed from the following link: http://regtour.lovech.bg/LOVECH_ENG/index.html 3.3 3D geometric and photorealistic modelling based on architectural CAD drawings 3.3.1 Input data.Usually, cultural heritage objects present a challenging task for 3D photorealistic modelling.This is because of their complexity, mainly in terms of geometry but also from the colorimetric point of view.For the third experiment in the current study.CAD drawings of a building with a proportional simple geometry was selected.At the time of the 3D modelling the building was at the stage of planning to be built in the central part of the city of Sofia.

3D modelling.
Having as an input only the CAD drawings, 3D modelling had to be done using software that can support this format.For that reason, and also for comparison with the first test, again AGK Visual Reality (VR) from Autodesk was selected.Using AutoCAD, the geometric 3D model was constructed via boundary representation.Since there were detailed plans with dimensions for each of the floors, there was clearly a good opportunity for accurate model construction (Figure7).

Results.
The visual representation for this case depends critically on the professional experience and personal vision of the model creator.This building was built in reality two years after the plans and the modelling was finished which gave an opportunity to make a realistic comparison with the model.The achieved metric accuracy of the geometric model was high because all the dimensions were according to the CAD plans.The average error calculated by five measured distances was 0.01m.

DISCUSSION AND CONCLUSIONS
The 3D modelling of Cultural heritage objects is a challenging task, from the data acquisition and gathering all possible input data, to the final phase of visualisation.One of the important aspects is how much the virtual representation reflects the real world (Groat and Wang, 2002).The metric content of the created 3D model primary depends on the accuracy of the sources of the data it was created from.
Comparing these three technologies for model creation leads us to draw conclusions in terms of their visualisation qualities, metric reliability and usage.In the first two experimental cases in this research, the main sources were images (planar and panoramic).For this reason, the final accuracy is directly related with the images' resolution.The problems during the image acquisitions mentioned in this paper are relate to both methods and, knowing the constraints they can be avoided by a proper selection of the equipment, objects of interest, weather conditions etc. Depending on which lens is used, there is also a difference in the number of images required for 3D model creation.This number is also highly related with the fast opportunity of web visualisation because the aim is that the user can see more panoramas at the same time easily at the highest possible quality.Since the main source of information for the third example are CAD drawings, in this case the accuracy is related to the geometrical quality of the 3D model which has been shown to be at centimetre level.As all three of tests do not use automatic functions, therefore one conclusion is that they depend heavily on the experience and precision of the people who created them.In terms of visualisation, all three tests can be considered as suitable but still depending from the projects requirements.Thinking about the users for the three of them, significant differences were observed.The image-based modelling technique was considered as more suitable for specific architectural and historical monuments.However, if high accuracy is not needed and there is an opportunity to obtain high-resolution data, the first test proved to be reliable for the visualisation of several objects simultaneously, together with their surroundings.Despite the fact that quite reliable results can be obtained using automatic algorithms for image matching, semi-automatic procedures can still be used in order to achieve more accurate results especially in complex situations.
From the obtained results, it has been proved that for tourism, virtual navigation and the demonstration of numerous cultural objects, spherical panoramas -together with tours and additional information has been shown to be a good solution for visual representation.As a recommendation, such interactive visualisations potentially can be extended with virtual tours using avatars, perhaps even with a voice.The overall results show that for non-existing or damaged historical objects, the third test is most suitable.The reason is that if CAD data is available, the obtained accuracy of the 3D reconstructed models as shown can be high (0.01m).Therefore, they can be useful for architects and prospective clients in representing various scenarios before a building is constructed.
This paper intended to show various techniques for the creation and interactive web-based visualisation of 3D models and to compare them in terms of their visual and metric qualities.Using low-cost solutions, the three tests proved that imagebased modelling and panoramic visualisation are simple, fast and effective techniques suitable for simultaneous virtual representation of many objects.However, for obtaining higher accuracy additional measurements or CAD information will be beneficial.
The added value of using digital 3D models have been demonstrated by many researchers.For cultural heritage, this is important because it gives us the opportunity to visualize special and temporal information for precious historical monuments and objects of interest.

Figure 1 .
Figure 1.National Palace of Culture in Sofia -geometrical model created in AutoCAD

Figure 4 .
Figure 4. Route with museums in the region

Figure 5 .
Figure 5. 3D visualization based on spherical panoramic images -Apriltsi 3.2.4Results.The results indicated that high-resolution spherical panoramas integrated in a web-portal with other types of information can be successfully used for the interactive representation of important historical monuments, architectural objects and cultural places of interest.Visual representation was extremely impressive, mostly because of the high-resolution images.To assess the metric accuracy of the models, taken from a distance less than 5m.from the objects, in the central part of the created models from panoramas, where no distortions were observed, was 0.1m.For those taken from a longer distance, the accuracy reached 0.3m.The calculations were done using RMSE calculation between measurements on the real objects and on the central parts of the panoramas.

Figure 6 .
Figure 6.3D printed model of Troyan Monastery.

Figure 7 .
Figure 7. Architectural drawing.The modelling work using VR included selection of the textures from the palette available in the software.For the surrounding environment, 3D models of additional elements were inserted such as cars, vegetation and road.(Figure8).

Figure 8 .
Figure 8. Photorealistic modelling based on architectural CAD drawings -Sofia