AN OPEN SOURCE SOFTWARE PLATFORM FOR VISUALIZING AND TEACHING CONSERVATION TASKS IN ARCHITECTURAL HERITAGE ENVIRONMENTS

In this work we present a new software platform for interactive volumetric visualization of complex architectural objects and their applications to teaching and training conservation interventions in Architectural Cultural Heritage. Photogrammetric surveying is performed by processing the information arising from imageand range-based devices. Our visualization application is based on an adaptation of WebGL open standard; the performed adaptation allows to import open standards and an interactive navigation of 3D models in ordinary web navigators with a good performance. The Visualization platform is scalable and can be applied to urban environments, provided open source files be used; CityGML is an open standard based on a geometry -driven Ontology which is compatible with this approach. We illustrate our results with examples concerning to very damaged churches and a urban district of Segovia (World Cultural Heritage). Their connection with appropriate database eases the building evolution and interventions tracking. We have incorporated some preliminary examples to illustrate Advanced Visualization Tools and architectural e-Learning software platform which have been created for assessing conservation and restoration tasks in very damaged buildings. First version of the Advanced Visualization application has been developed in the framework of ADISPA Spanish Project Results. Our results are illustrated with the application of these software applications to several very damaged cultural heritage buildings in rural zones of Castilla y Leon (Spain).


INTRODUCTION
A crucial issue for cooperation in Cultural Heritage is the capability of communicating different aspects which are embedded in physical and/or digital objects.To reuse the huge amount of material and due to recent developments of Semantic Web, it is necessary to perform semi-automatic recognition of contents from multimedia support in a reliable way.To fix ideas, we shall restrict ourselves to image-based accurate representations of cultural heritage architectural objects.The confluence of image analysis and expert systems for recognition provides a support to develop more advanced systems focused towards Architectural Visualization and e-Learning tasks.The hardest challenges concern to e-Learning issues with a two-fold goal: (a) Provide analysis tools to different workers in cultural heritage interventions (re-qualification), and (b) incorporate the implicit knowledge of experts in conservation and rehabilitation.
Standards for image analysis and 3D reconstructions are well known and acknowledged by the international community; in particular, architectural photogrammetry and 3D reconstruction share the same principles which are based in Bundles Adjustment.Unfortunately, there is no still a universal framework given by adaptive expert systems able of (1) recognizing complex components from views or 3D reconstructed models, (2) their role in the building fabrics and possible (3) "pathologies" involving structural aspects and materials (Alegre and Dellaert, 2004).Semiautomatic recognition implies to solve query, detection, extraction, labeling, indexing and classification issues from digital information analysis.All these tasks involve to the incorporation of implicit knowledge from experts in restoration and conservation tasks.It is necessary to design and implement software tools for inserting and communicating this implicit knowledge, referred to meaningful examples which are contained in shared cultural heritage repositories.Our work concerns to the incorporation of the implicit knowledge of experts by means an open source visualization platform.In this way, we intend to provide a support to identify problems and propose solutions from images and volumetric representations.
A visualization-based approach is transversal to different Documentation, Information and Management Systems (DIMaS in the successive).Indeed, (1) Documentation System involves to digital archives which are used for photogrammetric architectural surveying; (2) Information System involves to specific hardware, software and procedures to provide a support for different functionalities (capture, management, analysis, modeling and presentation) linked to architectural databases; (3) Management Systems are focused to solve evaluation, planning, execution and tracking tasks linked to an intervention.Nevertheless the relevance of the above aspects, e-Learning tasks require additional components.Indeed, beyond standard functionalities of databases linked to urban/building GIS, the design and implementation of an expert system layer involves to technical evaluation, diagnoses and possible interventions.
Each one of the above systems can be understood as a floor of a symbolic representation for knowledge representation.More precisely, (0) the ground floor is given by the physical object; (1) first floor is given by its documentation system which concerns mainly to architectural surveying (digital 2D/3D models, video, animations); (2) second floor concerns to the Information System (organized as a GIS) with corresponding databases and basic functions involving them which is organized as a GIS for buildings, following typical AEC or CityGML standards; (3) third floor concerns to the Management System, where interventions can be simulated, tracked and/or validated, including eventually a Decision support System.This scheme has been implemented for conservation and rehabilitation techniques in the Spanish Strategic Singular Project Patrac (Accesibility in Cultural Heritage); see (Calle et al., 2011).
The connection between different layers can be performed following two approaches which are related between them: Semantic and Visualization levels.Semantic approach concerns to the use of ordinary language which is internally organized in terms of a well specified Ontology.A cultural heritage Ontology includes a cultural heritage lexicon (collection of key words), thessauri (collection of definitions and descriptions) and taxonomies (sets of logical rules).We have developed a specific Ontology for cultural heritage with application to Accessiblity issues.In (Calle et al., 2011), some of authors have specified the software architecture.Visualization approach concerns to the use of smart interactive navigation systems for each level and for connecting different levels by using the previously specified Semantics.
The interaction capability of Visualization tools allows to perform queries (against a database) or perform computations (by using an appropriate scientific software) from an interactive navigation, depending ont eh available functionalities.An interactive exploration and queries processes requires to identify the "active" elements appearing in the digital support which are activated by the user in an interactive way.The very large casuistic of Cultural Heritage building and the very large diversity of pathologies which can appear (relative to structural or materials properties) require a flexible tool for analyzing them and communicating expertise from different showcases.The main focus of this article is oriented towards the integration of Advanced Visualization Tools to recover implicit knowledge of professionals.
More recent developments of e-Learning software platforms to support different kinds of academic and/or professional activities allow to communicate the knowledge for Cultural Heritage interventions.Our work concerns mainly to the incorporation of open source Advanced Visualization Tools and e-Learning software tools for interventions in architectural Cultural Heritage.Main novelties concern to a semantic-based approach to urabn/building GIS (by adapting the CityGML framework) and Advanced Visualization Tools to improve interaction by using graphical contents.
Our contributions are summarized as follows: In section 2 we provide a general semantic framework for a structural model which allows to connect different components or systems; a crucial tool is given by specific Cultural Heritage Ontologies.Section 3 is focused to explain the usefulness of image segmentation for advanced visualization tools for e-Learning issues; next we use 2D digital support for developing a prototype of interactive e-Learning Tools.Our approach is based on open-source software tools for e-Learning (in the LOM standardised framework), a commonly accepted Ontology for architecture (MACE) and the methodology of clinical sessions for inserting comments, and suggestions in order to provide a support for younger people.We illustrate our approach with two examples concerning to structural issues for very damaged buildings and some remarks involving damaged materials in facades.A short discussion illustrates the obtained results and ongoing work.Some conclusions relative to the achievements are sketched to conclude.

STATEMENT OF THE PROBLEM
To fix ideas, we restrict ourselves to isolated singular cultural heritage buildings.Their integration in a urban or rural environment can be performed by using the CityGML methodology.Usual methodology for architectural surveying of cultural heritage buildings uses image-and range-based modeling of buildings Three-dimensional documentation provides a support to integrate all the available as support.The 3D model must include the localization of pathologies involving structural elements and materials.Recent advances in mathematical modeling and improvements in software tools have allowed 3D reconstructed models according to photogrammetric precision requirements.A recent description, including a comparative analysis can be seen in (Remondino et al., 2012).
Apero-MicMac software for photogrammetric Surveying provides excellent solutions for dense accurate 3D reconstructions, with a high performance even for very complex facades.We have applied this tool to provide a partial reconstruction of the Baroque facade of the University of Valladolid (see figure 1).
The design and implementation of efficient segmentation algorithms for volumetric objects is a hard challenge.The problem becomes even harder in situations as the illustrated by Fig. 1 due to the presence of ornamental elements (sculptures, niches, mouldings, etc) around structural elements (columns, intercolumnar elements, etc).Disparity or depth maps are not enough fine to perform an accurate decomposition for a complex Baroque facade.It is necessary to work at different levels of detail and restrict our attention to simpler 3D models at each level where volumetric segmentation techniques are available.More generally, the automatic recognition of architectural elements (involving to structure and materials) is necessary for assessing conservation and restoration policies on building.
In our case, automatic recognition is focused towards the detection of structural elements and pathologies in materials.In particular, structural pathologies are detected in terms of a "meaningful deviation" (above a threshold) between the grouped data and expected models.Pathologies in materials are treated in terms of superimposed layers to a structural representation; in precedent works we have developed some image-based tools for the automatic identification of fissures or cracks; our precedent work concerns the appearances which can be detected from several views and it must be completed by means of a scientific analysis involving (atmospheric, environmental, chemical) factors which modify biological and geological aspects of materials.Hence, our contribution can be understood as a digital support to superimpose different layers in a very similar way to a GIS.

Clustering and simplification techniques
Clustering and simplification techniques concern mainly to architectural objects appearing in their digital representations.Our approach is focused towards the detection of structural aspects (components and their interrelations) and materials analysis (stone, wood, masonry, etc).In some cases, the availability of a 3D model provides important cues to ease detection of structural pathologies.A typical strategy for detecting structural failures from a 3D model consists of the following steps (1) construct a complete 3D model; (2) compute slices which are perpendicular to each principal axis of a triply orthogonal system linked to the building; (3) compare slices by re-projection on a common plane; (4) evaluate an interpret differences above a fixed threshold.This methodology is illustrated in (Jose et al., 2007).Nevertheless the existence of a 3D model, diagnoses of structural pathologies is performed by comparing different slices between them which are orthogonal to each axis of the building.Thus, it can be considered as a hybrid approach which combines range-based 3D models and image-based plane sections.

Towards an architectural taxonomy
For lowering the interactivity level it is necessary to develop a taxonomy (set of logical rules) for architectural primitives.The lowest level concerns to geometric aspects for structural elements which are implicit in the use of design tools.An architectural taxonomy is a set of logical rules acting on architectural primitives which allows to connect basic elements between them.There are three types of rules which can be labeled as logic of classes (validation of a key word), propositional logic (verification of attributes of materials or parameters describing geometric primitives) and/or descriptive logic (validation of an assertion).A lowlevel semantics for graphical representation uses basic geometric primitives as key words of the lexiccon; each architectural element can support different meanings which appear associated to different thessauri which are managed by the above taxonomies.Hence, taxonomies are in charge of knowledge management by means of different kinds of logic (classes, propositional or descriptive).We have implemented the two former ones, but the third one requires implicit knowledge whose formalization is a very difficult task.It can be incorporated in terms of comments performed by authorized experts.
The structural/materials analysis and their comparison with other similar buildings included in our database provides the support to identify the most urgent tasks to be performed.Interventions must be respectful with style, structures and materials; thus, it is necessary to perform hypothesis about the historic evolution of the whole fabrics and its environment.Well-founded hypotheses are based in previous knowledge of architectural techniques performed along historical periods where different parts were built.These tasks require a cooperation between different experts (arising from Architecture, Arts, History, Photogrammetry) which has been accomplished from the late nineties.A meaningful example based in the integration of different image-and range-based techniques is described in (Jose et al., 2007).Nowadays, we have implemented software tools for image-and range-based 3D modeling which provides a support for the coupling between DIMaS.

ADVANCED VISUALIZATION TOOLS FOR CONSERVATION AND RESTORATION
Furthermore monitoring buildings for detecting structural or materials, it is necessary to perform diagnoses, recommendations for interventions and tracking interventions.Volumetric models provide a support to include all these functionalities in a common software application which combines evolution along time (aligning of different representations linked to a unique model), access and update of databases (linked to different multimedia support), consults to external tools for evaluating structural problems and/or pathologies in materials.Advanced Visualization Tools allow to integrate all these functionalities in a structured way with multiple purposes.
From academic viewpoint Advanced Visualization Tools (AVT) increase scientific / professional productivity and provide an excellent support for communicate and collaborate around shared documents and applications.Hence, AVT play a central role for integrating different tasks involving the above Systems and services, such as e-Learning tools.Next, we present some AVT for architectural cultural heritage; we have adopted the nearest viewpoint to the final user which is given by students or construction workers in charge of intervention (conservation and restoration) tasks by following recommendations performed by experts.Nevertheless the volumetric character of our model, usual client is the administration which requires CAD 2D documentation for representing, identifying and tracking interventions.In addition final users have some troubles for interactive visualization of volumetric objects.Thus, we have adopted a down-to-earth approach and we have developed main applications of AVT and e-Learning tools for 2D representations.
The key for integrating the above DIMaS in a common AVT application which can be managed by using ordinary language is the specification of an appropriate cultural heritage Ontology giving support to the above three Systems.Along the Singular Strategic Project Patrac, we have developed an Ontology for architectural Cultural Heritage with specific applications to solve accessibility issues in terms of structural problems and specific materials with different protection degrees.Linked to this Ontology, we have developed a software tool for assessing decision making in conflict cases which can become useful for administrative entities in charge of Conservation and rehabilitation practices.
It is very important to have a database of architectural pathologies and proposed solutions involving them in showcases.The above figures display examples of different tasks, adapted to the proposed hierarchy for DIMaS through a common AVT.The software application for e-Learning tools has in account the above hierarchy to illustrate the steps to be done.This goal can be achieved at each step by identifying the components which must be successively activated with the right solutions (see below).
Proposed or executed interventions are compatible with AVT which are referred to existing repositories and/or databases of multimedia contents involving cultural heritage buildings.Our compatible is compatible with simple repositories or advanced semantic based databases; in the simplest cases the access is performed at the lowest level, i.e. by using key words involving contents of multimedia files.An example is given by the repository "Las ruinas de Dios" (Las Ruinas de Dios, 2013); this repository illustrates a large amount of cultural heritage buildings of the province of Valladolid (Spain) existing in the eighties.Unfortunately, around one third of these buildings have been demolished along last twenty years and no one intervention is currently possible.However, they provide materials for showcases and possible intervention strategies.
There exist several open source tools to integrate the above Systems in a common Visualization framework (see (CityGML, 2013)).Seemingly, neither of them has been applied to cultural heritage environments, including urban district or singular buildings which concern to our work.We have adopted a solution which is based in Collada (see Figure 2).
Currently, there exists a large diversity of solutions involving each one of systems appearing in DIMaS.To be more effective, it is necessary to bound the reaching of our solutions.The lack of a universally acknowledged standard requires the development of Seemingly, it is the first known example for integrating Information and Management Systems applied to very large AEC environments.

A SUPPORT FOR TEACHING CONSERVATION TASKS
From the academic viewpoint, it would be desirable that all the available information would be accessible to ease their reuse by other experts.It is necessary an additional effort in the development of collaborative environments for e-Learning cultural heritage.In the meantime, the Lab of Architectural Photogrammetry (High School of Architecture, Valadolid) has a large amount of surveyed cultural heritage buildings in the province of Valladolid (Spain); unfortunately around 40% have already disappeared, but their documentation allows to identify pathologies and propose solutions.All these materials are collected and included in an old digital repository which is ordered according to principles of Web 2.0; the documentation is performed by following RDF schemata.Thus, it is possible to migrate to a management in the Web 3.0 framework according to the above semantic principles.

A structural approach
Teaching intervention tasks in cultural heritage buildings involves to the development of a software application for teaching and training intervention tasks in cultural heritage buildings and their environment.Its Spanish acronym is ATRAECOM which corresponds to Learning of Rehabilitation in AEC environments on different Multimedia Support which are referred to 2D or 3D labeled documents.Our software application allows the creation of on-line courses and the management of e-learning and web cooperation.Some currently available functionalities concern to users management, courses draft, integration of advanced metadata for cultural heritage contents (beyond usual DCS, which are recommended by the EU), different access levels (depending on user's knowledge and their interests).All of them are integrated in a common visualization framework which is compatible with different multimedia contents for each cultural heritage building and its environment.
The most common structural approaches interpret a building as a collection of architectural elements (basic "cells") which are interconnected between them (relations) and which support the whole structure.From modeling viewpoint, this description can be considered as a representation based on boundaries of surfaces; this description is typical in AEC environments, which is the most convenient framework for recent construction involving Documentation and Management Systems.The software e-Learning application must include constructive aspects (involving documentation and management systems) and contextualization/evaluation (involving information systems).Thus, we have adopted a hybrid approach which combines both approaches.The alignment between corresponding Ontologies is developed in (Delgado et al., 2013).

Some functionalities and standardization issues
Our application (a) has a modular design which provides a support for adapting already existing methodologies, (b) eases the reuse of any type of multimedia information and the web access to internal contents of course, (c) incorporates tools for query processes on materials contained in the repository, and (d) allows to obtain an evaluation of the acquired knowledge in teaching and training processes.To avoid typical problems of professional GIS, a simple interface has been implemented which do not require advanced knowledge from the user viewpoint.The next figure displays the functionality linked to the Glossary management We have adopted international standards to provide a general framework of our software application.In our case, the development of our software application is given by LCMS (Learning Content Management Systems), where there exist different managers which allow to administrate, distributed and control the formation performance in terms of teaching and training activities.This framework allows to follow learning processes, perform evaluations, generate reports, and manage communication services.Our application is organized following a hierarchy organized in several layers (with their corresponding standards): (1) Communications (TCP/IP standards), (2) Formats of educational files (XML, HTML, Macromedia Flash), (4) Metadata schemes including learning applications (DCS and LOM IEEE standard), (5) Organization of courses (following an adaptation of Claroline), (6) Web portability to allow their reconstruction in target system, (7) profiles and permissions for each agent taking part in teaching / training and learning processes, (8) linguistic, cultural and social adequacy to different contexts.
The most difficult part concerns to the standarization of learning process which has been developed by following principles suggested by the IMS Learning Design (IMS Learning Design, 2013).This choice is justified by the availability of resources (implementation guide, information model, XML documental schemes) and the cooperation policy with a very large of entities (IEEE LTSC between them).We have performed several meaningful advances in the integration of different systems in terms of a specific shared Ontology.
Hardest problems concern to interoperability issues with other tools which has been solved by aligning Ontologies.Currently, the alignment is only automatic for lexicon; next step concerns to automatic alignment in terms of automatic recognition of components from image or range information.A higher level concerns to identify their role for structural aspects or to evaluate pathologies of materials from image/range processing and analysis.We have detected some structural problems such as lack of verticality, aligning between components or humidity and fissure/cracks in walls.However, there is a long road to be performed, still.In particular, the hardest challenge involves to the extension of the above method to 3D visualization framework.This extension is based on volumetric segmentation tools which have been partially developed, but not still integrated in the e-Learning application.
Nevertheless the complexity of semantic-based approaches, our software developments can be applied to traditional repositories such as "Las ruinas de Dios" which provides a good collection of examples (unfortunately, only in Spanish language).
In the e-Learning knowledge domain, we have adapted MACE solutions for teaching / training in Architecture to Cultural Heritage environments by making compatible different standards.However, the database for our application is very small, still.Further contributions are need to extend the reaching of the performed approach.Our ontology-based approach makes possible the exchange of materials with other larger repositories which share the same Ontology and methodology.In other cases, it is necessary to perform an alignment which implies a loss of information.We illustrate our approach with two lessons.Even if the available lessons are very meaningful, due to the lack of economic resources, our software application for e-Learning can be considered as a prototype of an innovative Learning Project which has been recently awarded to our group.To be more efficient from teaching and training viewpoint, it is necessary to incorporate new applications, new cases of use, extend methodologies, incorporate good practices and implicit knowledge of experts in conservation and restoration of architectural Cultural Heritage objects.

CONCLUSIONS
We have developed a software prototype which is based on international standards for e-learning of conservation and rehabilitation tasks in Architectural Cultural Heritage.Our application provides a support for 2 types of users: professionals and workers of construction sector.We reuse materials arising from standards involving AEC environments (for Documentation and Management Systems) and CityGML (for Information Systems).
Efficiency of interoperability issues is warranted by the development of standards and Ontologies compatible with such standards.Our approach uses a cultural heritage Ontology which extends usual standards and allows to connect Documentation, Information and Management Systems between them.This Ontology has been originally designed to solve physical and digital accessibility issues.In this work, we display how it can be applied to more general aspects related to rehabilitation strategies.
A key software application for exchanging information is given by advanced Visualization tools which are designed in a transversal way for connecting different components of DIMaS.We have illustrated our approach with some examples which have been mainly applied to old churches in rural zones in a very bad state of conservation.Our Visualization application has been designed and implemented to support different functionalities; in particular, it is scalable, compatible with different formats, allows to connect with standard tools for urban/building GIS and provides a support for identifying and tracking interventions by different agents participating in rehabilitation tasks.
We have integrated several standards in an e-Learning software application which is applied to constructive environments in cultural heritage buildings.These standards concern to teaching and training methodologies under LOM umbrella, with a special regard to Claroline-based applications.A careful design of interface allows to interact with the learning application, including some meaningful examples of Architectural cultural heritage objects with very damaged structures or very deteriorated materials.

Figure 1 :
Figure 1: Partial reconstruction of the Baroque facade of the University of Valladolid with Micmac extension software.

Figure 2 :
Figure 2: 3D WebGL visualization of the Collada model of "La Pedrera" (Barcelona) from Google 3D Warehouse knowledge transfer between different open source tools.An additional problem arises from the different concepts, goals and functionalities even for systems belonging to the same type.Interoperability issues between knowledge representations (including the above three Systems) is perhaps one of the most outstanding problems.At lexicon level of each Ontology, this problem can be solved by following fusion or aligning strategies: (a) Information fusion is very expensive from the computational viewpoint, because it duplicates a lot of information; (b) information aligning is less expensive, but the automatic aligning is based only on key words involving descriptors of digital contents.We have developed software tools for aligning open source applications involving Urban/Building Information Systems and Construction Systems for rehabilitation interventions such as CityGML (as OGC standard) and BIM (Building Information Modeling) in the AEC (Architectural, Engineering, Construction) framework.Even if the aligning involves only to a small part of lexicon, it is enough meaningful for connecting Information and Management System.Seemingly, it is the first known example for integrating Information and Management Systems applied to very large AEC environments.

Figure 5
Figure 5 exemplifies structural pathologies related to deformations and fissures.The former ones are evaluated in terms of lack of verticality or alignment by comparing families of sections which are parallel to a dominant plane.The most urgent interventions concern to the consolidation of walls and the insertion of additional elements.The construction of scaffolding for building vaults and a peripheral metal band in the upper zone will allow to stabilize the whole structure to warrant the stability of the building with a right distribution of charges in its new architectural covering.

Figure 4 :
Figure 4: Reconstruction and intervention phase of a damage cultural heritage building.