INTEGRATION OF 3D SPATIAL INFORMATION FOR MULTI-MODAL EXPERIENCE OF THE URBAN ARCHIVE

: This paper focused on the case of Semal Village, the region that will be demolished due to undergoing recent redevelopment in Paju. We experimented utilizing 3D spatial information to capture precise urban morphology of present status before redevelopment and to transform such data into multi-modal experience of the urban archive. We investigated feasible techniques of photogrammetry and hybrid 3D modelling that could render the situated reality of the village communicatively with various medium such as 3D-printed scale model, and augmented reality (AR) contents. Through drone photography, the entire Semal village was captured, and three-dimensional data were obtained using photogrammetry. Along with information from on-site surveys, the mesh model was segmented into buildings, terrain, and vegetation for focused work. Each models reconstructed and retextured using images from the on-site surveys. All data were compiled for full-colour 3D printing and assembly. The 3D-printed scale model replicating Semal is on display at the Paju Central Library. Additionally, an AR content was created using the 3D-printed scale model and ethnographic data, aiming to archive and share people's memories, thereby continuously building a sustainable archive of the village.


INTRODUCTION 1.1 Background
In South Korea, there are 15 cities located near the demilitarized zone (DMZ), the official border area against North Korea.The city of Paju is one of these cities where the live history of the divided nation is well preserved.Because Paju is adjacent to the Seoul metropolitan territory, the harsh demands of the housing market of Seoul have affected development policy and expedited redevelopment.The rapid transformation from natural settlement to high-rise apartment complex causes massive loss of historiographical resources as well as collective memories of the contemporary society of Korea.
The Paju Central Library has put consistent efforts on documentation of such demolished villages by ethnographic research and publication.Especially, to announce the documentation of the village and to show it to the local residents, library intended to create a 3D-printed scale model representing the village.However, there was a lack of data for creating the model because of the limitations of the border area, so it was necessary to begin by collecting data about Semal.In order to reproduce and display the appearance of Semal accurately, we intended to use 3D-scanned data, which could be collected through drones and terrestrial scanners.Along with scanning, to acquire field information that was difficult to obtain through scanning, on-site survey was conducted by walking around the village.By synthesizing data collected through various methods, we were able to proceed with modelling for output.

Urban Evolution in South Korea
South Korea's urban landscape, especially in major cities like Seoul, has undergone significant transformation since the late 20th century.This change, characterized by the shift from traditional housing to modern high-rises and commercial complexes, is rooted in the country's rapid urbanization and economic growth post the Korean War.Government-led urban planning, largely focusing on efficiency and modernization, has been pivotal in this transition.The 1970s and 1980s redevelopment projects, particularly in Seoul under the New Village Movement, exemplify this shift. 1However, this modernization has sparked debates about cultural identity and heritage preservation, with concerns about the sustainability of such growth and its social impacts, including community displacement and gentrification.
In response to these challenges, South Korea is increasingly embracing urban redevelopment methods that prioritize sustainability and balance development with preservation.This approach focuses on preserving and utilizing historical aspects of cities, such as historic buildings and local characteristics, which are integral to a city's unique identity.By involving residents in the regeneration process, projects become more inclusive and successful, blending various perspectives to achieve harmonious urban development.This strategy aims to guide cities toward a sustainable, prosperous future, while maintaining a delicate balance between progress and conservation of history.

Semal Village: From Past to Present
Semal Village was one of the six villages in Geumchon, Paju, Gyeonggi Province.During the Korean War in 1950, residents of the Jangdan-gun located above Paju fled south.At that time, the government relocated refugees further south, but a significant number refused and settled in Geumchon, near their hometowns.As a result, temporary shelters were established in the Geumchon area, and those who couldn't return to their hometowns settled in Geumchon after the armistice.One of these areas is the northern region of Semal Village, where people started building huts and living without altering the landscape.
The village's alleys were irregularly shaped along the buildings, and due to the narrow space, many houses were attached to each other, often two or three houses attached in a row.Among the old houses, there were many traditional wooden houses built using traditional construction techniques.The old Hanok 2 houses built after the Korean War and masonry houses constructed by refugees were also remained in the area.In the 1990s, as the new town development trend around Seoul, apartments were also constructed to Geumchon.Semal Village turned into an island surrounded by apartment complexes.Until the redevelopment of the 148,888 square meter Semal area in 2023, it remained untouched by urban planning, preserving the old appearance of Geumchon.

Archiving Efforts of Paju Central Library
Paju Central Library has been active to archive the memories and ethnographic data which can be shared by the residents.In addition, the library provides support to build a strong community identity.Library provides various courses and programs for the residents to enhance their understanding of Paju and enable self-improvement through record-based activities.Through these programs, citizens have acquired opportunities to engage actively in community archive as a Citizen Archivists and the Paju Citizen Archive Network.
As for the Semal Village, the archive project was planned to preserve the village's morphology, history, and the memories of residents, when the redevelopment project over Semal area by the library.From 2021 to 2023, the Paju Citizen Archivists conducted oral interviews, and a research team from the Department of Anthropology at Seoul National University investigated the village's history, seasonal customs, livelihoods, and social organization, resulting in the publication of a book (Figure2).The book consists of two main chapters: one documenting the history and lifestyle of the Semal village, and the other covering interviews with residents. 2Traditional Korean house built using conventional Korean architectural styles.
Meanwhile, to collect and archive the spatial information of Semal, the library requested our research team to develop a meaningful documentation of the built environment of the village.The team of guga Urban Architecture and TechCapsule conducted aerial drone photogrammetry, terrestrial laser scanning, and on-site field surveys, along with the drafting of drawings.Based on these data, we aimed to provide an opportunity to get an overview of Semal by creating videos and a 3D scaled model.

Urban Archive Multi-modal Experience
Multi-modal urban archiving projects make urban space understandable and comprehensive through a diverse range of senses.Integrating various mediums such as text descriptions, photographs, videos, audio recordings, maps, and graphics, they have provided users with a rich experience.As technology advances, these projects have begun utilizing an even wider array of data.
Piga et al. accumulated visual simulations of the urban environment and related materials of its changes.Based on these data, they have confirmed that media such as panorama images, videos, and rendered images, along with their accuracy, influence the sense archive of the urban environment.Additionally, they designed and implemented an interface for exploring data containing spatial information through virtual tours recalling urban experiences, aiming to assist users in interpreting and understanding the city.
Efstathia et al. stated that location information, media, and augmented reality (AR) technology are important for preserving, sharing, experiencing the stories of significant places.Additionally, in this paper, they explored the issue of urban archiving through AR by following the flow of stories along planned walking routes.They set tasks that allow users to actively participate in the system, enabling them to have repeated hybrid experiences.Daniele Calisi et al. aimed to explore new methods of communication for cultural landscape heritage that combine interest and information to attract users, and encourage exploration of virtual environments, by creating VR interactive content.They reproduced the Temple of Valadier through 3D scanning and modelling.Moreover, enhanced the realism by incorporating sounds recorded on-site into the content.

3D Modelling of Village Landscapes
Bernadette et al. focused on the indigenous villages of Kutch, Gujarat in India, which were rebuilt after an earthquake.They utilized 3D laser scanning technology as a method for heritage preservation and post-disaster reconstruction.This technology enabled assessment and design for repair, renovation, and reuse aimed at reducing heritage loss due to building replacement or demolition.To date, the historical value and significance of the buildings were not considered as important as restoring them to their pre-damage state.As an alternative, they employed a method of replicating and recording the buildings down to millimetre precision through 3D scanning, for modelling and documentation purposes.

Kuklina et al. conducted research into the necessary processes
for creating accurate geometries of models in water bodies.They addressed key modelling issues related to scanner and photogrammetry processing capabilities and provided conventional methods for generating mesh models.To resolve these issues, they utilized the boundaries of water bodies.By assuming a constant water level based on these boundaries, they proposed a new modelling method that involves connecting boundary points.
Martina et al. aimed to explore the concept and implementation of smart cities, focusing on representing data from various sources within digital urban models.This was done to facilitate decision-making for the growing urban population as well as to enhance the quality of life.They utilized spatial information such as the heights of buildings, land, and trees to model the city.Furthermore, they conducted research on methods to analyse wind at the building scale using this composite data.

DATA COLLECTION AND SURVEY METHDOLOGY
On-site survey of the region was initially conducted, due to the limited digital resources in the Paju Semal village as a border city.The Semal village was digitized using laser scanning and photogrammetry techniques, both from the aerial and on the ground.Ultimately, the goal was to reconstruct the collected data in various ways for various outputs.
Figure 3.The data processing procedure for the 3D-printed scale model and AR interactive content.

On-Site Field Survey
On-site field survey was carried out on site with architectural designers 3 , after conducting desk-based research on maps and the history of the Semal village.These investigations included not only a study of the historical context and specific conditions but also the collection of visual materials and interviews.
Based on online maps, the survey team navigated the area to collect data.They were able to identify narrow alleys which are not marked on the maps, consisting of low buildings and densely packed structures, as well as numerous paths only traversable by pedestrians.Due to the presence of many historical buildings, architectural features were recorded, along with the positions of walls, entrances of buildings, and their height and form.Additionally, sketches and photographs were taken of the location, size, and types of trees.Furthermore, photography and videography were conducted at various locations to document the appearance of places that might soon disappear, thus laying the groundwork for various outputs.

Terrestrial Laser Scanning
Terrestrial laser scanning in the Semal village were not applied across the entire area, instead, the research team focused on four buildings with historical significance, including Semal inns and traditional stone-tile houses.The survey was carried out using a FARO 360 FocusM LiDAR scanner with an accuracy of 0.7 mm from 25m. capturing both the interiors and exteriors of the buildings.

3D MODEL RECONSTRUCTION FOR 3D PRINTING
The 3D-printed scale model was chosen as the medium to depict the three-dimensional space, aiming to document and showcase the morphology of Semal Village.This method was employed to create publicly accessible visual material that effectively represents the evolving urban landscape.

Data Integration For 3D Mesh Creation
The data obtained from drone photography were aligned and matched using the 'Reality Capture' photogrammetry tool, enabling the creation of a high-poly mesh model with a highresolution texture.Although it was possible to form a mesh model of very high resolution, this caused the necessity to make some technical choices to answer themes and difficulties imposed by 3D-printng.Since the model was constructed based on drone images, the textures of building roofs were wellrepresented, but the facades of buildings in densely packed areas or with heavy shadows were not clearly depicted.Additionally, many alleys were obscured by buildings.The photography was taken in November, resulting in numerous leafless trees which did not appear in the 3D data and were represented in a clustered form in the model.Furthermore, in some areas, varying land heights created cliff-like formations in the middle of roads.This necessitated additional data processing for the utilization of the model in 3D printing.

Modelling Strategy
We reconstructed 3D model for the purpose of full-colour 3D printing at a 1:500 scale, commonly used in urban planning models to create a model that includes the 148,888 ㎡ Semal area and its surrounding regions.In the entire model, measuring 1250mm by 1050mm, we focused on representing the disappearing Semal area's topography, architectural features, and atmosphere.
The textured mesh model obtained through photogrammetry consisted of a single mesh model, including buildings, vegetation, and land.In addition, textures and UVs were not distinguished.Such a unified model form made image modification difficult and combining it with other data was not straightforward.In the 3D-printed scale model, rather than depicting the village just before redevelopment, our goal was to show the ambiance of the village as it was lived in, necessitating the combination of various data.
In the village scale model replicating Semal, we identified buildings, land, and vegetation as key elements, each representing distinctive features.Buildings, as artifact structures, held unique formal features.The land presented the natural feature such as landscape.Vegetations were displayed complex and organic forms due to numerous leaves and branches.In order to accentuate the distinctive features of each element, we divided the model into three components: buildings, land, and vegetation.
Figure 7.It shows the process for 3D-scaled model using the mesh model obtained by drone and on-site field data.

Land Model Terrain Segmentation for Ground Base
The land model had to be generated by removing the building parts from the overall mesh model, so the work was carried out after the building models were made.After removing the building models, we smoothed and flattened terrain errors caused by roads and densely packed houses.In the case of flat fields, grooves were created to add a sense of liveliness.We also removed or flattened areas with vegetations to align with the terrain.Small geometries such as grass and bushes were remained throughout the process to preserve a natural feature.We simplified the polygon count by up to 20% through simplification, as the geometry presented no issues for the intended 1:500 scale printing.We resolved duplicated surfaces partially through automatic correction, while remaining issues were meticulously handled by manually deleting and recreating.
We opted for the method of editing orthographic images for projection texturing to represent the village's ambiance, thereby preserving the model's overall tone.The shadows in the orthographic images were lightened and obscured paths caused by vegetation were revealed because of the tendency of dark areas to appear overly intense in prints.Additionally, to revive the region's vitality, areas like desolate fields or damaged roofs were edited.Organic Parting for Printing Size Capacity The maximum print size was limited to 332mm in width and 190mm in length.Therefore, the printing area had to be divided into 34 pieces.We created a parting plan suitable for the print area using orthographic images.Instead of simply dividing it into a grid format, we opted for an organic parting following the alleys, aiming to minimize gaps, and warping during the printing process that could make the model appear fragmented.We divided the model after completing the work on the terrain model.
Figure 7. Divided model into 34 pieces along the alley to fit the print size of 3D-printer.

Building Model
Distinct Roofscape vs Vague Façade Isolating and utilizing the buildings mesh from the existing mesh also presented challenges.When contemplating the exhibition of the 1:500 scale model, the lack of facade information from drone photogrammetry did not pose a significant concern.However, for some buildings such as isolated buildings, or taller structures, where facades are more visible, we edited the texture images.Because buildings had artificial features, we determined that modelling new buildings with simple NURBS surface.Additionally, it's efficient than reconstructing the topology of the mesh model of visible buildings and reproducing the sides of each low building.In this way, building models were constructed manually using the 'Rhino 7' referred to orthographic images and the overall mesh model.A total of 509 building models were created.
Figure 11.It shows the appearance of the building in the raw mesh model.
The only issue to resolve was the texture mapping for printing for the manually modelled building models.For the low building models, we grouped texture UVs by piece and automatically generated using the texture reprojection feature of 'Reality Capture', which was based on drone photogrammetry.However, for building models with more visible facades, we created texture UVs individually for each structure.After reprojecting in 'Reality Capture', the visible facade textures were edited using photographs to produce clean surfaces.Additionally, some damaged roof textures were further modified using edited images from orthographic projections.overcoming printable thickness The building models were primarily designed in a box shape, and specific details were omitted, which could later be replaced with textures, such as windows.However, given the importance of the bird eye view perspective, we conducted simplifying or exaggerated modelling to include details in order to adjust into printing capacity such as parapets and gabled roofs with eaves.
Shelling was performed to minimize deformation in preparation for printing.The consistent thickness was necessary for stable 3D printing.Therefore, we hollow out the inside of the building models to achieve a uniform thickness of 3mm, from the terrain to the buildings.We offset All surfaces by 3mm, resulting in a concave shape model from the terrain to the buildings when viewed from behind.
Figure 14.The grey part represents section of the building and the land model after shelling.

Vegetation Model
Vegetation Models: Organic Wireframe to Volumetric We decided vegetation models to use volumetric models.In a 1:500 scale model, the vegetation models have inherent limitations in terms of representational morphology.We trade off detail lushness of the tree with the outline of the shape.We produce vegetation models using the 'Tree generator' to create volumetric models.However, the trunks of these trees were too thin for printing, so they were altered to have a diameter of 2mm for print feasibility.Additionally, closed tree models form that maintained the lush appearance were created with fewer polygons (Figure 16). 4 Winter Hinterland to Coloured Autumn The species, sizes, and colours of the trees were determined based on information collected through on-site survey.The 26 different form of tree models produced in the reconstruction phase were placed in appropriate positions (Figure 17).For the 4 The 'shrink wrap' command in 'Rhino 8 WIP' was used.
image textures, we designed a combination of autumnal gradient colours to capture the essence of autumn trees.

Hybrid Modelling & 3D Printing
Finally, the completion of all modelling, for printing purposes, models of buildings, land, and vegetations were compiled then parted into 34 pieces.Jet Fusion 580 3D printer 5 , capable of colour 3D printing, was utilized to accurately convey the ambiance of the Semal village as captured by drone photogrammetry.Although there was a height difference of about 4cm between the lowest and highest parts of the terrain, this variation was not entirely addressed with 3D-printing.Instead, we extended only the visible outer surfaces to the bottom with a thickness of 3mm.The remaining surfaces were designed with a grid mesh up to 5mm in height at the base of the land model, considering deformation prevention during printing and the height of other materials used for support in assembly.
Issues commonly arose from overlapping lines or remaining unjoined meshes.For the vegetation, we performed automatic corrections and proceeded without making modifications to the model even in cases where issues with UV deformation arose, as there was no discernible difference in colour in the tree models.
Figure 18.It is an image of a 3D-printed scale model that has been assembled.
We compared the models with their digital counterparts once printed.In one instance, it was observed that two building models were absent from a single model.Consequently, a separate printing process was undertaken to produce the missing building models, which were then affixed accordingly.As for the vegetation, despite the attempt to reinforce the trunks, they were prone to breakage.Therefore, supplementary prints were processed and attached.
Currently, these models are on display at the Paju Central Library, serving to evoke memories of the disappeared local landscape among the community residents.

TOWARDS MULTI-MODAL EXPERINCE FOR THE URBAN ARCHIVE
The collected data, available in various forms and formats, was not only transformed into accurate representations of the village's site plan, detailed building plans and sections, but also into 3D-printed scale models, utilizing on-site surveys and scan data.The essence of our strategy was to transcend these static models, elevating them into a dynamic, multi-modal medium.
We prototyped Augmented Reality (AR) content, a deliberate move to animate the static data with layers of interactive storytelling.It aimed to convey the village's stories such as history and lifestyle of Semal village.Furthermore, AR content facilitate the sharing and collecting of residents' memories, thus adding a layer of interactivity to the 3D-printed scale model, which was previously limited to visual representation.
The content of this AR experience was primarily derived from the material published in a book by the library.We restructured the history and interview chapters of the book into distinct scenes.In one scene where the history chapter is reconstructed, users could explore the summarized stories of Semal, akin to a digital adaptation of the book.This provided a comprehensive overview of the village's history, segmented into thematic sections enriched with content and images.In contrast, the other scene, interview of residents restructured to align with specific locations within the village.This reconfiguration transformed the documented interviews into narratives, intimately tied to specific locations in Semal Village.We carefully crafted each narrative to be evocative, aiming to kindle empathy among users and inviting them to contribute their own stories.Moreover, it allowed users to explore the 3D-printed model, now annotated with interactive icons representing specific locations tied to personal memories.In addition, for areas where ground-based LiDAR scanning was conducted, we developed internal 360° panoramic views, enabling users to experience the space.
Figure 19.Markers are displayed on the 3D printed model.
The objective was to awaken the recollections of residents by engaging with the memories of others.Empathizing with shared memories fosters communication among those with similar experiences.Additionally, archiving stories about new locales triggers novel interactions.This accumulation of stories not only documents a spectrum of memories from various individuals but also transcends traditional static documentation.It evolves into a dynamic archiving system, continuously enriched with new and diverse memories.

CONCLUSIONS
This study explored the transformation and archiving of Semal Village in Paju, highlighting the importance of a multi-modal approach in urban archiving.We examined the rapid urbanization experienced by Semal Village and the subsequent loss of its cultural and historical value and explored methods to archive these changes.In addition, reaffirms the importance of urban archiving, demonstrating how the integration of digital technology and ethnography can aid in preserving community memories and identity.We have shown that utilizing technologies like 3D scanning, modelling, and 3D-printing to visually document spatial transformations can strengthen connections with local communities.3D-printed scale models were produced using urban data acquired through on-site surveys and mesh data acquired through drone scans.AR content was established using the 3D-printed scale models and information obtained through ethnographic research.These are the results that combine 3D spatial information with the traditional form of ethnographic archives.Additionally, visual materials that serve as a factor that evokes people's memories are on exhibition at the Paju Central Library.If combined with the progressive integration of various media types and 3D spatial information, along with an online system, it can become a sustainable archive that accumulates memories over time, rather than being a static record like a book or document.Future research should apply this approach to various urban environments and expand its effectiveness through collaboration with more local communities.Additionally, there is a need to explore more comprehensive and sustainable approaches to urban planning and development using the archived materials.

Figure 2 .
Figure 2. The book published by the library.The current page shows the interview of resident.

Figure 9 .
Figure 9.The ortho-image(left) was transformed into bright and vibrant image(right).

Figure 10 .
Figure 10.The land model which was project-mapped with edited ortho-image.

Figure 12 .
Figure 12.Manually modelled building(left) and manually textured model which has clear texture.

Figure 13 .
Figure 13.Manually modelled building(left).Automatically textured model which has the distinct roof and the vague facade.

Figure 15 .
Figure 15.The appearance of the vegetation in the raw mesh model.

Figure 17 .
Figure 17.Various types of vegetation models(left) and various colours of vegetations(right).