A STUDY ON MULTI-MODELING FOR ARTIFACT RESTORATION

: Restoration is carried out in order to restore damaged antiquities to their original form. Recently, research has been active in obtaining the target three-dimensional (3D) model by optical scanning and X-ray computed tomography (CT), as well as restoring artifacts via modeling. Because the cultural artifacts are restored in a virtual environment without direct contact with the cultural artifacts, secondary damage can be preserved. Furthermore, the restoration results can be used for the physical restoration of cultural artifacts via 3D printing. In this study, multi-digital modeling technology was used to restoration horse-shaped earthenware artifact that still had its missing part. After 3D printing, the restoration part was applied to the artifact. In order to record the shape of the artifact in detail, a high-precision 3D scanner was used to construct the shape of the artifact into a 3D model. Two models were then built as a reference for restoration: The legs of the excavated horse-shaped earthenware are believed to have been bent during burning, but they were reproduced in their entirety for restoration by conducting rigging and keyframe animation. For the restoration of the missing horse body, horse and rider-shaped vessels excavated from the same site were used as references. However, since the decorations on the reference surfaces obscured the horse body, a 3D model of the horse shape was obtained through X-ray CT segmentation modeling. Using the obtained model, the missing 3D horse body was restored. For the hind legs, the leg geometry reconstructed by the rigging was used as a reference. The hollow space inside the artifact and the structures that could be restored were further modeled. The finished model was completed by outputting a mock-up using a material extrusion system 3D printer, conducting a bonding test with actual artifacts


INTRODUCION
Artifacts are damaged by interventional or environmental causes and are restored in a variety of ways. The most common method of restoration is the manual in which materials are brought in direct contact with the surface of the artifact; however, this method may cause secondary damage and is difficult to apply when the area of loss is large. To supplement this method, research on the digital restoration of artifacts utilizing various technologies is conducted (Fatuzzo et al., 2011;Kaja et al., 2012;Di Paola et al., 2017;Stergios et al., 2018;Rodríguez et al., 2018;Gherardini et al, 2018;Jo and Hong, 2019;Baya et al., 2020;Makris et al., 2021;María et al., 2021;Shui and Gao, 2021;Radu et al, 2022;Nikolaos et al., 2023).
Digital restoration is a method of restoring damaged cultural heritage in a virtual space rather than a physical environment. Generally, a three-dimensional (3D) model of the artifact is obtained through 3D scanning, photogrammetry, etc., and then a restoration plan is established through historical research, followed by modeling. There are various methods of modeling which can be used not only for digital restoration but also for the preparation of basic research materials, recreating the past form, and the reproduced artifact itself (Hayet and Karima, 2020;Ioannis et al., 2021;Jo and Lee, 2021;Arapakopoulos et al., 2022;Kim et al., 2022;Óscar et al., 2022).
Herein, a virtual restoration that horse-shaped earthenware with more than half of it missing was performed by employing the multi-modeling technology. Horse and rider-shaped vessels excavated from the same site were used to establish a guideline for the restoration. Specifically, archived X-ray computed tomography and 3D scanning data were utilized. The results were 3D printed and used for post-processing to restore the artifact.

MATERIAL AND METHODS
The horse-shaped earthenware was excavated from the Geumryeongchong Tomb in Gyeongju. The maximum height is about 56 cm, and the front body of the horse, including the head, chest, and front legs, exists, but the rest of the hind part is missing. The right front hoof is bent to the right, and the right front leg is bent to the left. Additionally, a fragment of what is believed to be a buttock side was found. Due to a large number of missing parts, the horse-shaped earthenware had limitations for manual reconstruction. Therefore, the earthenware was digitally restored using multi-modeling techniques (Figure 1).
In the digital restoration, the geometry of the horse-shaped earthenware was recorded using a high-precision 3D scanner (Space Spider, Artec 3D). Additionally, the bent leg was virtually adjusted using a dedicated modeling software (Geomagic Freeform Plus, 3D Systems) to reconstruct its original shape, and this was used as a basis for the restoration of the hind legs. For the restoration of the missing parts, horse and rider-shaped vessels excavated in the same site were set as references. The decorations on the surface of the reference model were removed from the X-ray computed tomography (CT) software (VGSTUDIO MAX, Volume Graphics), and the geometry was analyzed to establish the restoration guidelines. Based on the established data, multi-modeling was conducted to perform the virtual restoration. The results were then printed using a photopolymerization 3D printer to restore and display the artifact at an exhibition ( Figure 2). The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy

3D scanning and rigging
A 3D scan was performed to record the artifact's shape in high quality and to use it for modeling. Polygon data and the textures of the horse-shaped earthenware were acquired using a highprecision 3D scanner. The average point resolution of the acquired 3D model was 0.3 mm, the number of points was approximately 5 million, and the number of polyfaces was approximately 10 million, which indicates the high quality of the data. The point resolution on the buttock portion piece was 0.1 mm, with approximately 1 million points and nearly 2 million polyfaces represented by a 3D model. Both datasets include high-resolution RGB textures of 16384 × 16384 ( Figure  3).
The horse-shaped earthenware's right front leg is twisted to the left and its front hoof is twisted to the right. To reconstruct the bent leg intact, a skeleton was inserted into the 3D model and rigged to deform its shape. First, the skeleton was set based on the bone structure of a horse. Then, the angles of the leg and the hoof were observed in real-time as they deformed through keyframe animation. The most natural shapes for the leg and hoof were verified through animations that deformed within a range of 0°-15° and 0°-40°. This allowed the leg to be adjusted by approximately 12° and the hoof by approximately 38° from their original angles to reconstruct the horse's body. All models produced were used as the basis for the digital restoration ( Figure 4).  The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy

X-ray CT and Segmentation Modeling
Missing parts of the horse-shaped earthenware was reconstructed based on horse and rider-shaped vessels excavated from the same site, rather than trying to guess the shape of the missing parts. After analyzing the shape of the horse and the rider-shaped vessels, it was found that the horse's eyes and mouth were similar to those of the restoration target; hence, it was selected as a reference ( Figure 5). The overall shape of this remains intact, but the decorations on the horse make it impossible to see the body. To solve this problem, the X-ray CT data was analyzed. X-ray CT is a technique that directs penetrating X-rays on an object to reveal internal structures that are invisible to the naked eye, enabling the analysis without destroying the artifact. Tomographic images can be reconstructed into voxel-based 3D models using algorithms. The 3D model is constructed by setting a threshold between the air layer and the surface of the artifact. The X-ray CT data analysis of the horse and ridershaped vessels revealed the presence of an air layer between the horse's body and the decoration. This suggests that first the horse body was made and then the rider figure and saddle were decorated on top of it.
X-ray CT segmentation modeling was performed in the following order: data optimization, set threshold surface, mesh model generation, and optimization. In the X-ray CT data, there were 2048 tomography images of 2048 × 2048 pixels. To ensure the smooth operation of the dedicated modeling program, the corresponding pixel value was reduced to 512 × 512 for optimization. After determining the surface shape by setting the air layer as a threshold, the horse's body with the decorations removed was acquired. The obtained horse body was meshed, optimized, and the 3D model was completed.
The point resolution of the horse body's 3D model was 0.4 mm; a comparison with an archived 3D scan model with a point resolution of 0.1 mm showed that the surface was represented with low quality. This is presumably due to the optimization of the tomogram pixel count for the modeling. To solve this problem, it was necessary to create a model that fused the highquality geometry obtained from the 3D scanning data with the horse body geometry obtained from the X-ray CT data. Therefore, the 3D scan model was aligned with the X-ray CTbased segmentation model using the same coordinates, and a boolean operation was performed. Consequently, a model was obtained with the high-quality surface of the 3D scan data and preserving the shape of the segmented horse. The point resolution was 0.2 mm, which was better than the quality of the model acquired using the conventional X-ray CT modeling. The created data was selected as a reference model for restoration modelling ( Figure 6).

3D virtual restoration modeling
The virtual restoration of the missing parts of the horse-shaped earthenware was conducted in the following order: historical research, alignment of the original fragments, and modeling of the restoration. First, dimensions and proportions were measured to verify the reference model. The front legs and hind legs of the reference were measured to be 43 mm and 60 mm, respectively, exhibiting a ratio of 1:1.4. The width of the front body was approximately 60 mm, and the width of the hips was approximately 75 mm, exhibiting a ratio of 4:5. Restoration modeling was performed based on the above calculated ratios.
The horse and rider-shaped vessels in the reference model is smaller than the horse-shaped earthenware being studied, making it difficult to restore and utilize. By measuring the dimensions of both the artifacts, it was found that the studied earthenware was 2.8 times larger than the reference model; thus, the reference model was enlarged and utilized accordingly. The artefact's original fragments were aligned on the enlarged horse with respect to the face. The rear width of the reference model was adjusted to match the dimensions of the original fragments such that the ratio of front to rump was 4:5.
The positioning of the hip-shaped fragment was based on expert consulting. From the archival data and the expert's opinion, the shape of the buttocks was taken to be symmetrical based on the decorations on the saddle. Therefore, after establishing a reference axis for modeling, the opposite side was restored through symmetrical modeling. At this time, the cracks and scaling of the original fragments restored by virtual conservation treatment.
The hind legs were reconstructed using the rigged foreleg model. First, the restored forelegs were aligned with the hind legs. The 3D model of the restored forelegs was aligned with the position of the hind legs. For the missing hoof, the opposite original hoof was used for symmetrical modeling. Finally, textures were mapped to compensate for surface heterogeneity between the original form and the restoration (Figure 7). Prior to utilizing the resultant virtual restoration in the exhibit, an assembly structure was modeled for inserting into the empty space of the body to enhance the artifact's stability. A 3D printer was used to output a mock-up to confirm that the restored section and the assembly structure fit well with the artifact. The joint between the mock-up and the artifact was physically verified, and any problems encountered were corrected to complete the final model ( Figure 8). Figure 8. The assembly structure model for increasing the artifact's stability.

3D Printing and Exhibition
A photopolymerization 3D printer was used to print the finished virtual restoration model. An acrylic-based resin was used to create high-quality surface texture for the restoration. The printouts were colored white to ensure that the artifact and the restoration were clearly distinguishable to the naked eye after processing. The finished restoration sections were finally applied to the damaged horse-shaped earthenware and used for a special exhibition at the Gyeongju National Museum ( Figure  9).

DISCUSSION
The resulting digital restoration of the horse-shaped earthenware has a volume of approximately 8986 cm 3 , and a length of approximately 44 cm, indicating that 60% of the total length has been restored. The lengths of the front and hind legs are approximately 270 mm and approximately 215 mm, respectively, in the ratio of 1:1.3. The dimensions of the front torso and hips are approximately 140 mm and 176 mm, respectively, which were restored in a ratio of 4:5, similar to that of the historical research ( Figure 10). In this study, multi-modeling technologies were used to build reference and virtual restoration models for the horse-shaped earthenware. Horse and rider-shaped vessels were used as a reference data through X-ray CT segmentation modeling. Using rigging, the angle of the front hoof of the remaining fragments The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy was modified and used as a reference material for the virtual restoration. Furthermore, the physically restored earthenware via a 3D printing technology was used for a special exhibition. This flexible use of modeling technology can broaden the range of available materials used to restore cultural heritage. This study is expected to aid research on various forms of cultural heritage that to date are not yet verified and restored due to complications with the original shape or condition of the artifacts.
X-ray CT segmentation modeling was performed after data capacity optimization for the smooth operation of the dedicated program; however, the segmentation 3D model had a lower quality shape than the original X-ray CT data. This is presumed to have resulted from adjusting the number of pixels in the Xray CT tomographic image during the optimization phase. Since historical research and restoration modeling using low-quality data can lead to errors in the results, this study further advanced accuracy via supplemented boolean modeling. However, this method is a temporary solution, and in future, optimization settings should be carefully selected to ensure minimal degradation of data quality when using X-ray CT data.

CONCLUSION
Herein, digital modeling and printing technologies were employed to restore a horse-shaped earthenware that was missing approximately more than half of its body. In particular, to take full advantage of the remaining original fragments, the bent leg in its original form was modified using rigging and keyframe animation, and the hind leg was restored referring to the front leg. X-ray CT data obtained from the vessels model chosen as a reference were utilized for modeling because the horse's body was difficult to study through the decoration. The air layer between the horse's body and the decoration was set as a threshold, and only the horse's body was segmented based on this condition. The segmented portion was enlarged approximately 2.8 times and utilized for restoration.
The position of the hip fragment was determined through historical research, and the symmetry and shape of the legs were restored using the original form. The digital restoration is approximately 60% of the result, with a chest to hip ratio of 1:1.3, similar to the reference model. The virtual restoration model was printed via 3D printing and displayed as part of the physical restoration in the exhibition. The multi-modeling technologies can create models of different shapes without altering the original model of the artifact. This helps to create a reference model for restoration and is thought to be a highly versatile technique that enables restoration that would otherwise be impossible in the physical environment.