VISION SYSTEM FOR MONITORING SEISMIC TESTS IN CONCRETE SHELLS

: Structural monitoring plays a crucial role in assessing damage and detecting potential faults. Vision systems provide real-time information about the behaviour and state of conservation of structures, enabling precise and efficient analysis. Computer vision algorithms are employed to analyse images and extract relevant structural information. This information may encompass key points' displacement and deformation within the structure. This paper introduces a methodology for monitoring dynamic tests using computer vision. The approach employs a camera to track artificial targets placed on the structure. The camera captures multiple images during the tests and detects the coordinates of the target centres. Subsequently, displacements are calculated based on the differences in coordinates among the aforementioned targets. To validate the methodology, an experimental study was conducted using dynamic tests on a reduced-scale model of a thin concrete shell tested on a seismic table. Five dynamic tests were carried out, involving varied amplitude and frequency values to test the proposed approach. The results substantiate the efficacy of computer vision in monitoring dynamic tests. In conclusion, the developed methodology facilitates dynamic test monitoring through computer vision by continuously tracking artificial targets throughout the tests and computing their displacements.


INTRODUCTION
Structural monitoring employing vision systems has emerged as an increasingly pertinent topic in the field of Civil Engineering (Oliveira-Santos, 2022).Monitoring assumes a crucial role in ensuring the safety and maintenance quality of structures.With the advancements in vision systems, they have evolved into a more prevalent tool within the construction sector (Valença, 2017).The domain of concrete prefabrication, including 3D concrete printing, aligns seamlessly with the broader principles of Construction 4.0.This approach capitalizes on optimizing structural behaviour through innovative shaping.Within this context, structural analysis assumes paramount importance and stands to gain from computer vision methodologies.
The integration of a computer vision system into dynamic testing presents many advantages over traditional instrumentation.Notable benefits include simplified installation procedures and the capacity to comprehensively and accurately monitor the dynamic response of structures in real time.Achieving this objective require a sequence of steps encompassing camera calibration, the identification of points of interest within images (targets), the reconstruction of threedimensional movements of the structure, and the subsequent analysis of acquired data (Pavese, 2012).In scenarios involving seismic response assessment, the current norm involves the use of reduced-scale models for testing purposes.
A vision system tailored for geometric monitoring during seismic tests on shaking tables has been invented.This system employs a 3D camera to capture images, from which a 3D geometric model is generated.The initial calibration of the system was executed using a resembling 'Mickey Mouse' reduced-scale model.Subsequently, a series of laboratory tests utilizing a 1:3 scale concrete shell are executed, wherein the methodology is being employed for validation purposes.

METHODOLOGY
The methodology developed involved the following sequence of main steps: 1. Placement circular red targets on the shell, hereafter referred to as 'Moving Targets.'These are the targets to be monitored for displacement; 2. Placement of red rectangular targets outside the shell, named 'Control Targets', which is used to measure the displacements applied to the seismic table, and also used for blunders control; 3. Start video recording before starting the seismic table , and check if the centers of all targets are detected; 4. Detect the coordinates of the target centroids in each frame; 5. Compute the differences in targets coordinates between consecutive frames to achieve target's displacements.6. Compute the vibration of the moving targets by measuring the differences between Moving and Control Targets.
To evaluate the quality of the results, the Root Mean Square Error (RMSE) was calculated in all targets (Moving and Control) between the observed values and those that were expected.This calculation was performed in the initial frames, before actions or movement to the seismic table, i.e., with all targets static.Therefore, the variation between the coordinates of consecutive frames is expected to be zero (Valença, 2013).It is important to note that the value of the RMSE obtained depends on many factors, such as the camera used, the light condition, the nature of the applied seismic motion, the quality of the target painting, among others.Finally, depending on the algorithms used, the methodology enables to monitoring 2D and 3D target coordinates during the tests.

Set-up
The procedure was implemented and subsequently applied to the reduced-scale model for calibration purposes (Figure 1 and Figure 2).The ultra-thin concrete shell model was fabricated using 3D printing technology and Acrylonitrile Butadiene Styrene (ABS) material.The model had a maximum height of 15 cm, measured from the base to the highest point, and a span of 46 cm between supports.Geometric quality assurance for the produced 3D model was accomplished through photogrammetry, as outlined by Tomé (2018).This technology's application facilitates the modelling and 3D printing of intricate shapes, leading to a reduction in both time and cost associated with producing physical reduced-scale models of structures.To monitoring of tests displacements during the testes, 19 circular red targets were placed in the shell surface, the moving targets, and 12 rectangular 'control targets' in the table surface.
These models find common application in physical testing and structural analyses to evaluate the strength and behavior of structures.The extent of scale reduction can vary based on the model's intended purpose and the dimensions of the original structure.In the present context, the produced model represents a concrete shell.The shell model was positioned atop a seismic table, and the configuration of the setup comprised: 1. Intel RealSense D435 Depth Camera.The depth camera is a device that uses stereoscopic technology to acquire three-dimensional images and depth information.The D435, a cost-effective camera, has the capability to generate a real-time depth map, enabling the three-dimensional perception of the surrounding environment (Intel RealSense, 2023); 2. Quanser Shake Table II Educational Seismic Table.This seismic table was chosen to perform tests the reduced-scaled model of the shell.It was specifically designed to simulate controlled seismic movements, enabling precise application of forces and vibrations.This equipment allows for the reproduction of real seismic movements and the study of the dynamic behavior and seismic response of structural models with a maximum weight of 15 kg; 3. Control Computers.Two computers are used, one to control the seismic table and another to store the collected images.The camera information will be stored on an Asus VivoBook computer, which is equipped with an 8th generation Intel Core i7 processor and an NVIDIA GeForce MX150 graphics card; 4. Wooden base with perforation and connection supports.
Due to its solid, compact, and extremely lightweight structure, it was decided to release one of the supports of the shell to allow for greater vibration of the structure.It is important to emphasize that the purpose of these tests is the calibration of the developed methodology, and not to simulate the behavior of the shell when subjected to seismic actions.Additionally, in two of the tests (4 th and 5 th Test), an additional weight of 200 g was added and attached to the shell to increase the mass of the structure.

Procedure
The methodology uses the RealSense camera's Software Development Kit (SDK) library to capture colour and depth frames, thereby creating a point cloud based on this information.The code is implamented in (Mathworks, 2021) and aims to capture and process frames from a RealSense device.The SDK library provides methods that enable the synchronous or asynchronous retrieval of colour and depth frames, depending on the user's needs.The video recording process was conducted and distinct approaches were applied for monitoring 2D and 3D information.
For 2D monitoring, pre-recorded videos are processed, which is analyzed to compute the coordinates of the centroids of red targets.The X and Y coordinates should be regarded as belonging to the same plane, without taking the Z coordinate (vertical) into consideration.Hence, for targets that are not on the same plane must be corrected.The method implementation in MATLAB (Mathworks, 2021) performs detection and tracking of red targets, as well as segmentation based on the shape difference between Moving Targets (circular shape) and Control Targets (rectangular shape).This segmentation is achieved with the circularity function, which applies a condition to the obtained results to separate the two types of targets.In this property, the relationship between the area and perimeter of the selected red object is established (Valença, 2013).After obtaining the coordinates of the targets' centroids, in pixels, it is necessary to convert them into millimeters.For this purpose, a homography [H] is computed for the wooden table, using the knowledge of the distances between four pre-defined reference points in both coordinate systems as a basis.A relation for plans parallel the base plan ([H]) was stablished to compute [H'] matrix for each target level (Aliakbarpour, 2016).The displacements of the targets begin with the selection of two control targets.One of the targets is located on the seismic table that undergoes the displacement, while the other target is located outside the seismic table.The difference between the coordinates of these two targets establishes a reference point for calculating the displacement in each target of the structure.The selection of the control targets is carried out based on: (i) combining the data that best represents the expected displacement of each target; (ii) the target that appears in the largest number of frames; (iii) the target with the lowest RMSE value.To evaluate the quality of the results obtained, the RMSE was calculated in all targets (moving and control) between the observed values and those that were expected, in a set of frames before applying actions or movement to the seismic table, i.e., with all the targets stationary.Therefore, the variation between the coordinates of consecutive frames is expected to be zero.It is important to note that the exact value of the RMSE obtained may depend on many factors, such as the precision of the measurement equipment, the nature of the applied seismic motion, the quality of the target placed on the structure, among others.
For 3D scanning, a proper synchronization between colour and depth frames was ensured by synchronous capture A real-time acquisition was carried out in order to enable the complete storage of point cloud vertices coordinates information and their respective RGB coordinates during the scanning process.Once again, the objective is to extract the coordinates of the centroids of the red targets.A 3D model of the object is also generated, allowing for the creation of realistic and accurate representations of the structures.These representations assist engineers in visualizing and planning projects, identifying potential issues, and communicating their ideas more clearly.
For the calculation of the final displacements, the tracking of each target was performed.Post-processing operations was applied in the point cloud of each frame, to obtain the coordinates (XYZ; RGB) from the targets.In this case, the RMSE was also calculated for three targets with different characteristics: (i) a moving target on the shell; (ii) a control target on the seismic table; and (iii) finally, a control target outside the seismic table.In these frames, the seismic table remains stationary, so no variation in coordinates between consecutive frames is expected.
Regarding the amplitude, it has consistently proven to be lower than the amplitude imposed on the seismic table in all tests.Since this seismic table is used for educational purposes, there is no guarantee that the value set in the software precisely matches the value achieved on the table.Given that the proposed method involves placing control targets on the seismic table, it is ideal to always measure the displacement on this table and compare it with the displacement on the structure, ensuring their equivalence.

Geometric survey
In an initial approach, the geometry of the 3D model was measured during testing (Figure 3).A shell height of 15 cm and a distance between supports of 46.3 cm were obtained, values in line with measurements previously performed with other methods (Tomé, 2018).

2D monitoring
To perform the 2D monitoring, the automatic identification of the respective moving and control targets can be observed in Figure 4.The results obtained from the tests were carefully analyzing and shows that the X coordinates of the moving targets and control targets, which recorded the motion of the seismic table, exhibit a sinusoidal pattern of evolution.This characteristic is typical of an oscillatory behavior that repeats periodically over time, precisely the expected type of motion of the seismic table.This is clearly observed in the graphs of Figure 5, for all moving targets evaluated.
In the control targets, due to their rectangular configuration with one side significantly larger than the other, the most significant error occurs in the coordinate corresponding to the target's largest dimension.The maximum error observed in all the tests was 0.6 mm.This fact suggests that the methodology has sufficient accuracy for target detection in a structure.Despite this maximum error, the selected control targets used as reference point exhibit much smaller error, 0.1 mm.In moving targets, RMSE values indicating a higher error in the X coordinates compared to the Y coordinate.While there is some variability in the error measurements, the maximum value obtained was 0.16 mm.It is worth noting that the error manifests more significantly in the X coordinate, which aligns with the seismic table's movement along the XX axis.
The amplitude and frequency analysis of all moving targets revels the shell appears to possess a natural frequency identical to the frequency of the applied motion on the seismic table.This indicates that the structure responded efficiently the vibrations, with the observed amplitude on the seismic table being equal to the amplitude observed on the shell targets.This observation suggests that the structure possesses a remarkably high inherent frequency due to its low mass, causing it to respond solely to the excitation frequency.When observing the Y coordinates of the mobile targets, it becomes evident that, as expected, there has been no movement along the YY axis.Therefore, there will be no need to calculate displacements for this axis, as no positional changes have been recorded.
The comparison of displacements among the moving targets will be performed based on the criterion of selecting targets that, theoretically, hold the most extreme conditions to exhibit different displacement values.As example, Figure 6 shows the evolution of the targets coordinates along the seismic test for the target located in the centre of the shell.The blue line represents the 'Moving Target' while the red line represents the corresponding displacement that this target has submitted.The Figure 7 provides a detailed representation of the displacement of the target.It can be observed that it exhibits a maximum displacement of 3.9 mm.   Figure 8 illustrates the displacement evolution of the three respective targets.In the first test, upon analyzing the three targets, it was observed that the maximum displacement occurred in frame 253 of Target 19, with a displacement value of 2.85 mm.However, upon examining all the frames, it was found that Target 9 experienced the highest displacements in a greater number of frames.This suggests that, in this particular case, the central region of the shell structure is the most vulnerable to displacement variations.As already mention, the maximum error limit established for these tests was 0.6 mm.However, this value has never been reached, as the control targets selected for reference points have RMSE values lower than this maximum.This result suggests that the system under analysis possesses adequate accuracy for detecting targets within the structure in question.Table 1 presents the displacement range measured in the tests.The amplitude has consistently proven to be lower than the amplitude imposed on the seismic table in all tests.Since this seismic table is used for educational purposes, there is no guarantee that the value set in the software precisely matches the value achieved on the table.Given that the proposed method involves placing control targets on the seismic table, it is ideal to always measure the displacement on this table and compare it with the displacement on the structure, ensuring their equivalence.Table 1.Summary of displacement intervals measured.

3D monitoring
The targets used were the same to those employed in the "D monitoring.The identification of the respective targets can be observed in Figure 9.The analysis herein presented corresponding to 4 th test, with an amplitude of 20 mm and a frequency of 4 Hz imposed on the seismic table.Ten frames were used for the analysis, namely frames 20, 30, 40, 50, 79, 80, 81, 82, 83, and 84.Additionally, for this same test, the RMSE of the X, Y, and Z coordinates was calculated for three targets with different characteristics: (i) a moving target located on the structure; (ii) a control target located on the seismic table; and finally (iii) a control target outside the seismic table.To perform this calculation, six consecutive frames were selected, ranging from the 10 th to the 15 th frame.It is expected that in these frames, before apply actions in the seismic table, there will be no significant variations in the coordinates between consecutive frames.Table 3 mentions the maximum amplitude values obtained, which indicate that the amplitude in the X coordinate was, in the majority of targets, higher than the imposed amplitude.
Between frame 30 and 84, the seismic table is in motion, which is reflected in the graph of the coordinates in this area, thus representing the existing movement.The frequency was calculated based on the curve of the X-axis coordinate, which is equivalent to the imposed frequency.It is worth noting that, in the graphs, the horizontal axis is not depicted to scale and is used solely to demonstrate the displacement of coordinates.

CONCLUSIONS
The results achieved in the reduce-scale model printed in ABS material reached promising results, and enable to evaluate the performance of the system, namely the ability to evaluate: (i) the geometry produced, in terms of precision and accuracy of the values measured; and (ii) the seismic response of the structure to seismic tests.The application of the proposed methodology for tracking the 3D coordinates of the artificial targets, enables to compute displacements of the targets' during the dynamic tests.It is important to note that these tests require greater computational capacity in order to perform postprocessing of the point clouds for 3D monitoring.
The methodology proposed was improved and applied during seismic testing of a concrete shell, for validation purposes.The tests were performed within the scope of Pre-Shell funded research project (Pre-Shell, 2023), whose aim is to develop shells composed of thin shells formed by precast modules in UHPFRC (Ultra High Performance Fiber Reinforced Concrete) and connected by non-adherent pre-stressing (Figure 11 and Figure 12).The results of the concrete model will be analysed shortly, and will result in system improvements.

Figure 1 .
Figure 1.Set-up for testing of the reduced scale model.

Figure 4 .
Figure 4. Numerical identification of the targets in the 2D tests.

Figure 5 .
Figure 5. Coordinates in XX axis of the moving targets (2 nd test).

Figure 6 .
Figure 6.Example of moving target: total displacements (blue);displacement relative to the seismic table (red).

Figure 7 .
Figure 7. Displacement of the central target (9) in one of the tests.

Figure 9 .
Figure 9. Identification of the targets.

Figure 10 .
Figure 10.Example of targets overlap in two stages of testing.

Table 2
presents the RMSE values of the 3D coordinates in 6 consecutive frames.Targets 9 and 14 exhibit error values that can be considered acceptable.However, Target 29 displays a significantly elevated error.Upon analysing the 3D coordinates in the frames under study, variations in their positions are evident.This suggests that the detection of this target may have encountered difficulties in generating the point cloud that represents. it Figure10shows an overlap between the targets during two stages of the test.

Table 2 .
RMSE of the coordinates for 3 control targets.

Table 3 .
Amplitude and Frequency values obtained.The analysis of Table4shows that the displacements in the Y coordinates exhibit lower values, consistent with the seismic table's motion being perpendicular to the Y-axis.Conversely, the displacements in the X and Z coordinates appear to exhibit comparable magnitudes.In certain instances, the displacement is more pronounced in the X coordinate, while in other situations, the opposite occurs.

Table 4 .
Maximum displacements of the analyzed frames.