THE APPLICATION OF SMARTPHONE BASED STRUCTURE FROM MOTION (SFM) PHOTOGRAMMETRY IN GROUND VOLUME MEASUREMENT

.


INTRODUCTION 1.1 Background of Study
The improvement and integration of photography and computer technology have led to a dynamic development in the field of geomatics, in both efficiency and precision.Initially, these developments focused on constructing digital elevation models (DEMs) or digital terrain models (DTMs) using both photogrammetric and differential global positioning system (DGPS) data (Micheletti et al., 2015).Data presented in 3D reconstructed model allowed clearer and more effective visualisation.
Precise measurement of volume is one of the concerns which contribute toward the development of 3D modelling, in order to map the DEM or DTM accurately.There are a lot of activities that required volumetric measurement for example cut and fill, soil erosion and deposition, deformation modelling etc.The developments of photogrammetry have become the fundamental and key knowledge in succussing the construction of elevation model.
Over the last decade, Terrestrial Laser Scanner (TLS) has proved its capability in generating very high-quality DEM data and has become one of the favourable approaches in collecting elevation model information, showing it capability to measure topographic surface up to millimetre or centimetre precision.However, such TLS solution does contain some drawbacks including cost, required high professionalism etc.
The technique of SFM photogrammetry has been widely used by the industry in various activities include monitoring of Geomorphological structure (Dabove et al., 2019), erosion and deposition (Nadal-Romero et al., 2015), Coastal Morphodynamics (Jaud et al., 2019).This shows that SFM photogrammetry has been widely used in various field and purposes.
Structure from Motion (SFM) photogrammetry served as a cheaper yet affordable solution for industry to acquire 3D model.SFM outperform itself as a trending technique to construct a 3D model with easily transport instruments such as smartphone, compact camera or tablet (Dabove et al., 2019) and cheaper while easier to used (Nadal-Romero et al., 2015).Some researchers also propose the uses of smartphone camera lens to capture images that used to construct the 3D model.
In conclusion, the application of smartphone based SFM photogrammetry in the uses of ground volume measurement become a topic of interest, to determine whether it could generate a satisfactory output that is comparable with TLS.

Research Aim and Objectives
The aim of this study is to assess the potential of smartphone camera sensor captured images in 3D model reconstruction using SFM photogrammetry techniques.Precisely, there are several objectives that should be fulfilled in order to achieve the research aim, include: (a) To generate a 3D model of stockpile by using smartphone based SFM photogrammetry.(b) To determine the accuracy of 3D model point cloud of smartphone based SFM photogrammetry for ground volume assessment compare with TLS solution.

Photogrammetry
According to; (McGlone, 2004), photogrammetry is the art, science, and technology of obtaining reliable information about physical objects of the environment through process of recording, measuring and interpreting photographic images and patterns of recorded radiant electromagnetic energy and other phenomena.The key idea of photogrammetry which brought outstanding performance when comparing across conventional measurement approaches is the ability to take measurement without physical contact with the objects.
The development of photogrammetry has an inseparability connection to the improvement of science and technology, specifically camera system and computer vision technology.Surprising, the introduction and paradigm shifts are directly related to the invention of photography, airplanes, computer and electronics (Schenk, 2005).The four phases of photogrammetry included plane table photogrammetry, analogue photogrammetry, analytical photogrammetry and digital photogrammetry.Digital photogrammetry is the era we are experiencing, which bring benefits to various profession.

Categories of Photogrammetry
Two general type of photogrammetry exists which are terrestrial photogrammetry (with the camera handheld or on a tripod) and aerial photogrammetry (with camera in the air) (Fryer, 2007).The selection of photogrammetry types should depend on the characteristics of job, expected output as well as budget of the project.
Terrestrial photogrammetry is also categorized into three different sub-category which are close range photogrammetry, macro-photogrammetry and micro-photogrammetry.This subcategory photogrammetry is differentiated based on their observed surfaces or objects.Close range photogrammetry which deals with object or surfaces up to 200m are known as close range photogrammetry, thus is the most frequently used for Earth surface measurement.Some examples of closed range photogrammetry platform included Digital Single Lens Reflex camera (DSLR), smartphone, compact camera etc.Also, terrestrial laser scanner is the most accurate non-contact method of measurement to derive information at plot level (Iglhaut et al., 2019).
Aerial photogrammetry is a photogrammetry solution where images are captured from the air using airborne or Unmanned Aerial Vehicles (UAVs), providing an orthophoto images after geometrically corrected.Aerial photogrammetry normally deals with a huge area of object or surface, normally such as mapping of building or topography.In contrast of that, it has a relatively lower spatial resolution when compare to terrestrial photogrammetry.

SFM Photogrammetry
SFM is a photogrammetric technique that is used to estimate the 3D models from a sequence of 2D overlapping images.For every point feature, it is necessary for it to be captured from at least three images from different perspectives of vision, known as overlapping as to allow the reconstruction of 3D point features by SFM techniques (Snavely et al., 2006).With implementation of SFM algorithm in software environment in line with techniques such as scale invariant feature transform (SIFT), the complex computation and processing workflow can be executed automatically with the aids of computer vision technology.Mankind served as the initiator and supervisor who ensuring the processing work to be conducted successfully.
Point cloud is a set of feature points which has been denotated in a 3-dimensional space where it can be used to represent a model or surface on the Earth.Every single feature point has their own Cartesian coordinate (X, Y, Z) on such a 3dimensional space, thus it can be used to conduct measurement after such model has been correctly scaled or georeferenced.The reconstructed 3-dimensional point cloud model is the product which is required for ground volume measurement.

Photogrammetry Platform and Sensors
In the application of photogrammetry, sensor is such an important element which should be considered during project planning.Generally, there are two specific cameras which are employed, namely metric camera and non-metric camera.
According to (Koelbl, 1976), metric camera can be used for precision measurements or for restitution in analog plotters without additional control of the elements of inner and relative orientation, thus it is very suitable for the application of photogrammetry work.Moreover, external factors such as vibration, temperature or pressure will not affect the internal mechanism of metric camera, preventing errors in camera parameter.Generally, camera calibration workflow is not required when metric camera is used.
A non-metric camera is a relatively affordable camera which is widely used by most amateur and professional photographer in capturing high resolution images.Some of the examples of nonmetric camera including smartphone camera, compact camera or DSLR.The outstanding benefits of non-metric camera other than inexpensive prices are simplicity and greater efficiency in capturing images.However, non-metric camera experienced greater distortion when compare to metric camera.Since nonmetric camera has unstable interior orientation, thus it is necessary for the execution of camera calibration before every project is launched, in order to obtain the camera parameter for further processing work.The acquisition of camera parameters through camera calibration allowed non-metric camera captured images can be used for photogrammetric measurement, allowing the application of non-metric camera in the field of photogrammetry surveying.

SFM Photogrammetry Applications
Photogrammetry is primarily concerned with making precise measurements of three-dimensional objects and terrain features from two dimensional photographs (Aber et al., 2010).Photogrammetry has outperformed itself with certain characteristics where conventional technique could not achieve.Other than greater working efficiency and accuracy, the properties of remote measurement of the dimension of object allowed surveyor or other professions to carry out measurement without having physical contact with the objects.
In engineering industry, photogrammetry has been widely used in mapping the geospatial data of Earth including forestry, topography, landscapes, coastal and others (Dabove et al., 2019;Danielle Cullen et al., 2018;Iglhaut et al., 2019;Jaud et al., 2019).The application of photogrammetry has performed a high efficiency and accuracy data collection approach when compare to conventional method.Other than metric camera captured images which expensive, non-metric camera captured images could also be used to deliver a high-quality product that represent the surface of the Earth.
According to (Eltner et al., 2013) , SFM photogrammetry is scale dependent, at plot and hillslope scale, 3D reconstruction is a very efficient method of soil surface and ground volume studies, even outperforming TLS in some circumstances due to the drawback of TLS in cost, mobility and professionalism requirement.SFM method is capable and very suitable for generating interpolated DEM up to millimeter resolution, which could meet the requirement by most research and industry in measuring soil surface and ground volume variation, for the application of cut and fill, deformation survey etc.
According to (Dabove et al., 2019), a solution for the 3D mapping of natural caves can be found in terrestrial digital photogrammetry, using mass-market high dilution and low-cost technologies, such as smartphones or amateur commercial cameras for image acquisition.In line with that, SFM photogrammetry techniques is adapted in reconstructing the model from the images captured.The ability of active optical sensor which could captured color information other than features point position allowed the interpretation and analysis of geomorphological structure to be more diversify.
Monitoring of coastal morphdynamics is an important issue especially toward population living in coastal area.Previous method adapts Real Time Kinematic (RTK) Global Navigation Satellite System (GNSS) for beach surveying, however the spatial resolution is underestimate although it provides a high accuracy survey result.In (Harley et al., 2019) paper, smartphone camera sensor with SFM techniques has been adapted for the modelling of coastal, for the application of beach monitoring initiative.The images captured relating to the coastal area such as beach, cliff or shoreline can be used to reconstruct a 3D model that allowed further interpretation and analysis.

METHODOLOGY
The flowchart shown in Figure 3.1 explains the research methodology.Generally, research workflow can be structured in five phases.

Research Preparation
The phase one of research methodology involve various research preparation.Literature review is conducted as to build deep understanding on research topic and to find suitable research gap.It is founded that the application of smartphone based SFM in ground volume measurement is a good research topic to be determined.The method to conduct entire research study is also determined as well as the scope of study such as research objects, instruments and software.

Research Area Selection
The research study is focusing on the volumetric measurement of stockpile using smartphone based SFM photogrammetry technique.Two stockpiles located inside UTM campus has been selected as research objects.Stockpile A located beside Cengal roadway, having a size of approximate 2.5m height, 2.5m width and 20m long while stockpile B located behind the construction site near UTM gate 3, having a size of approximate 2.5m long, 2.5m width and 1.5m height.The figure 3.2 below shows the stockpile located at site A and site B. Both sites are open space and reachable, thus no permit is required.

Figure 3.2.
Stockpile A (left) and Stockpile B (right)

Equipment and Software
The research study conducted required various professional survey equipment and photogrammetry software.A smartphone camera, Honor View 20 which mounted with a non-metric camera is used to captured image for smartphone based SFM photogrammetry solution.Leica RTC360, a geodetic grade Terrestrial Laser Scanner (TLS) is used to collect point cloud model, served as the benchmark for the comparison with smartphone SFM approach.Other than that, some artificial target such as 1.5m X 1.5m photogrammetry carpet and painted nails is also used.Table 3.1 below tabulated the detail information for Honor View 20 and Leica RTC360.

Data Acquisition
In the phase of data acquisition, fieldwork is conducted to collect data of both stockpiles located at site A and site B with smartphone camera and TLS.It is necessary to establish artificial target as control points because control points are crucial and necessary for any photogrammetry projects where accurate scaling is required (Mosbrucker et al., 2017).Moreover, well distributed artificial target as control points should be adhere to ensure the generated point cloud is properly scaled.There are total of six artificial targets planted on top of stockpile located at site A while four planted on top of stockpile located at site B.
A camera calibration is a process to determine the camera parameter, specifically for each kind of non-metric camera.The execution of camera calibration is a must in every photogrammetric project as to obtain the parameters of camera.Such parameters are important as it could provide essential information toward the software algorithm for constructing a 3D model, provide accurate result when measuring object length, height or volume.The calibration of smartphone camera sensor is conducted by capturing images of the chessboard from different perspectives.Next, importing the images captured into software environment to calculate and acquired the 13 camera parameter which would be used for the correction during data processing.The calibrated camera parameters are shown in appendix.
The data collection for preliminary input of SFM photogrammetry is done by using Honor View 20, a smartphone camera with 48MP Sony IMX586 camera sensor.The images are captured by using default mode where ISO, Shutter Speed and Exposure are set to auto.It should be ensured that every single feature point should be visible from at least three images, the more is better.Images are capture from the object with a distance of 10m away to ensure the resolution of each feature points are clear, and with a radial motion of 10-15°.The artificial targets are included in as much images as possible to increase the integrity of constructed 3D model.Data collection with smartphone camera is conducted at mid-noon where illumination is greatest.The total number of images captured for site A is 135 images while site B is 67 images.

Figure 3.3. Smartphone Data Acquisition
Data collected by using RTC360 are used as a benchmark when compare to 3D model constructed by using Smartphone based SFM photogrammetry.The type of scanning proposed for this research is by scanning and images.This type of scanning acquired not only the position however also the spectral information.Within three scanning mode provided by RTC360, low resolution mode is adopted which spent only 1 minutes and 26 seconds for every station.However, RTC360 manage to capture enough point cloud within this period, providing optimum point cloud density while suitable for applicable for volumetric measurement comparison.The TLS instrument is set up and collect data at different location to ensure that entire object is observed and the data collection is perfect.The TLS data collection of stockpiles located at site A take five station to completed data collection while stockpile located at site B only take four due to the consideration of stockpile shape and size.

Data Processing
The phase of data processing included the processing of collected data from smartphone camera and TLS instrument.
There will be three software that involved in data processing including Agisoft Metashape Professional for Smartphone SFM data processing, Leica Cyclone for RTC360 TLS data processing and CloudCompare for point cloud model comparison.
The processing of smartphone based SFM photogrammetry required two processes, which are camera calibration and point cloud generation.The execution of camera calibration is to obtain the parameters of camera which provide essential information toward algorithm for point cloud model construction with correct scaling.The camera calibration is conducted with Agisoft Metashape Professional.A set of images of chessboard from the software itself would be captured from different perspectives with the uses of Honor View 20 camera.These images are then imported into the software environment for block corner analysing, in term of position.
The generation of fusion point cloud should be done after the camera calibration parameters and the captured images has been imported into the software environment.The establishment of Manual Tie Points (MTPs) and referencing inside the software environment are crucial and necessary to ensure the point cloud model generated consist of accurate scaling which based on the space resection compliance with the camera calibration parameter which were imported on the earlier stage.The entire fusion point cloud generation workflow is considered to be automation where user should only initiate and supervise the process.The generated point clouds for both stockpiles can be exported in .lasformat.
The preliminary data captured by TLS is not clustering as TLS instrument is moving from station to station.The captured TLS data has been imported into Leica Cyclone 360 working environment.The data registration workflow is then executed to cluster the different group of point cloud from different station into a single clustered point cloud.Next, the clustering dense point cloud of the target object has been exported in the form of .lasuniversal format for further data comparison and analysis.

Data Analysis
The point cloud model generated from smartphone SFM and TLS solution is imported into CloudCompare software for volume measurement and dimensional validation.The volume measurement is executed on the segmented point cloud model, and the result is tabulated for comparison between different approaches.Next, total of 15 dimensions is measured on various man-made features such as electric box, walkway, drainage cover etc. and a comparison between the distance measured is made.Root Mean Square Error is used estimate the distribution of error.
(1) Where zfi = Smartphone SFM point cloud measured dimension z0i = TLS point cloud measured dimension N = Sample size

RESULT AND ANALYSIS
This research study has several expected outputs including the comparison of ground volume measurement and dimensional validation.These numerical outputs allowed the determination of accuracy for smartphone based SFM photogrammetry when compare to TLS solution.Moreover, the formation of point cloud and its visualization is also analyzed.

Ground Volume Measurement
Assessment on ground volume measurement is the vital objectives of this research study.The volume of stockpile measured from the generated point cloud is tabulated in table 4.1 as shown below.The accuracy is computed from the differences in volume measured while TLS is served as the benchmark throughout the comparison.Man-made features selected for dimensional validation for site A (top) and site B (bottom) From the table, there are significant differences between the distance measured.The differences increase as the distance increase because the entire smartphone SFM point cloud model is proven to be larger than TLS generated point cloud model, referring above.The RMSE computed indicates that the standard deviation of the differences between Smartphone SFM and TLS generated point cloud is 0.310 m.

Point Cloud Formation and Visualisation
The formation of point cloud model generated by smartphone SFM and TLS method can also be compared from visualization approach.The density of point cloud model generated by Smartphone SFM are much lower when compare to TLS method, although TLS are set to lowest resolution.The lower point cloud density will affect the accuracy of model generated as measurement process conducted might not be corner to corner.Generally, the local neighborhood radius for TLS generated point cloud is 0.015m while smartphone SFM generated point cloud would up to 0.030m.The greater the radius, the lower the point cloud density.
The data collection range is also one of the differences between smartphone SFM and TLS generated point cloud model.TLS solution could provide a greater data collection range, up to 30 meters or greater while smartphone SFM could only focus on the research object, even if time spent for TLS is shorter.In short, we agreed that TLS has a better working efficiency in large working area or survey object when compare to smartphone SFM.

Summary
The accuracy of generated point clouds is analyzed by the error in ground volume measurement where TLS method generated point cloud is served as the benchmark of this research study.The ground volume measurement for smartphone SFM method could only reach up to 68.24%.From the dimensional validation where 15 samples are taken in consideration, the RMSE computed is 0.310m which mean the average error between smartphone SFM and TLS generated point cloud has such dimensional errors.
The vital source of error which might lead to such faulty in point cloud model reconstruction is photogrammetric errors.The photogrammetric errors served as the most critical errors in this research study come from camera internal parameters, image capturing techniques as well as data processing strategies.Although camera calibration is executed and the parameter obtained is applied, however the estimated camera parameters might still remain some inconsistency which could lead to scalar failure in model reconstruction.Besides, the image capturing techniques should be further enhanced so that every perspective of the model could be captured, including the top of the model.The data processing techniques could be further improved by applying control point which has been surveyed as manual tie point.
From visualization aspect, TLS method are proven to have higher point cloud density compared to smartphone SFM method, although the time spent for data collection is much lower.Moreover, the data collection range for TLS method is also much further compare to smartphone SFM which only focus on the research object.Such visualization aspects might also bring limitation toward the analysis process or used for other purposes.

Conclusion
With all analysis made in previous chapter, the accuracy of 3D model point cloud of smartphone based SFM photogrammetry for ground volume assessment is only 68.24% when compared to benchmark, TLS method.This result is further proved with dimensional validation and RMSE analysis.The result of RMSE obtained from this research study is 0.310m.Almost every dimension measured from smartphone based SFM generated point cloud model are 30% greater than TLS method generated point cloud, which correspond to ground volume measurement output.
We can conclude that smartphone based SFM photogrammetry is not suitable to be bring into real industry for ground volume measurement.The error up to 31.76% is considered too critical to fail any industry projects.Honor View 20 as a mid-range smartphone, its non-metric consumer grade camera sensor is not suitable for geodetic purpose practices.

Recommendations
There are lot of challenges and issues faced throughout the entire research work.Some recommendations could be give to future researcher who interest in smartphone SFM related topic in order to better success the research.a.It is recommended to apply absolute orientation where control points with GNSS or traverse method observed coordinate is ready.It allowed the model to be better scaled.b.The survey object selected for smartphone SFM should not be too huge as larger object required image capturing from further away, however this will affect the resolution.Moreover, there will be some blind spot where smartphone could not capture.In fact, TLS solution or UAV survey technique is a better alternative compare to smartphone SFM.

Figure 4. 1 .
Figure 4.1.Point cloud generated for site A (top) and site B (bottom)

Figure
Figure 4.2.Man-made features selected for dimensional validation for site A (top) and site B (bottom)

Table 3 .1.
Other than equipment, there are 3 professional software which are used in this research project.Agisoft Metashape Professional is used to generate 3D point cloud model from the image captured by using smartphone camera.
Leica Cyclone 360 is used to process the data collected from Leica RTC360, clustering as a complete point cloud model.CloudCompare software served as the software which provide environment for comparison between point cloud generated by using Smartphone SFM and TLS solution.

Table 4 .1. Ground Volume Measurement Site Name Volume from Smartphone SFM Solution (m 3 ) Volume from TLS solution (m 3 ) Differences (m 3 )
From table 4.1, we could observe that there is a significant difference between the ground volume measured from two solution delivered.The difference increases when the size of stockpile increases, in fact the entire point cloud model delivered by smartphone SFM approach is bigger than TLS solution.Based on the ground volume measurement conducted on both site A and site B, we can conclude that the smartphone SFM solution could only estimate the ground volume at accuracy up to 68.24%.The figure4.1 below illustrate the point cloud model generated.