CAMERA CALIBRATION IN 3 D MODELLING FOR UAV APPLICATION

In recent times, small types of Unmanned Aerial Vehicles (UAVs) have been receiving attention in areas such as 3D modelling, maintenance engineering, and personal interest (hobby) usage. However, the payload a small type of UAV is able to carry is limited. Given these circumstances, small consumer grade digital cameras are often used for UAV photogrammetry with small types of UAV. Though, digital photogrammetry using the consumer grade cameras is enormously expected in various application fields such as UAV photogrammetry. There is a large body of literature on camera calibration. However, the lens distortion of small consumer grade digital cameras is still an issue from the viewpoint of accuracy aspect. The authors have been investigating camera calibration using various cameras. However, small consumer grade digital cameras, which is called as entry-cameras, have an accuracy degradation problem. The issue was addressed by conducting calibration tests using two kinds of consumer grade digital cameras, each with a resolution of 12 mega pixels. This paper presents the results of an investigation into the cause of accuracy degradation in digital photogrammetry using entry-cameras.


INTRODUCTION
In recent times, small types of UAVs have been receiving more attention, because they are increasingly in areas such as 3D measurements, maintenance engineering, and for hobby use and so on, because of their simplicity of operation and lightness of body.However, the payload a small type of UAV is able to transport, is limited.Therefore, small consumer grade digital cameras are often used for UAV photogrammetry.On the other hand, consumer grade digital cameras have been rapidly gaining interest, with many low-priced consumer grade models available on the market.Therefore, performance evaluations for consumer grade digital cameras have been investigated from the viewpoint of digital close range photogrammetry (Fraser, 1997, Habib andMorgan, 2003).In digital close range photogrammetry using consumer grade digital cameras, lens distortion is one of important parameters.The authors have been investigating lens distortion using various cameras (Chikatsu and Takahashi, 2009).From these results, digital close range photogrammetry using consumer grade digital camera is expected for various application fields.However, not all of these cameras are necessarily suitable for use in 3D modeling.In particular, small consumer grade digital cameras which are called as entry-cameras have an accuracy degradation problem.In other words, in some of these cameras the accuracy is not improved after camera calibration.In order to resolve the issue, at first, the factors for accuracy degradation have been investigating using a camera of which the accuracy did not improve after camera calibration.Finally, the authors' results led them to propose a calibration method for improving the accuracy of the abovementioned camera in this study.

Cameras
In order to confirm the camera which does not show the accuracy improvement after calibration, the two consumer grade digital cameras shown below was investigated in this study (Table 1, Figure 1).details of the image sensor had not been published.Furthermore, the technical details of the lens used in each camera were also unknown.

Test Target
Figure 2 shows a test target that was used in the study.The size of the test target is H: 640 mm, W: 480 mm and D: 20 mm.The black circular targets in the test target were manufactured with an accuracy of ±0.05 mm.The diameter of each circular target was 20 mm, and the targets were arranged in intervals of 40 mm.The height of the central targets (three rows in the center) was 20 mm.GCPs (ground control points) and checkpoints were arranged as shown in Figure 2. The pixel coordinates of these points were obtained as the area gravity by using image processing procedures.

Lens distortion model
In this study, the lens distortion was evaluated by using Brown's model (Brown, 1971), as follows.
where, 2 2 y x r   , x ̅, y ̅ are the corrected image coordinates; x, y are the measured image coordinates; K 1 , K 2 , and K 3 are coefficients of radial distortion; P 1 and P 2 are coefficients of tangential distortion.

EXPERIMENTAL RESULTS
In this study, five stereo images were taken as a base-height ratio of 0.34, and altitudes of 667 mm and 736 mm for the S3000 and IXY 200F, respectively.The focal length of both lenses was set to the largest possible wide angle setting.Camera calibrations were performed using bundle adjustment based on 13 GCPs (Figure 2). Figure 3 shows the average of five calculations as the experimental results.In Figure 3, the proportional accuracy of the results was computed using equation ( 2).In addition, Figure 3 also shows the curve of the standard error, which was computed using equations ( 2) and (3) from the results of the camera calibration for the various cameras.
where, σ X0 , σ Y0 , and σ Z0 are the standard errors; H is the altitude; f is the focal length; B is the baseline; and σ P is the pointing accuracy (estimated to be 0.1 pixels (Schaefer and Murai, 1988)).
The calculated results obtained for the IXY 200F closely approximated the curve of the standard error in Figure 3. On the other hand, those obtained for the S3000 were positioned further from the curve of the standard error.Figure 4 shows the results of the camera calibration for cases in which lens distortion was not considered and considered.Note: Figure 4 shows that the proportional accuracy was normalized by the standard error.From Figure 4, it is clearly understood that the accuracy of the IXY 200F in terms of lens distortion was improved after camera calibration.However, the accuracy of the S3000 was not improved, even after camera calibration.Based on these results, the cause of accuracy degradation in the S3000 was investigated and the results are presented in the next section.

Distortion
In previous experiments, the S3000 was calibrated by using equations (1).However, Brown's model is able to divide into six models such as shown in Table 2 (Wakutsu and Chikatsu, 2011).Therefore, suitable distortion model for the S3000 is investigated in this session.
Figure 5 shows the results of the camera calibration using each of the six distortion models and the results show the proportional accuracy normalized by the standard error.Figure 5 indicates that the 3rd polynomial function with tangential distortions model (3RD) shows the best accuracy.However, the accuracy show quite different low accuracy from the viewpoints of digital photogrammetric aspect.The cause of the degradation in accuracy was investigated using the 3RD model.

Chromatic aberration
As one of the cause of accuracy degradation, chromatic aberration was considered.Chromatic aberrations are well known to occur in relation to the wavelength of light.The green channel in an image has been attention for correction of the chromatic aberration, and Guan, et al. were corrected chromatic aberration based on green channel (Guan, H., 2005).Consequently, the green channel images that were extracted from the stereo image were used for the investigation in this study.Figure 6 shows the results of the camera calibration for which the proportional accuracy was normalized by the standard error using the normal images and the green channel image.The distortion model that was used was the 3RD model.It can be seen that the result for which the green channel images were used show almost the same accuracy as the result obtained using a normal image.It is concluded that chromatic aberration does not show a significant factor.

Verification Result
In this study, lens distortion models and chromatic aberration were investigated.However, neither of these approaches led to an improvement in the accuracy.At this stage, the reason as to why it has not been possible to improve the accuracy is not known, neither is it known whether the aberration of the lens is responsible.On the other hand, there are many consumer grade digital cameras with an internal image processing function.It is unclear as to whether image distortion is automatically corrected by the internal image processing function.Thus, it is necessary to treat the situation as a black box by inferring whether the image data are automatically adjusted.If automatic image adjustment were used, it would be difficult to obtain an original image which had not been adjusted.Therefore, a calibration method for the camera which does not improve the accuracy even after calibration was proposed in this study.

Verification for distortion
In this study, it was taken center image to confirm the shape of the distortion for the S3000.Figure 7 shows the result of the camera calibration based on the center of the image using the 3RD model.Generally, it is known that lens distortion is symmetrical and many calibration models were developed based on symmetrical images.However, as seen in Figure 7, it is clearly understood that lens distortion for the S3000 is not symmetrical.Generally, it is well known that wide-angle lenses, such as that of the S3000, show barrel distortion.The barrel distortion was automatically adjusted previously using internal image processing function.The influence of the distortion adjustment appears in the center of the right-left row in the image.Therefore, the proposed calibration method focuses on this row for adjustment to compensate for the large distortion, such as shown in the row on the right in Figure 7. Table 2. Distortion model and its abbreviation (Wakutsu and Chikatsu, 2011)

Proposed Method
In the first step of the proposed method, the block in which the largest distortion occurs (e.g., the row on the right side in Figure 7) is extracted after ordinary camera calibration.Second, a fitted curve is computed by using GCPs in the extracted block.The extracted points are adjusted along the computed fitted curve (Figure 8).And in order to adjust left-right, the block of opposite side was computed by the same procedure.Finally, the camera calibration is repeated using all the points, including the adjusted points.
It is observed that the points of the row on the right side in the center of the image (Figure 7) have the largest distortion.Therefore, the points in this row were adjusted by using the proposed method.Furthermore, the row on the left side was also adjusted in this work.Figures 8 and 9 show the adjusted points of the side rows on the left and right.Table 3 compares the results of an ordinary camera calibration and the proposed method.These results show that the proportional accuracy was improved by the proposed method.However, the accuracy improvement was still issue.

CONCLUSION
In this study, the cause of accuracy degradation for consumer grade digital cameras, known as entry-cameras, was investigated.It is concluded that one of the reason for the accuracy could not improve even after procedure of camera calibration is not known whether the internal image processing function had been applied previously or not.The second reason is the occurrence of asymmetrical distortion.Generally, it is well known that wide-angle lenses, such as that of the S3000, show barrel distortion.Previously, this distortion was automatically adjusted using the internal image processing function.The influence of the distortion adjustment appears in the center of the right-left row in the image.Based on the above facts and assumption, the authors proposed a camera calibration method for the camera that does not show accuracy improvement after the camera calibration procedure.As a result of our method, it is observed that the accuracy shows some improvement and is expected to be improved even more by upgrading the proposed method.

Figure 1 .
Figure 1.Consumer grade digital cameras , DX, DY, DZ are the object field diameter; σ X , σ Y , and σ Z are the root mean square error for the check points. p

Figure
Figure 3. Experimental results

Figure
Figure 5. Results of camera calibration

Figure 7 .
Figure 7. Result of camera calibration of center image

Table 3 .
Result of proposed method Figure 8. Adjusted points : Original points (GCP) : Original points (check points) : Adjusted points