TOWARDS AN EASIER ORIENTATION FOR SPHERICAL PHOTOGRAMMETRY

For architectural metric documentation, Spherical Photogrammetry (SP) has demonstrated its validity and efficiency in many projects already. The speed of surveying is high, the accuracy and completeness of the plotting are satisfactory. However, there are still many problems to be solved. The weakest point is the orientation procedure, which is rather difficult to perform, in the sense that only very experienced people can run it, and few people only make use of it. The old orientation steps are 1) model formation (limited to binocular panoramas couples); 2) link of all the models in a block adjustment with independent model triangulation; 3) block bundle adjustment with 4 parameters/pano (3 coord.+1 orientation bearing); 4) block bundle adjustment with 6 parameters/pano, say the previous 4 + 2 correction angles around the horizontal axes. The panoramas must be spherical and quasihorizontal. In order to make easier the orientation, enabling more people to use SP, an improved approach has been set up. It consists in the combination of any possible model formed either by three and two panoramas. The trinocular vision, say the combination of three different panoramas to form a unique model, has the advantage to be much more robust in comparison to binocular vision in the sense that the trinocular model is likely to be more error-free than any of the three composing binocular models. It contains less model deformation, the model coordinates are validated by the mutual comparison of the three intersecting binocular models. In addition, the number of possible trinocular models is normally much larger than the one of binocular models. The steps for a semi-automatic orientation of a block of panoramas proceed as follows: Form any possible trinocular models by combination of the panoramas; in case that no trinocular model has been formed, form any possible binocular model; run a block adjustment with the algorithm of independent model, to link together the models in a unique reference system; run a block bundle adjustment with collinearity equations with 4 unknown parameters per panorama; run a block bundle adjustment with collinearity equations with 6 unknown parameters per panorama. The control requirements are limited to one fixed control point and two perpendicular constraints. Given a project with n panoramas, the trinocular possible models are the combination of n, 3 by 3, say (n*(n-1)*(n-2))/3!. As an example, in a project with 10 panoramas the trinocular possible models are 120, while the possible binocular models are (n*(n1))/2= 45. Obviously not all the panoramas will be formed, because a minimum of four common tie points is required. It is possible to select and filter out the best models by fixing a critical value on sigma naught for testing. The advantages are as follows: no need of approximate value for the unknown coordinates and parameters; easiness to find gross errors in the gradual block formation. The epipolar geometry has been investigated, the expression of the curve described and assessed: it can be useful in the trinocular model formation. For any try of the model the operator, thus causing troubles to him, must approximately choose formation, up to now the starting value of the orientation bearing for any panorama. However, a new procedure, not requiring any approximate value of the orientation bearing, has been set-up and tested: the orientation bearing can vary stepwise by a given selected interval, until the model is successfully formed. The computation time increases significantly, but it is still negligible. Up to now the control and tie points selection and observation is manual, making the orientation and the plotting to be slow and time consuming. Nevertheless, on the other end, the procedure is fully controlled and supervised by the operator, enabling the observation of the residuals and allowing the improvement of the orientation, and to perform some statistical quality analysis on the adjustment, which are not possible with automatic procedures based on Structure from Motion (SfM). Another weak point of the SP is the speed of plotting. Commercial programs are now available, based on SFM algorithm, producing point clouds of good quality in an unknown scale and orientation. The correct scale and orientation can be therefore supplied by SP orientation with high accuracy. Examples are shown taken from two real projects.


INTRODUCTION
Panoramic Spherical Photogrammetry (PSP) makes use of spherical panoramas as information support.They are formed by overlapping images taken from the same point and stitched together on a sphere (Szelinsky and Shum, 1997) then mapped on a plane with the so-called latitude-longitude projection for the stitching for the type of cartographic projection (Snyder, 1993).The advantages of the spherical panorama consist in the Field Of View (FOV) that can reach 360°x180°, and in the high image resolution, image being omnidirectional.From the panorama image point, the two direction angles f and a towards the object point can be derived as they were taken with a theodolite (Figure 2).The coordinates of an arbitrary object point can be obtained by intersection of corresponding straight lines connecting the image point with the center of two or more spheres (orientation).The PSP can be suitable for fast and accurate recording of Architectural Cultural Heritage (Pisa et al., 2010).Attempts have been done for the automatic detection of corresponding points in the panoramas (Barazzetti et al.,2010).To overcome the lack of stereoscopy and enable complex surfaces to be plotted, the photo-modeling has been used; say the back-projection of the oriented panoramic spheres     , et al. 2009, Fangi, Piermattei, Wahbeh 2013).
The nature and algorithm of the epipolar curve has been investigated and shown (Fangi, Nardinocchi,2013).So far, the integration with point clouds produced by SFM algorithms has been fruitfully used in (Fangi, Pierdicca, 2012).Here the trinocular model for spherical panoramas is introduced.

THE ORIENTATION PROCEDURE OF THE SPHERICAL PANORAMAS
The orientation procedure (Fig. 1) for spherical panoramas has been explained (Fangi, Nardinocchi, 2013).This procedure takes place as follows: -Relative Orientation by coplanarity ending with the models formation -Block Independent Model Triangulation -Block Bundle Adjustment One model can be composed by two panoramas, having in common three or more tie points (binocular model).All the formed models by coplanarity, are linked together in a block adjustment with independent models (Independent Models Block Adjustment -IMBA).IMBA provides the approximated values for parameters and coordinates to be input in a Block Bundle Adjustment, this last using the collinearity equations.Block Bundle Adjustment can be performed in two steps: the first one with 4 parameters/per pano and the second final one with 6 parameters/per pano.The reason for that is the possibility to better find the gross errors.The coplanarity condition is: where x' and x" are the spherical coordinates of the image point P', M' and M" the two rotation matrices, function of the angles φ =y/R and a = x/R+ q 0 being q 0 the zero bearing for the a panorama (figure 2), x and y the panorama image coordinates, X the spherical coordinates, b the base vector.
For the absolute orientation of the block of the panoramas, a set of control points can be supplied, taken with traditional means (GPS, total station), as it is advisable for a professional project.As an alternative, the minimum requirements are one fixed point, one distance for the model size, together with three geometrical conditions in orthogonal directions.In (2) we can set by=0 and bx=1 and the unknowns reduce to three q 0 ' , q 0 ", and b z .Since the coplanarity condition ( 2) is not a linear equation, it has to be linearized.Therefore it is necessary to input approximate values for the unknowns, for b z it can be set to zero but for the two orientation angles q 0 ', q 0 ", the operator must give approximated values, which are troublesome.But we made an improvement: the new computation procedure can start with arbitrary values for q 0 ', q 0 ".In case of failure of the iterative computation, the next attempt will start with initial values differing from the previous ones by a preset value.The approximated orientation bearing can vary stepwise by a given selected interval, until the model is successfully formed.The computation time increases significantly, but it is still negligible.The novelty of the here described orientation procedure consists in the trinocular model, which is the model formed by three panoramas (trinocular modelsee Fig. 3,4,5).Trinocular model have been already investigated by many authors and by the A. (Fangi, 1998).Trinocular vision is used for machine vision in industrial application (Heinrichs et al.,2007) and medical application (Sechidis, Malian et al., 2000).The advantages of the trinocular models are in terms of amount of needed tie points, in terms of number of formed models, but mainly in their strength: while the binocular models of spherical panoramas can be easily deformed due to the orientation imperfections and bad estimation of the orientation parameters (Fangi, 2010), the trinocular models have less deformation, because of any tie point not one coplanarity condition only, but even three coplanarity hold.

The Trinocular Model
In case of a triple intersection, say intersection of the arbitrary point P from three panoramas S1, S 2 and S 3 , the interested planes are three (figures 3,4,5), three independent stereo-pairs, i.e. left-right, left-center, and center-right.Per any observed point we can then write three condition equations.The unknowns are in this case:  the three coordinates of a station (3)  the two components of one of the three bases (2)  the three components of the remaining two bases 3x2= 6 We can select the type of orientation such as to set equal to zero the coordinates of one of the three stations, S1, then equal to zero the Y coordinate of S2.For quasi-horizontal panoramas, as they are in PSP, we can set the rotation angles α x , α y around the horizontal axes x and y, equal to zero, thus reducing the amount of unknowns, and enabling a better detection of gross errors.They are estimated later on, α x , α y , starting from initial zero value, in the last stage of the orientation, the block bundle adjustment.We define a positive versus in the triangle sides.We define the components (with sign) of the three bases: To the coplanarity equations three conditions must be added, relative to the components in the three directions of the three bases.
Not all the 10 parameters of orientation are indeed independent: In fact, set b x

  
(5) Then the independent parameters are only the following seven: being q1 , q2, q3 the three zero bearing for any panorama, say the direction clockwise of the origin of the image with respect to the reference system.We can write three linearized coplanarity conditions.For the approximated values of all parameters, for the bearings we behave like for binocular models.We can set to zero bz components, b x (2) =b y (2) = 0.5.In case of failure of the computation we change the sign of b y (2) =-0.5.To solve for the 7 unknowns, at least three points must be observed, with a redundancy equal to 2 (in the case of intersection only from two stations the redundancy would be zero).
With n observed points (m= observ.dir., c =equat.; u= unknowns ; r =redun.:)from 2 stations: From the graph in Figure 6, one can see as follows: when n  5 for increasing the redundancy, it is convenient to set up three pano-stations better than to enlarge the number of the observed points from two stations.A third panorama helps to solve problems and provides additional constraints on panorama link, dramatically reducing the chance of gross errors.For the same project, the number of trinocular models is larger than the one of binocular models.Than it is necessary to filter out the worse ones.The filter can be performed by testing the final sigma naught with a preset value.Another test is the interval of admissible dimension for the model.Given d max the distance to an arbitrary point from one of the three base point and b max the value of the largest of the three bases, the ratio d max /b max should be included in a prefixed interval to avoid ugly configuration.

THE PSP IN COMBINATION WITH POINT CLOUD FROM SFM ALGORITHM
Another weak point of the PSP is the speed of plotting.
Commercial programs are now available, based on SfM algorithms, producing point clouds of good quality in an unknown scale and orientation.The correct scale and orientation can be therefore supplied by PSP orientation with high accuracy.In the following, an example taken from a real project is shown.Santa Maria della Piazza façade, Ancona (Italy) has been surveyed with 9 spherical panoramas (figure 8).
The possible triplets are (9x8x7/6!=84).Choosing the limit for sigma naught value σ 0 =0.001 rad, none model was formed, for σ 0 =0.002 one model only, for s0=0.003 the formed triplets are 13, and finally for σ 0 =0.005 the triplets successfully formed are 43.By increasing the value for sigma naught limit, increases the number of formed models, but in the same time increases also the possibility of undetected gross errors.
The orientation could provide nine control points well distributed on the surface.A point cloud project has been performed by means of a commercial software (Photoscan).The control points provided by PSP have been input in the Photoscan project, and the results are in Fig. 7.The average discrepancy is 1.08 cm while the maximum difference is 2.5 cm.

CONCLUSIONS
The most difficult phase for PSP is the orientation phase.In order to make it easier some improvements have been set up in the procedure, the most significant one is the trinocular models.They are more robust in comparison with the binocular ones.
The combination with commercial software for the point cloud production like Photoscan has been tested, giving good results.PSP provides the needed control points for the point clouds.

Figure 1 :
Figure 1: Flow-chart of the SP orientation procedure.

Figure 2 :
Figure 2: The sphere and its mapping.Figure 3: Triple intersection: the intersected planes.

Figure 3 :
Figure 2: The sphere and its mapping.Figure 3: Triple intersection: the intersected planes.

Figure 4 :
Figure 4: Selected reference system for trinocular model Figure 5: The three intersecting straight lines

Figure 6 :
Figure 6: Relation between the number of the observed points and the number of observations.The model formations can be a) fully automatic, b) select one panorama, c) select one couple, c) select a triplet.a) The procedure tries to form trinocular model with any possible combination of existing panoramas; b) The procedure tries to form trinocular models with any possible combination of two panoramas with the selected one; c) The procedure tries to form trinocular model with any possible combination of the two selected ones with the remaining panorama; d) The procedure tries to form the trinocular model formed by the selected triplet From experience the most advisable choice of the four mentioned is the c).It permits to find gross errors as soon as they show up.

Figure 7 :
Figure 7: Errors of the control points produced by SP in a Photoscan Project -Santa Maria della Piazza façade, Ancona, Italy.