A SEMIAUTOMATIC ANOMALOUS CHANGE DETECTION METHOD FOR MONITORING AIMS

In the framework of the development of a web site devoted to the documentation and monitoring of the cultural heritage (above all monumental buildings), a semiautomatic method for anomalous change detection has been set up. The method uses the grouping of the image difference values, to detect both small and diffused changes. Three tests are described to evaluate the performances of the method. The results show that good performances are obtained in case of cloudy days, while the presence of shadows requires the interpretation of an operator to distinguish true and false changes.


INTRODUCTION
In the framework of the development of a web site devoted to the documentation and monitoring of the cultural heritage (above all monumental buildings), with particular regard to emergency management, anomaly detection and early warning (Artese,G., Gencarelli,M., 2008), a semiautomatic method for anomalous change detection has been set up.The system uses images captured and sent by camera phones; these images are compared with archived ones to detect anomalous changes and, consequently, to activate an early warning procedure.The availability of high resolution digital cameras combined with the possibilities offered by today's computers, both for camera calibration and image processing, allows the execution of controls and monitoring by using change detection techniques, based on the comparison of frames acquired at different times.The use of these techniques is widespread in various sectors, ranging from security to traffic monitoring, to the processing of radiographic images, etc.. .The issues involved range from the calibration of the cameras used, to the feature extraction, registration, and actual change detection.Some authors have proposed techniques that do not require prior calibration and optimal resampling (registration) and make use of the classification process of pixels (Theiler, J.,Perkins, S., 2006).

Calibration
Many algorithms have been proposed for automatic calibration of the images.For example, Cronk et al. (Cronk, S., 2006) have proposed an effective method in the case of many converging acquisitions.Another possibility is to consider non-rigid geometric deformations, due to lens distortion; for such a case Arsigny et al. (Arsigny, V., 2006) have proposed a general methodology to parameterize the deformation of the image with a finite number of rigid or affine components, while maintaining the reversibility of the global deformation.
Considering the presence of flat surfaces and straight lines, different strategies can be followed.Habib et al. (Habib, AS, 2002, Habib, AS, 2004, Habib, AS, 2005) have proposed a method for both calibration and registration, based on the use of straight lines.In the case of monitoring, internal and external orientation parameters are known, at least for a base image.In our work, straight lines were used during calibration of the camera lens to eliminate distortion, after having obtained the outlines with classical technique (Canny, J., 1986).

Identification of interest points
To obtain a good image registration, you must choose a set of interest points, which must be visible, and whose image coordinates must be known or measurable.For monitoring aims, one has in general the availability of one or more images in which interest points are detected through automatic or manual techniques.The classical operators Forstner (Forstner, W., 1987) Harris (Harris, CG, 1988) and Moravec (Moravec, HP, 1979) can be used effectively.In our work we used a semiautomatic technique, along with the operators of Harris and Moravec.

Cross-correlation
To perform registration, of fundamental importance for change detection, it is essential to find, in the images obtained at later dates, the interest points which were chosen in the base frames.For this purpose you can use the matching techniques.Among these, the cross-correlation is simple and effective (Jaehne, B., 1989).The cross-correlation allows to obtain the correspondence between two digital images, based on two assumptions: the images differ geometrically only due to a translation and radiometrically only for brightness and contrast.The accuracy of the cross-correlation decreases rapidly when the geometric assumptions are not met, especially when you have rotations greater than 20° or differences in scale greater than 30% (Forstner, W., 1984).The different perspectives of the images to compare cause both translations and rotations.For the buildings, it's in general possible to individuate target points on vanishing lines; in this case you can use a coarse cross-correlation to derive the transformation of the known targets (if you have four points and two vanishing lines).Thus, after processing of the second image, the cross-correlation can be performed with subpixel accuracy (Artese, G., 2007a).

Image processing and Change Detection
Many widespread programs allow processing of images, varying the brightness and contrast, and resampling after rotations.Regarding the resolution, the optimum depends on the minimum size of features that identify a change.In the case of cracks on the walls of buildings, it is essential a very high resolution, whereas in case of monitoring of degradation of the external plaster, it is preferable to have a normal resolution, to minimize computation time and memory use.For the change detection several techniques can be used.The simplest one you can effectively use is image difference.There are many factors that influence the result of a change detection: among the main ones we remember the average resolution of the images to compare, the not uniform resolutions in different parts of images, the differences due to different points of view, the environmental conditions at the time of the shots (variations in brightness, shadows, height of the sun, etc. ..), the presence of hidden parts.The ideal situation is with the same camera, point of the shot and identical environmental conditions.It is obvious that this situation almost never occurs.Even if the camera is fixed to a support, the image registration is necessary.Strictly speaking, one should speak of anomalous change detection, that is of changes non diffused in the entire image due to various causes (focus, different lighting conditions, etc..).In fact, the number of variations which one is really interested in identifying, is generally low, and the variations involve a few pixels.Most of pervasive differences are not interesting; often this differences are due to misregistration.An overview of anomalous change detection methods is given by Theiler (Theiler, J., 2008).

PREPROCESSING AND MUTUAL INFORMATION REGISTRATION
Due to the different characteristics of the images, a registration process has to be previously done.For this aims, a semiautomatic processing of the archived image is performed: i) the Canny filter is applied, for edge extraction, ii) Harris (Harris, C.G., Stephens, M., 1988), Moravec (Moravec, H.P., 1979) and Forstner (Forstner, W., Guelch, E., 1987) operators are used to obtain corners and interest points, iii) Straight lines are selected, iiii) a 'divide and conquer' strategy is applied and a Constrained Delaunay Triangulation is performed.In this way, every triangular zone of the image corresponds to a plane region of a building facade.
It is presumed that every camera phone shot is obtained from a point of view close to the one of the corresponding archived image.On the camera phone images, after the application of the Canny filter, the straight lines are used for the calibration.Through a cross-correlation, the triangles are found, corresponding to the ones obtained in the archived image, by using the constrained Delaunay triangulation.
To the sent image, a piecewise affine transformation is applied.
To detect the real anomalous changes, a radiometric registration should be performed, followed by the evaluation of the image differences.The registration is obtained using the well known mutual information value (Shannon, C.E., 1948): IM RS (r,s)= sum ( P RS (r,s) log ((P RS (r,s)/ P R (r) P S (s)) (1) where r and s are the pixel values from R and S images, PR(r) and PS(s) the probability distributions of r and s in each image, and PRS(r,s) the joint probability distribution of r and s.Thus, the minimum anomalousness registration can be used (Wohlberg, B., Theiler, J., 2009, Wohlberg, B., Theiler, J., 2010).This procedure is generally sufficient in case of concentrated changes (e.g.cracks on the walls).It works not properly when the changes are diffused, like in case of deterioration of a plaster zone in a facade.In this cases, a different strategy must be followed.

SEMIAUTOMATIC CHANGE DETECTION
If we simply take into account small outliers, we can think of an approach as Wohlberg and Theiler (Wohlberg, B., Theiler, J., 2010): you can compare the two images to be analyzed after subtracting the mean value for each pixel of the image, or portion of the image considered.This is not good, obviously, in case of differences that affect large areas, e.g. a large area of deteriorated plaster with a solid color: the anomalous change could then consist of a widespread change of an entire area, better described, e.g., by a linear regression, as reported by Heo and FitzHugh (Heo, J., FitzHugh, T., 2000).
To obtain the two coefficients (intercept and slope) one should have some sample areas; in which case you must know how various conditions of sun radiation and position (season, time and weather) influence radiometric properties and cause shadows.
If sample areas are not available, one can think to generate a georeferenced 3D model of the buildings, thus obtaining the shadows and the light intensity in the case of sunny days, but there is an unknown due to weather conditions (cloud cover may be thick or just a haze, etc. ..).In practice this is a very difficult way.
In this work, a recursive procedure has been used, to find the radiometric parameters, for which the maximum number of pixel with equal radiometric values is obtained.In this way, changes characterized by small values, but diffused on a large area are detected.
The image difference (new -base) is performed; the difference values are grouped thus obtaining a histogram for a grey scale image, or three histograms for a colour image.The shape of the histogram shows, in general, one or more peaks; once obtained the radiometric value of the maximum peak, a shift equal to this value is applied to the pixels of the new image.The procedure can be repeated for the red, green and blue channels of a colour image The difference between the resampled new image and the base image is then performed and a non maxima suppression (high pass filter) is applied.The remaining non zero pixels are considered as changes in the original image; a BW image is then obtained, where the white pixels correspond to the changes.After morphological operations (Canny filter, dilatation, filling and erosion) a segmentation is performed and the anomalous areas are classified.For every area, several properties can be extracted (centroid, area in pixels, etc..); among these, area and eccentricity can be useful for our aims.In fact, areas with less than 100 pixels can be generally considered as noises and eliminated; for a crack this isn't true, but in this case the value of the eccentricity is very high, generally greater than 0.95.The filtering is then performed by eliminating the areas with less than 100 pixels and an eccentricity less than 0.96.

Test 1: synthetic image
On a base image (figure 1) some changes have been applied: a small rotation and a translation to simulate a misregistration, a stain and a crack on the wall (figure 2).The histogram of the images difference (figure 2figure 1) is shown in figure 3: we can observe a second peak, due to the dark stain (images have been converted in grayscale).The 3D representation of the differences is shown in figure 4 (before shift and non maxima suppression) and in figure 5 (after shift and non maxima suppression).The crack and the stain are detected, along with false changes, due to misregistration.

Test 2: real case in a cloudy day
Figure 6 shows the base image.In this case a costrained Delaunay triangulation has been performed and the homologous points of the vertices are found in the new image to compare.The preprocessing previously described is performed.Figure 7 shows the new image with the triangulation.It must be underlined that the triangles obtained have been extracted after the resampling (affine transformation) of a wider area, to avoid misregistration on the borders.The comparison will be performed for every triangle; figure 10 shows the upper-right triangle after the affine transformation, with the contour of the detected anomalies.The histogram of the images difference shows a high peak (figure 8).The 3D representation of the differences (the triangle of figure 7 has been subtracted) is shown in figure 9 (after shift and non maxima suppression).The 3D representation of the differences (figure 14) shows both positive (true) and negative (false) differences.In the shadowed zone no differences are detected.
To better understand this case, we perform the differences limited to the single humps.Figures 15 and 16 show these differences.It is evident that the judgement of an operator is necessary, to distinguish the real changes from the false ones.Table 1.Results of the performed tests A quantitative analysis has been performed.In the Table 1 we found the results of the tests (for the test 3, the data are referred to the bright points).The actual anomalies have been obtained manually.
In the columns we find, for every test, the total number of pixels of base images and new images, the number of anomalous areas, both actual and detected before and after filtering, the number of anomalous pixels both actual and detected, the wrong detections and the number of not detected anomalous pixels.
The number of the detected areas after filtering is wrong in the first case, due to misregistration.The number of pixels detected in the test 3 is almost correct, but the contemporary detection in both illuminated and shadowed zones is not performed automatically.

CONCLUSIONS AND FUTURE WORKS
A semiautomatic method for anomalous change detection has been described.The method uses the grouping of the image difference values, to detect both small and diffused changes.
Three tests has been realized to evaluate the performances of the method.The results show that the better performances are obtained in case of cloudy days, while the contemporary presence of lighted and shadowed zones makes impossible an automatic detection.
The foreseen development of the research will regard the isolation of single areas and the use of the procedure for images with noises due to, e.g., rain or fog.

Figure 7 .
Figure 7.The new image of test 2 with the triangulation

Figure 9 .
Figure 9. 3D view of image difference of test 2 after radiometric shift and non maxima suppression

Figure 11 .Figure 12 .
Figure 11.The new image of test 3