SINGLE IMAGE DEHAZING FOR VISIBILITY IMPROVEMENT

Images captured in foggy weather conditions often suffer from poor visibility, which will create a lot of impacts on the outdoor computer vision systems, such as video surveillance, intelligent transportation assistance system, remote sensing space cameras and so on. In this paper, we propose a new transmission estimated method to improve the visibility of single input image (with fog or haze), as well as the image’s details. Our approach stems from two important statistical observations about haze-free images and the haze itself. First, the famous dark channel prior, a statistics of the haze-free outdoor images, can be used to estimate the thickness of the haze; and second, gradient prior law of transmission maps, which is based on dark channel prior. By integrating these two priors, to estimate the unknown scene transmission map is modeled into a TV-regularization optimization problem. The experimental results show that the proposed approach can effectively improve the visibility and keep the details of fog degraded images in the meanwhile. * Corresponding author


INTRODUCTION
The presence of haze in bad weather will result in poor visibility and contrast lost in images, as a result, a lot of bad impacts will arise on computer vision applications, such as outdoor surveillance, object recognition and tracking, unmanned vehicle systems etc. and make the performance of these vision applications lose authenticity and practical value because all of these vision system are based on the assumption that the radiance from a scene point to observers is not altered by the airlight and intermediate medium.Therefore, in order to obtain the haze-free images and recover color, visibility and details of scene from degraded images, haze removal from images is inevitable.
Recently, image dehazing has become an important and urgent research problem in the field of computer vision, it has been receiving more and more increasing interests.However, since it is depth dependent, some classical image enhancement techniques, such as histogram equalization, histogram specification, can't receive satisfying effect.Dehazing techniques from the perspective of physical model of degraded images have emerged.
In earlier years, some dehazing approaches have been proposed mainly based on using multiple hazy images as input or additional depth knowledge.The basic idea is to exploit the differences between multiple images captured in the same scene under different atmosphere conditions.Narasimhan et al. (S. G. Narasimhan 2003a andS. Nayar 1999a) propose a physicsbased scattering model for scene structure from two or more degraded images.Polarized filter-based DOP method from multiple images taken with different degrees of polarization of same scene is described by Schechner et al. (Schechner 2001a andS. Shwartz 2006a).In addition, some depth-based methods are proposed for dehazing by requiring to provide some depth information from user inputs or known 3D models.For example, Tan et al. (K.K. Tan 2000a) dehaze images by assuming the scene depth is given.Kopf et al. (Kopf, 2008a) also proposed a dehazing method by using the scene depth information provided in 3D geographic models of scene.
However, above dehazing methods by taking multiple images of the same scene or additional depth knowledge usually difficult in many practical applications.Single image dehazing methods by using some stronger priors or assumptions have received much attention in these years and achieved significant progress.By directly maximizing the local contrast of a haze image, Tan et al. (R. T. Tan, 2008) propose an effective method for single image visibility enhancement, especially for regions with heavy hazes.However, sometimes it will fail since the method is not based on physical model.By assuming the transmission and surface shading are locally uncorrelated, Fattal (R. Fattal, 2008) estimates the haze by independent component analysis and then infers the medium transmission and the color, but can't work well for heavy fog image.Some novel dehazing methods from a single image using a Markov random field (MRF) framework are also described in recent papers (Fan Guo, 2014a andTarel 2013).Tarel and Hautiere proposed a median filter-based single image visibility restoration algorithm which can effectively preserve both edges and corners (JP Tarel and N.Hautiere, 2009); In addition, Ketan Tang et al. (Ketan Tang, Jianchao Yang, Jue Wang, 2014) try a learning-based new idea for single image dehazing by using random forest to learn a regression model for transmission estimation of hazy images.
So far, the most effective prior used for single image dehazing is the dark channel prior proposed by He et al.( K. He, 2009a), the prior is based on the important observation that most local patches in outdoor haze-free images contain some pixels whose intensity is very low in at least one color channel, this prior plus a soft-matting technique will work very well for single image dehazing, however, it is computational expensive.In order to solve the computation problem, many improved algorithms were presented, in which, Hu et al. (Hu Wei, 2010a) propose a novel gradient prior law based on dark channel prior.
In this paper, inspired by the patch-based dark channel prior and the gradient prior law, we propose a new transmission estimated method to enhance the visibility of the degraded image and preserve more image details at the same time.Our proposed new method is a two-step estimated method: firstly, a combination transmission function is defined by integrating the dark channel prior and gradient prior law, the new function can help to preserve more details because of its using a different patch size for transmission computing in depth abrupt jump regions; Then, the optimal scene transmission is further modeled into a TV-regularization problem for reducing halo artifacts.In this way, the final restored image can be obtained by taking the optimal transmission map into the image degradation model.
The rest of this paper is organized as follows.In Section 2, we introduce the image degradation model and the two useful priors -dark channel prior and gradient prior law; our two-step transmission estimation method including the combination transmission function and the optimal transmission computing by TV-regularization is described and detailed in Section 3; the experimental results and conclusion are discussed in Section 4 and Section 5.

Image Degradation Model
In computer vision field, the foggy image degradation model is widely described as follows: This equation is the model defined on the three RGB color channel.Here, x is the pixel location, I(x) represents the hazy image, i.e., the observed image, J is the scene radiance, i.e., the haze-free image, A is the global atmospheric light, usually we assume it as a constant, and t(x) is the scene transmission, which is correlated with scene depth based on the Lambert-beer law, and be expressed as follows: Here,  is the medium extinction coefficient, () dxis the scene depth.
In equation ( 1), the term ( ) ( ) J x t x usually is called the direct transmission of light from the object surface, and the second term (1 ( ) t x A  usually is called the airlight, as well the intensity of the atmospheric veil (JP Tarel and N.Hautiere, 2009) The goal of image dehazing is to recover the scene radiance () Jx from () Ix based on eq.( 1), this requires us to estimate () txand A from observed image () Ix firstly.

Dark Channel Prior and Block Effect
To solve the single image dehazing problem, He et al. propose a very simple but effective dark channel prior, which is a statistical assumption based on their observations of plenty of haze-free outdoor images.According to the dark channel prior, in most of the non-sky local image areas, at least one color channel will have pixels with very low intensity, in other words, the minimum intensity in such local patch should have a very low value, even close to zero and three main factors attribute the low intensity in the dark channel, that is, shadows, colorful objects and dark objects.
For any image () Jx , the dark channel is defined as follows: Here, c J is a color channel of J , and () x  is a local patch centered at x.
According to the dark channel prior, in most of non-sky local image areas, we have, With this statistical prior as theory constraints, He et al. suggest a 15*15 patch size to get dark channel with the assumption that depth is always same in a local patch, that is, the transmission () txis constant in the window, then the following eq.can be deduced: Here, () tx is the initial estimate value for transmission, further, the eq. is modified by Here, [0,1]   is a factor which describes the sense in the real world.
However, the assumption will fail when there are abrupt depth jumps in the patch, halo effects will appear and the bigger the patch size, the more serious the effects.
In order to avoid the block halo problem, He et al. use a softmating operation to get refined transmission map () tx , then the haze-free image () Jx can be recovered by the following Eqution, 0 ( ) ( ( ) ) / max( ( ), ) Here, 0 t is a small constant (a typical value is 0.1 by He)for avoiding division by zero.

Transmission Map Gradient Prior Law
The soft matting method works very well for halo removal, however, this operation will have expensive computational cost, therefore, lots of improved algorithms appear recently aiming at solving this problem.In which, Hu et al. (Hu Wei, 2010a) propose a transmission map gradient prior law based on dark channel prior.
This new prior comes from the experimental results in which the difference between the recovered haze-free image () Jx  and () Jx by using initial transmission map () tx  and the () txafter soft-mating has been studied.In plenty of experiments, they find that tt   is very small in most image areas, which result in corresponding J J   is very small in most areas, only except in a few abrupt depth jump areas, and it is also corresponding to the areas that have obvious different brightness in the transmission map.According to this experimental result, they propose a new prior assumption, called transmission map gradient prior law.
From this prior law, Hu et al. (Hu Wei, 2010a) draw the conclusion that the areas with bigger gradient changes are the areas need to be further optimized; and the areas with smaller gradient changes have no urgent need to be further optimized.The areas with bigger gradient changes are also the areas with bigger depth changes in the meantime.
Therefore, when soft-mating optimal operation or other refining operation is done to initial transmission map () tx  for refining, we don't need all the pixels in map () tx  image involved and just need to pay attention to the pixels in abrupt depth jump areas that correspond to intensity edge abrupt jump area, and it will reduce the computation cost greatly.
From Hu et al. experimental results, we can draw a further conclusion that, the halo size in the problem area (that is the halo effect area corresponding to the depth abrupt jump area) is very relevant to the block patch size, the halo effect will be more serious when the patch size is bigger.However, a big size guarantee us the statistical validity of dark channel prior, so, in order to reduce halo artifacts, a small patch size should be chosen just in the depth abrupt jump areas, which will help to preserve more details in the meantime.
In order to illustrate this new gradient prior assumption and conclusions more intuitively and easily, here, we use the experimental result images in Hu et al. 's paper (Hu Wei, 2010a) directly, as Figure 1 and Figure 2 shown, we can see above conclusions from this prior in images with whatever smooth depth variation or steep depth variation.

PROPOSED NEW METHOD
To solve the single image dehazing problem, appropriate transmission function () txand the global atmospheric light A should be estimated from the observed image I(x), it is essentially ill-posed problem.To cope with the ill-posed nature of these problems, some techniques under regularization have been developed and work well by adding some prior knowledge expressed into the regularization term.
In this paper, based on He's method for the atmospheric light A estimation, that is, by picking up the top 0.1% brightest pixels in the dark channel, we take the average intensity as the estimate of atmospheric light A.
Dark channel prior is based on the statistical assumption that the pixels in a local image patch has the same value.However, pixels in a local image patch will not always share a similar depth value, the assumption fails in abrupt depth change areas.
In this section, we propose a two-step estimated method for the appropriate transmission map: define a new combination transmission function and further scene transmission optimization by TV-regularization.In this way, the final dehazing image can be obtained by taking the optimal transmission map into the image degradation model.

A. Combination Transmission Function Defination
In section 2.2, we have conclusions from gradient prior law that initial calculated transmission by dark channel prior can be used directly for the recovery of the vast majority of the region, we only need to optimize a few pixels in problem areas; In addition, the size of the problem areas is very related to the patch size used in dark channel prior, the bigger the patch size, the more serious the halo artifacts, that is to say, the loss of the details will be more.So, in order to get more appropriate transmission value and preserve more details for further image dehazing, we should adjust the patch size to an appropriate size according to the detail information in patches, that is, in the abrupt depth jumps area, we need to use a smaller size patch, and regular size patch in others.Therefore, we define a new transmission combination function as follows: Here, B represents the area in which depth change is not obvious or abrupt, and B represents the area in which depth change is obvious and abrupt, that is, there is edge pixels in this area.() tx  is the initial transmission value under dark channel prior by the patch size usually chosen as 15*15.
Notice that halo artifacts always appear in the depth jump areas corresponding to transmission abrupt jump areas, so, in our paper, we detect edge abrupt jump position firstly in the initial transmission map () tx  image, not directly in the fog image to avoid too many unnecessary edge pixels, and then extend the edge area to a wide area according to the patch size used under dark channel prior, as Hu et al. does in (Hu Wei, 2010a).We can regard this as an 'inverse process' to achieve the halo area, and we call it problem areas need to be processed; Then, for each pixel in the problem areas, we use 3*3 block size for the patch for () new t x by dark channel prior just because small patch size can be fit better to the edge jump area.
In this way, that is, using different patch size in different region for transmission estimating, the new defined combination function can help us preserve more details in the abrupt depth jump regions and reduce the halo artifacts simultaneously. .

B. Transmission map Smoothing by TV-Regularization
The above defined combination transmission function can get a more accurate value compared with that based on a constant size patch, however, some small halo artifacts will still exist in the dehazing image where there is abrupt depth change, because essentially above proposed combination transmission function is still a block-based method.In this section, we will try to do some further improvements for optimal transmission map.
Note, in order to avoid letter confusing with time t in derivative operation, in the following text, we use () ux to represent transmission t(x).
Considering the facts that the depth jumps generally are related to the pixels at image edges, and so as the transmission, because , therefore, transmission map () ux should have the patch-wise character, is to say, gradient magnitude is sparse, and Total variation (TV) -regularization based method can describe well this character and preserve more details in edge regions, so, the problem that smoothing the transmission map for reducing the halo artifacts in abrupt jump regions can be modeled into a TV-regularization optimization problem, then its iterative optimal solution is just the optimal transmission map with fewer halo artifacts.
TV-regularization was introduced by Rudin, Osher and Fatemi(called ROF model) in (Rudin, Osher, Fatemi,1992a), and become popular in recent years and has been successfully used in image processing with ill-posed problems, the TV energy model is as follows: Here, the first part is fidelity term and the second part is TV regularization term, the parameter  balances the TV regularization term and the fitting term.Ideally, its optimal value should be estimated as well, and in most practical applications, it often serves as a tunable parameter to balance the fidelity of the restored image features and the suppression of oscillatory image noises(Tony.F, Jianhong Shen, 2005).
Computationally, the TV regularization model ( 9) is usually solved via its formal Euler-Lagrange equations.
Adopt the steepest descent marching with artificial time t: Then, we can get the discrete iterative formula as follows, Here, , ij is the coordinates of pixel x .
So far, we have described the two stages for transmission map, for completeness, the proposed algorithm is as follows: 1) Input fog image 8) Output the final refined transmission () ux and let ( ) ( ) t x u x  .
Till now, haze-free image J(x) can be derived by Eq. ( 7).

EXPERIMENTAL RESULTS AND ANALYSIS
In order to describe proposed method in detail and verify its practicability and effectiveness, we will give three group experimental results in this section.MATLAB R2014a is used to implement experiments on computer which has the windows 7 system with intel core i7 and 4G RAM.
Since the abrupt depth jump region in image is mostly corresponding to the edge, here, we firstly use edge detection to estimate the abrupt depth jump region.Simultaneously, in order to ensure its accuracy and avoid unnecessary weak edge pixels, we detect edge pixels on initial transmission map, not directly on original fog degrade image.As shown in Figure3, (a) is the input original fog image with poor color and contrast, if we detect edge pixels directly in it, we will get too many weak and unnecessary edges, as shown in (c).
From the experimental results, we can see that, by detecting the edge pixels on initial transmission map Figure 3(b), not directly on original fog image will make us describe the abrupt depth jump region more accurate, because we just need to leave the significant edge pixels as a reference, just as Figure 3(d) shows, the significant edge pixels are detected.
After that, the depth abrupt jump region can be got by extending from the edge pixels according to the patch size in dark channel prior.this is the most possible depth abrupt jump region, that is also the halo region we need to process.
Then, our experimental results about smoothing transmission map by TV-regularization are given.As Figure4 shows, we smooth the transmission map Figure 4(a) by TV-regularization and the parameters including iterations times niter, time step timestep and regularization parameter lamda.Specifically, we set niter=200 lamda=0.1,timestep=0.01 and the smooth result is shown in Figure 4(b).The iteration process and iteration formula are described in section 3.
From the contrast results in Figure 4(a) and Figure 4(b), we can see that the patch-wise constraints of transmission function in TV-regularization model will give us better smoothing transmission map.
Finally, to demonstrate effectiveness of the proposed method, we compare our method with He's method on single image dehazing.Figure 5 and Figure 6 illustrate the comparisons of proposed method with He's work.
As can be seen from the results, our method can produce comparable results with He's method, also can effectively enhance visibility of single input image with no more other information, and preserve image's details.This benefits from the two important statistical observations priors about haze-free images and the haze itself and the TV-regularization patch-wise character.Therefore, the hazes in the images can be fully removed and produce the clear image with good visibility, good color with more details.

DISCUSSION AND CONCLUSIONS
In this paper, we have proposed a new transmission estimation method for single image dehazing.Our method benefit from two priors, that is dark channel prior and transmission gradient prior law, based on them, a two-step transmission map can be achieved, that is, a combination function-based transmission map and a further optimized transmission map based on TVregularization, respectively.
The proposed new method not only can preserve more details in depth abrupt jumps areas, but also can be without no more assumption on the fog model.Experimental results show that proposed approach works well for dehazing single image.In the future, we will try to add more appropriate constraints on our model and extend our method to video and make it suitable to surveilance and some unmanned vehicle systems.

Figure 1 .
Figure 1.Dehazing photo with smooth depth variation.(a) Input image I (b) Coarse transmission map t' (c) Refined transmission map t (d) J J   (e) Dehazing image by t' (f) Dehazing image by t (a) Initial transmission map (b) TV iteration result Figure 4. Initial transmission and TV smoothing result (a) Input fog image (b) He's method (c) Proposed method Figure 6.Comparison with He's work (a) Input fog image (b) He's method (c) Proposed method Figure 5.Comparison with He's work (c) Edge detection for (a) (d) Significant edge for (b) Figure 3. Edge detection for abrupt depth jumps (a) Original fog image (b) Initial transmission map