Nonlinear Smoothing-Enhancing PDEs

Perona-Malik filter

An important improvement of the classical linear analysis, with a more accurate multi-scale edge detection, was proposed by Perona and Malik [191]. The main idea of Perona and Malik is to

2 Regularization for Image Deblurring and Denoising

introduce a part of the edge detection step in the filtering itself, allowing an interaction between scales into the algorithm. They proposed to replace the linear heat equation by a nonlinear equation. The nonlinear diffusion equations Eq. 2.14 can thus behave locally as inverse heat equations due to the choices ofc. Perona and Malik suggested two diffusion coefficients ofc,

c(|∇f|²) =e^{−|∇f|2/k2} and c(|∇f|²) = 1

1 +|∇f|²/k² (2.18)

where k is a contrast parameter to be tuned for a particular application. These two diffusion coefficient c are scalar-valued, decreasing functions with isotropic but non-homogeneous [230]

nonlinear diffusion [259]. They also have similar basic properties such as positive coefficient, non-convexity and the ability of local enhancement for distinguished gradients. Detailed analysis and comparison of two diffusion coefficients andk are shown in Fig. 2.8. The P-M process removes noise while keeping edges and discontinuities. Some isolated noise points still remain, while some detailed textural information is lost during the process.

Catte filter

However, some drawbacks and limitations of the original model can drive the diffusion process to undesirable results [191] as mentioned in this original paper. For example, “staircasing” effects can easily happen around smooth edges. The ill-posedness of the diffusion may be alleviated through a smoothing operation to the variable in the diffusion coefficient c(s) = c(|∇f|²) in a regularized framework. This idea was introduced by Catte et al. [38] and the P-M model Eq. 2.14 is extended to in the following,

∂f

∂t = div(c(|∇G_σ∗f|²)∇f) (2.19)

with Neumann boundary condition. ∇G_σ∗f denotes a convolution of the image at timetwith a Gaussian kernel of standard deviationσ, which is to be given a priori. This formulation has solved a theoretical problem associated with Perona-Malik process. However, the selection of σ is critical to the Catte diffusion in the sense that the diffusion process would be ill-posed for too small scale, while the image features would be smeared for too large scaleσ. One possible solution is to use a large scale initially to suppress the noise and then to reduce the scale so the image features are not further smeared [283]. Thus, the optimal selection of scale is still an open question. In Fig. 2.9, we show the role of the standard deviation σ. The isolated noise points are “cleaned” using the rightσ. However, some detailed textural information is still weakly lost.

Different smoothing-enhancing nonlinear filters

Through the literature study, we list some other state-of-the-art nonlinear diffusion filters in Ta-ble. 2.3. These nonlinear smoothing-enhancing diffusion filters have been developed for achieving different purposes in image restoration:

1. Based on the pioneer work of scalar-valued diffusion technique from Perona and Malik [191], we can directly smooth color images in each channel independently. However, this

(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

(j) (k) (l)

Figure 2.8: Perona-Malik (P-M) scalar-valued image diffusion filter. (a) Original tulip image. (b)(c) Input color image with independent Gaussian noise in each RGB color channel, sigma =20. (g)(h)(i) zoom in (d)(e)(f). (d)(e)(f) are processed on each channel using P-M(I): c(s²) = exp(^−s_2k2²) with respect to k =5, k= 20, k=35. (j)(k)(l)are processed on each channel using P-M(II):c(s²) = _1+s¹_2/k2 with respect to k=5, k=20, k=35. Comparing (d and j), (e and k), (f and l), we can note that P-M(II) is relatively stronger than P-M(I) for image diffusion. However, these two filters have the same properties. Isolated noise points have disappeared in the whole image, some textural information is completely lost. For such methods it can be shown that small scales are smoothed faster than large ones, so if the method is stopped at a suitable final time, we may expect that noise is smoothed while large-scale features are preserved to some extent.

2 Regularization for Image Deblurring and Denoising

(a) (b) (c)

Figure 2.9: Comparison of differentσvalue to suppress isolated noise points in Catte diffusion. Perona-Malikc(s²) = exp(^−s_2k2²) with k= 20 based Catte diffusion ^∂f_∂t = div(c(|∇Gσ∗f|²)∇f). We input a similar noise color image for testing. (a)σ= 0.1. (b) σ= 0.2. (c)σ= 0.3. Whileσ= 0.3, isolated noise points have disappeared in the whole image, but some detailed textural information have lost.

procedure is not an optimal method in that the diffusion ignores the information from its neighbor channels. To find an optimal approach, vector-valued diffusion filters have been proposed for solving this problem [246]. The main idea is to smooth all vector channels f_i using a joint gradient information of all channels which can achieve better optimized results.

2. Following the vector-valued diffusion scheme, one can introduce nonlinear diffusion for matrix-valued data which is useful for tomography data processing. Different from the matrix-valued diffusionscheme [244], a coupling between all matrix channels is crucial for preserving matrix properties [262], orientation estimation in the matrix fields and different goals in smoothing.

3. Although these diffusion schemes enhance and restore images in an edge-preserving man-ner, the diffusion strength is performed equally in all directions. An ideal diffusion manner is to perform the smoothing along edges without smoothing across edges. Following this idea, Weickert et al. [259] developed an anisotropic diffusion filter which replaces the scalar-valued diffusivity c by a matrix-valued diffusion tensor D. The matrix tensor D controls smoothing along edges with forward diffusion and enhancing edges by backward diffusion in perpendicular direction simultaneously. The anisotropic diffusion method is useful in many area. Further study is referred to [259], [256], [258], [261], [267].

4. Thestructure tensor[79] is listed here because it is closely related to those diffusion filters and have gained significant importance in the field of scientific visualization and image processing [263]. It is also named second moment matrix which includes the estimation of orientation and the local analysis of image structure. Structure tensor is a fundamen-tal concept for corner detection [79] [207], passive navigation [103], image segmentation [155] as well as surface reconstruction. The linear structure tensor is based on Gaussian convolution, while a nonlinear structure tensor is based on different nonlinear diffusions.

Therefore, the underlying mechanism defining structure tensors can be implemented by means of different optimization approaches in scale space based on the demands of the goal. Currently, many elegant methods have been developed in the tensor field referring to the book by Weickert and Hagen [263].

Synthetic image Blurred image Noise blurred image

Time =0.1, spatial step =1 Time =0.2, spatial step =1 Time = 0.5, spatial Step =1

Time =0.1, spatial step =1 Time =0.2, spatial step =1 Time = 0.5, spatial Step =1

Time =100 spatial step =100 Time =50, spatial step =100 Time =1, spatial step =100

Figure 2.10: Shock filter diffusion and sharpening. From these experiments, we can summarize the main properties of the shock filter. Firstly, the filter is local extrema remain unchanged in time. No erroneous local extrema are created. Secondly, the steady state (weak) solution is piecewise constant (with discontinuities at the inflection points off0. Thirdly, the process can be approximated to deconvolution.

Finally, the shocks amplify at inflection point (second derivative zero-crossings).