4 Double Regularized Bayesian Estimation for Parametric Blur Identification
Figure 4.13: a|bc|d (a) Original video. (b) blurred background. (c) Restored image (d) Restored image based on different sampling area.
and real images with a wide range of noise, SNR’s (15→+∞) dB. The image restored by the adaptive weighted double regularization have better visual quality than the image restored by the traditional Wiener filters and by the non-adaptive algorithms.
Finally, a direction of the next step work is the development of iterative algorithms in removing nonlinear distortion in the presence of noises, based on recent results on regularization theory.
This work will be presented in the next chapter.
sampling feature space. The sampling feature space is “never fullfilled” (in some sense) in that the asymptotic results are usually assuming an infinite sampling numbern→ ∞. More over, the quality of sampling features also need to represent the characteristic properties of targets. For example, inhomogeneous regions including discontinuities are more useful than homogeneous sampling regions for blur identification. Although there are many different measures of descrip-tive information such as entropy, Gibbs distribution etc., the measure of blurred images is still difficult.
Practically, to achieve high accuracy blur identification in the case of non-stationarity for a sampling area, the first step is to classify and predicate the blur kernels in a family of right parametric PSF models. The second step is to optimize the parameters of the predicated para-metric blur kernel PSF. Global nonparapara-metric estimation can estimate the right distribution density in a sampling feature space. However, the choosing of right parameters is impossible in nonparametric regression methods or kernel density estimation methods. On the other hand, in the nonparametric theory there exist a number of modern welled developed methods such as automatic parameter selection like Cross-Validation [94] and Generalized Cross-Validation [94], Acaike [3], or Schwarz methods (Bayesian Information Criterion, BIC) [223]. These methods are not based on asymptotic minimax considerations but they are still hard for adjusting parameters of blur kernels without the constraints of image restoration.
The iterative structure of the suggested algorithm offers a number of advantages over non-iterative and recursive techniques, including the possibility of directly incorporating determin-istic knowledge and soft statdetermin-istical learning models into the restoration process, with the use of hard constraints. These constraints actually represented by projection onto convex sets. The soft or statistical constraints is in turn a function of regularization parameter. The weight of local variances representing human visual system were incorporated into the algorithm according to the observed noises and image discontinuities.
Furthermore, the convenience of the suggested algorithm is its interleaved prior property and double constraints. The estimation with respect to the PSF and the image is locally parametric optimized in the alternating minimization. The image estimation step estimates the true image assuming that the current estimates of PSF is correct prior knowledge, and vice versa. The predicated parametric model as an accurate initial value in regularization is obtained using the nonparametric Bayesian model selection technique. Thus, the approach is actually based on a local parametric optimization in nonparametric estimation strategy which was described by V.
Spokoiny [231]. The nonparametric estimation is adaptation of the parametric methods to the situation when the parametric structural assumption is not fulfilled.
4.6.2 Discussion of Related Optimization Approaches
Constrained Optimization
The important property of the Bayesian approach is that the Bayesian method which minimizes the deviation depends on an a prioridistribution. It is both the main advantage and the main disadvantage of the Bayesian approach. The advantage is that we can develop methods in accordance with average properties of the function to be minimized. The disadvantage is the arbitrariness and uncertainty of how to fix the a prioridistribution. To solve this problem, we use nonparametric estimation techniques to find a prior distribution for Bayesian estimation.
4 Double Regularized Bayesian Estimation for Parametric Blur Identification
Also we put the Bayesian estimation in a convex energy optimization functional which can ensure the global convergence.
D. Geman and G. Reynolds developed a constrained restoration approach for the recovery of discontinuities [86]. The idea was developed with a somewhat different coupled objective func-tion. The model is also “half quadratic ” and the auxiliary variables are also noninteracting but there is one crucial difference: the quadratic form is not block circulant FFT transform in the frequency domain. As a result, optimization must rely on updating pixels one by one in the spatial domain in the usual fashion. In contrast, the optimization method in our algorithm is to update pixel values in the FFT frequency domain based on a global optimization strategy.
Furthermore, D. German and G. Reynolds combine the first and second order terms that give consistently good results in their experiments than using the first and second order alone. With only first order terms, the objective function would favor regions of constant grey level. This suggests that purely first-order models would introduce an artificial patchiness or mottling, which is exactly what has been covered in a variety of studies. To the extent that grey-level images of real scene have homogeneous regions, these regions are better defined by constant gradient, or even constant curvature, then by constant grey level (This would become a TV method). These analysis give us some hints to develop a visual perception based data-driven image restoration approach.
On-line learning
Normal optimization techniques such as gradient ascent are undesirable because of their slow convergence. Alternatively, conjugate gradient or various preconditioned forms of gradient ascent techniques can be used due to their rapid convergence for quadratic optimization problems. The cost function being minimized is strictly convex and the cost function converges to the global minimum. Therefore, the exact restoration will be identical to a reconstruction compute using the modified EM algorithm.
On the other hand, much interest was devoted to the problem of on-line learning in pattern recognition. When data are presented sequentially to the estimator, on-line algorithms change their hypothesis and use the most recent data only. Hence the storage of the entire set of data is avoided. As discussed by Opper previously [180], [?] when one applies a smooth realizable stochastic rule to a random data sets. The on-line algorithm can achieve similar asymptotic generalization rates as the more complicated optimal batch algorithms.
Furthermore, Amari et al. [10] proposed a different on-line learning algorithm which minimizes a statistical dependency among outputs. The dependency is measured by the averaging mutual information (MI) of the outputs. A natural Riemannian gradient in structured parameter spaces is developed to minimize the MI based on information geometry theory [8], [9]. The on-line learning method based on the natural gradient is asymptotically as efficient as the optimal batch algorithm. This algorithm is transformation invariant and can be directly applied to Independent Component Analysis (ICA) problems and can be further extended in solving computer vision and pattern recognition problems.