Computational Analysis of Different Image Super-Resolution Reconstruction Algorithms

Super-resolution image reconstruction produces a highresolution image from a set of shifted, blurred, and decimated versions thereof. Super-resolution image restoration has become an active research issue in the field of image restoration. In general, super-resolution image restoration is an ill-posed problem. Prior knowledge about the image can be combined to make the problem well-posed, which contributes to some regularization methods. In these regularization methods, however, regularization parameter was selected by experience in some cases. Other techniques to compute the parameter had too heavy computation cost. This paper presents a generalization of restoration theory for the problem of Super-Resolution Reconstruction (SRR) of an image. In the SRR problem, a set of low quality images is given, and a single improved quality image which fuses their information is required. We present a model for this problem, and show how the classic restoration theory tools-ML, MAP and POCS-can be applied as a solution. A hybrid algorithm which joins the POCS and the ML benefits is suggested.


INTRODUCTION
The Classic theory of restoration of a single image from linear blur and additive noise has drawn a lot of research attention in the last three decades [1]- [4]. In the classic restoration problem in image processing, a blurred and noisy image is given and the purpose is to somehow restore the ideal image prior to the degradation effects. Such problem is typically modeled using the linear vector-matrix equation Y = H X  N -----------------------(1) Many algorithms were proposed in the literature for this classic and related problems, contributing to the construction of a unified theory that ties together many of the existing methods [4]. In the single image restoration theory, three major and distinct approaches are extensively used in order to get practical restoration algorithms: 1) Maximum likelihood (ML) estimator [1]- [4], 2) Maximum a posteriori (MAP) probability estimator [1]- [4], and 3) Projection onto convex sets (POCS) approach. Superresolution techniques offer a possibility to produce an image with a higher resolution from a set of images with lower resolution. The underlying mechanism is implied in the fact that different sub-pixel displacement of each low resolution images contains different information of the high resolution image. The pioneer work of super-resolution reconstruction may go back to 1984 by Tsai and Huang. Since then, many researchers have devoted themselves to the work in this area. Currently, the research is focused to a few points, such as high precision subresolution registration algorithm, blind super-resolution methods, robust and efficient reconstruction, real-time processing techniques. The goal of Super-resolution restoration is to reconstruct the original scene from a degraded observation. By "Superresolution", we refer to removal of blur caused by the image system (out of focus blur, motion blur, non-ideal sampling, etc.) as well as recovery of spatial frequency information beyond the diffraction limit of the optical system. This recovery process is critical to many image processing applications. And extracting a high resolution image from some low resolution image is required in many facets of image processing. For example, in remote sensing field, where several images of the same area are given, and an improved resolution image is required; or in video processing, where single frame in video signal is generally of poor quality. Enhancement of a single image can be done by using several successive images merged together by a superresolution algorithm. Super-resolution restoration from a still image is a well recognized example of an ill posed inverse problem. Such problems may be approached using regularization methods that constrain the feasible solution space by employing a-priori knowledge. This may be achieved in two complimentary ways.(1) obtain additional novel observation data and (2) constrain the feasible solution space with a-priori assumptions on the form of the solution. We identify three critical factors affecting super-resolution restoration. Firstly, reliable sub-pixel motion information is essential. Secondly, observation models must accurately describe the imaging system and its degradations. Thirdly, restoration methods must provide the maximum potential for inclusion of a-priori information. In tradition single image restoration problem only a single input image is available for processing. Super-resolution image restoration addresses the problem of producing super-resolution still image from several images, which contains additional similar, but not identical information. The additional information makes it possible that construct a higher resolution image form original data. Super-resolution techniques can be divided into two main divisions: frequency domains and spatial domain. w w w . i j c t o n l i n e . c o m The three approaches merge into one family of algorithms, which generalizes the single image restoration theory [1]- [4] on one hand, and the existing super-resolution algorithms proposed in the literature [5]- [14] on the other hand. The proposed restoration approach is general but assumes explicit knowledge of the linear space-and time-variant blur, the (additive Gaussian) noise, the different measured resolute ions, and the (smooth) motion flow. The presented methodology also enables the incorporation of POCS into the ML or MAP restoration algorithms, similar to the way it is done for the iterative single image restoration problem [4], yielding hybrid super-resolution restoration algorithm with further improved performance and assured convergence. .These noise vectors are uncorrelated between different measurements. Translating the above description to an analytical model as in Fig.1. we get,

Fig.1. Degradation Model for the Super-Resolution Problem
All these matrices (F k , C k , D k , W k ) are assumed to be known in advance. Justifying such an assumption is treated in [15]. Having the above model, grouping the N equations into one can be done for notational convenience. This way we get: Where we have defined H k = D k C k F k , and the autocorrelation of the Gaussian random vector E is ----------------------(4) The obtained model equation Y = H X + E is a classic restoration problem model [1]- [4]. Thus, we can easily apply the Maximum Likelihood (ML) estimator, the Maximum Aposteriori (MAP) estimator or the Projection Onto Convex Sets (POCS) methods in order to restore the image X, which is w w w . i j c t o n l i n e . c o m exactly our purpose here. In the following sections we shall briefly present the way to apply each of those tools.

ML RESTORATION
Applying the ML solution [15] 7) Locally adaptive regularization can be included in the above analysis with both algebraic and physical interpretations. Using the Laplacian operator S and a weighting matrix V (penalizing non smoothness according to the a priori knowledge on the smoothness required at each pixel), we get ------ (8) Differentiating again with respect to X and equating to zero yields the equation, which is the same as in (6), but a new term β S T V S, is added to the matrix R.

MAP RESTORATION
If we assume that the unknown X is a zero mean Gaussian random process with auto-correlation-matrix Q, the MAP estimator becomes the Minimum Mean Square Error estimator. Performing several algebraic steps [6] gives, And the resemblance to the ML result is evident. It can be shown [4] that if an autoregressive (AR) model is assumed on the image, a simple and direct connection between the Laplacian regularization matrix and the AR coefficients can be established. The ML, the MAP estimator reduces to a huge sparse set of equations which can be solved iteratively.

SET THEORETIC RESTORATION
According to the set theoretic approach [5], each a-priori knowledge on the required restored image should be formulated as a constraining convex set containing the restored image as a point within this set. Using the model presented earlier, we can suggest a group of such convex sets based on L 2 distance measure ------(12) for 1≤ k ≤ N. This defines a group of N convex sets -ellipsoids in this case. Since POCS requires a projection onto these sets, and since projection onto an ellipsoid is computationally very complex, L∞ constraints can be proposed instead [15].
--------------- (13) where θ k is the support region of the k-th measured image, and δ k stands for the uncertainty of the model [16], [17]. Another set which can be used is the one constraining smoothness. We can suggest L 2 or L∞ convex set versions as before -------------------- (14) or (15) where θ0 is the support region of the ideal image. We can incorporate additional nonlinear constraints such as constraints on the output energy, phase, support, and others. An often used constraint is the one posed on the amplitude of the result ---------------------------- (16) Having a group of M convex sets, each containing the required image, the Projection Onto Convex Set (POCS) method suggests the following iterative algorithm for the recovery of a point within the intersection of these sets [1,15] w w w . i j c t o n l i n e . c o m -------------------------- (17) Where P j is the projection of a given point onto the j th convex set. A different approach towards the POCS idea is the bounding ellipsoid method [15].For the case where all the constraints are ellipsoids this approach suggests finding the ellipsoid bounding the intersection of all the participating constraints, and to choosing its center as the output result. In [15] it is shown that the equation for the bounding ellipsoid center is exactly (for a specific case) the ML solution as given in equation (8).

HYBRID RESTORATION
While the ML and the MAP are numerically simpler to apply, the POCS is more general and can incorporate non-linear constraints into their construction process as well. In order to gain both these properties, a hybrid algorithm is proposed. We start by defining a new convex optimization problem which combines a quadratic scalar error with M convex constraints.
where the quadratic error takes care of the model and the smoothness errors, and the M additional constraints refer to the non-ellipsoids a-priori knowledge. Following the iterative methods presented by [1],we propose a simple yet effective two-phase iterative algorithm to solve the above optimization problem. Analysis of this method can be found in [15]. Suppose that an efficient iterative algorithm which is known to converge to the minimum of the scalar squared error is given -denoted by It. Algorithms such as the Conjugate Gradient or the Gauss-Siedel can be considered as excellent candidates for It. Beyond this first iterative algorithm It, M projection operators denoted by Jtk k= 1,2. . . M can be constructed; each of them projects onto a convex set representing a given constraint. Assuming that the M projections are all given using the Euclidean metric, we suggest the following global iterative step, This interlaced approach is generally converging to the suboptimal point of the problem given in equation (18).Adding several new iterations, where now It is replaced by the [19]Steepest Descent, updates the previous result and assures that the final convergence is to the optimal point, as is proved in [18].

SIMULATION RESULTS
In this section we present elementary example which demonstrate the effectiveness of the proposed method for the super-resolution restoration problem. A single 100 ×100 image was taken (gray values in the range 0-63), and from it we have generated 16 blurred, down-sampled and noisy images of size 50 × 50. The degradation includes random affine motion (with zoom in the range (0.9.1.1), rotation in the range (0,50°), and translation in the range -5,5) pixels, blur with the 1-D separable kernel (0.7 1.0 0.7) / 2.4, a 2:1 decimation ratio, and additive white Gaussian noise with ζ = 3. Fig.1 presents the ideal image, Fig.2 presents 4 images from the measurements, and Fig.3 shows the reconstructed image using the hybrid restoration algorithm with regularization.

CONCLUSION
This paper addresses the Super-Resolution Reconstruction problem and its solution, where given a number of moved, blurred, and noisy versions of a single ideal image, and one wants to restore the original image. To solve this problem, a new general model was introduced here. This model enabled the direct generalization of classic tools from restoration theory to the new problem. In this context, the ML, the MAP, and the POCS methods are all shown to be directly and simply applicable to super-resolution restoration with equivalencies between these methods. The restoration problem at hand in each of these approaches reduces to the problem of solving a very large set of sparse linear equations. A hybrid algorithm is proposed that combines the benefits of the simple ML estimator, and the ability of the POCS to incorporate non ellipsoids constraints. This hybrid algorithm solves a constrained convex minimization problem, combining all the a priori knowledge on the required result into the restoration process. An efficient iterative two-phase algorithm is presented for solving the defined problem, and convergence is assured to the optimal point. Simulations are performed to demonstrate superresolution restoration using the hybrid algorithm.