Structure Based Image Magnification or Super-resolution

Multiscale structure based image representation using a set of regions

The application fields are (i) of appropriate granularity for best image compression, (ii) of appropriately rescaled size for image magnification or superresolution, and (iii) for smoothing for image quality restoration through structure-preserving denoising.

Resolution enhancement involves the problem of magnifying a small image to several times its size while avoiding blurring, ringing and other artifacts. We tackle the problem of magnifying an image without incurring the edge enhancement effects and other structural distortions characteristic of classical image magnification techniques. We propose an iterative algorithm based on a Projections onto Convex Sets (POCS) formalization.

Application fields are Image Magnification and Phase Retrieval. Classical image magnification methods include bilinear, bicubic and FIR interpolation schemes followed by a sharpening method like unsharp masking. Such interpolation schemes tend to blur the images when applied indiscriminately. Unsharp masking, which involves subtracting a properly scaled Laplacian of the image from itself, produces artifacts and increases noise. More sophisticated schemes involving wavelet- or fractal-based techniques have also been proposed. Such methods perform extrapolation of the signal in either the wavelet or fractal domain, which leads to objectionable artifacts when the assumptions behind such extrapolation are violated. It may also be noted that such extrapolatory assumptions predict and actively enhance the high-frequency content within the image, thus increasing any noise present in the unmagnified image.

The proposed method starts with an initial magnified image obtained through selective interpolation followed by an iterative procedure which aims to avoid edge-related artifacts while retaining and enhancing sharpness. The initial image is a composite image formed from a base interpolation scheme in the smooth areas of the image and from a selective interpolation mechanism in the non-smooth (or edge) areas. The proposed iterative algorithm aims to find a magnified image satisfying two constraints: one of the constraints is derived from sampling theory while the other constraint reflects the confidence that we place on the initial iterate. Both the constraints are convex sets; thus, we seek a solution which is at the intersection of these two convex sets and can be obtained using the projection on convex sets (POCS) method. Starting with the initial iterate, we project alternately on the two constraints. Convergence is guaranteed since we operate within the POCS formalism.

Results

Related Publications:

K. Ratakonda and N. Ahuja, POCS-Based Adaptive Image Magnification, Proc. International Conference on Image Processing, Vol. 3, Chicago, IL, October 1998, 203-207.
K. Ratakonda and N. Ahuja, “Super Resolution with Region Sensitive Interpolation”, Proc. Image Understanding Workshop, New Orleans, LA, May 1997, 537-540.