On Interpolation Methods using Statistical Models

1
On Interpolation Methods using Statistical Models

• RONEN SHER
• Supervisor MOSHE PORAT

2
Outline

• Black White image interpolation
• Motivations
• Concepts
• Flow
• Results
• 1D Signal interpolation
• CCD Demosaicing
• Structure
• Methods Overveiw
• Components correlation
• Statistical extension
• Results
• Summary

3
The Interpolation Problem

• Factor of 2

Input
Output
4
Image Interpolation Methods

• Nearest Neighbor
• Bilinear
• Bi-Cubic
• Spline

5
Motivations 1 Pixels Correlation

• Normalized histograms of Lena (gray Levels)
• 256x256-dashed 512x512-solid

6
Motivations 2 Image Compression Results
Compression rates in bits/sample
7
Proposed Approach
8
Approaching the problem
Near Lossless Compression Scheme
9
Lossless Compression predictors
10
Lossless Compression - Context modeling

• The error value is subtracted from the average
  error in a given context

Horizontal edge
Vertical edge
11
Outline

• Black White image interpolation
• Motivations
• Concepts
• Flow
• Results
• 1D Signal interpolation
• CCD Demosaicing
• Structure
• Methods Overview
• Components correlation
• Statistical extension
• Results
• Summary

12
Image Regions

• In regions of edges, averaging will result in a
  smoothing effect.
• The edge must be preserved.
• The edges exist in the input image and the same
  distribution is assumed in the larger
  interpolated image.

13
Image Regions

• In case of a horizontal edge

14
Pixels fitting
From Lena 256x256
15
Image Regions

• In each region a different weighted sum is valid
  for the prediction

• The coefficients
• are learned from the input image

16
Outline

• Black White image interpolation
• Motivations
• Concepts
• Flow
• Results
• 1D Signal interpolation
• CCD Demosaicing
• Structure
• Methods Overveiw
• Components correlation
• Statistical extension
• Results
• Summary

17
Step 1 Coefficients calculation

• Scanning the Input Image
• for the x type pixel we determine its
  permutation from its four neighbors and save its
  value and its neighbors values in VMx
• Modeling only the regions
• with significant changes
• in gray levels
• Same treatment for the type pixels

18
Step 1 Coefficients calculation

• For each permutation we find the four
  coefficients using the Least Square solution

• Same technique for the coefficients

19
Step 2a x type Reconstruction

• Scanning the sparse Image, for each pixel we
• determine its matching
• permutation (coefficients)
• from its four neighbors and predict its value
• using

20
Step 2b type Reconstruction

• The Input is Ix, for each pixel
  we find its matching permutation
  (coefficients) and calculate
  its prediction by

21
Experiments - Lena

• The 4 coefficients in 24 cases of x-type

a2
a1
a4
a3
Errors

• Lena size 512x512
• Lena size 256x256

22
Example 1 - BW images (128x128-gt256x256)
Original
Bilinear
Nearest neighbor (Input)
Proposed
Bi-Cubic Spline
Bi-Cubic
23
Example 2 - BW images (128x128-gt256x256)
Original
Bilinear
Nearest neighbor (Input)
Proposed
Bi-Cubic Spline
Bi-Cubic
24
Outline

• Black and White image interpolation
• Motivations
• Concepts
• Flow
• Results
• 1D Signal interpolation
• CCD Demosaicing
• Structure
• Methods Overveiw
• Components correlation
• Statistical extension
• Results
• Summary

25
One-Dimensional Interpolation
Interpolating yd, using NR. Its adjacent samples
serve as the four neighbors for the coefficients
calculation.
26
Synthetic Test Signal

• y1sin(r.(53.sin(2.(r0.7)))).sin(7.(r0.9))
  
• t11,2..N1
• r(t1OS1)/100
• N12400
• f11
• Ts2
• OS13000
• L2

27
1D Interpolation result 1
28
1D Interpolation result 2
Voice signal the word Diskette
29
Outline

• Black and White image interpolation
• Motivations
• Concepts
• Flow
• Results
• 1D Signal interpolation
• CCD Demosaicing
• Structure
• Methods Overveiw
• Components correlation
• Statistical extension
• Results
• Summary

30
CCD structure
31
CCD Demosaicing Methods

• Bilinear
• Kimmel - gradient based function and hues
  R/G,B/G.
• Gunturk data consistency and similarity between
  the high-frequency components.
• Muresan - interpolates R-G,B-G.

• Not Linear
• Changing the Input

32
Basic Method

• Treating each color component as an individual
  BW image

Original
Bilinear
Proposed
33
Basic Method Aliasing Effect
Original
Bilinear
Basic Method
34
Components method

• Using all colors neighbors for the green
  reconstruction.
• Reconstructing the difference of the colors
  components Hues (R-G, B-G, R-B). Processing
  smoother signals.

35
Statistical generalization

• Separating each case to sub-regions for better
  characterization.
• Using the mean and the standard deviation of each
  neighbors set for the division (size invariant).
  
• Each Sub-region will have its own coefficients
  better representation of the region.

36
Case Study
From Light-House

• Maximal Size Region

37
Case Study 2

• 1 Region
• 14 Sub-Regions
• 98 Sub-Regions
• 140 Sub-Regions
• 196 Sub-Regions

38
Results 1 (384x256)
Original
Bi-Linear
Gunturk
Optimal Numeric Values s 2 divisions E 7
divisions
Optimal recovery
Kimmel
Neighbors Rule
39
Results 2 (384x256)
Original
Bi-Linear
Gunturk
Optimal recovery
Kimmel
Neighbors Rule
40
Summary

• A new interpolation method has been presented for
  1D signals, BW images and CCD color demosaicing
  based on the correlation between low and high
  resolution versions.
• A non linear localized method was developed to
  overcome the artificial effects caused from under
  sampling.
• The proposed method outperforms the traditional
  scheme in terms of MSE.
• Good results has been achieved in 2D
  interpolation and CCD demosaicing.

41
Appendix
42
Comparison Basic vs. Components
43
Mean and STD histograms
Mean
STD
Green
-- 192x128 -- 384x256
From Light-House