Interpolation Method using Statistical Models

1
Interpolation Method using Statistical Models

• RONEN SHER
• Supervisor MOSHE PORAT

2
Outline

• Black and White image interpolation
• Motivations
• Concepts
• Flow
• Results
• 1D Signal interpolation
• CCD Demosaicing
• Structure
• Methods Overveiw
• Components correlation
• Statistical extension
• Results
• Summary

3
Outline

• Black and White image interpolation
• Motivations
• Concepts
• Flow
• Results
• 1D Signal interpolation
• CCD Demosaicing
• Structure
• Methods Overveiw
• Components correlation
• Statistical extension
• Results
• Summary

4
The Problem

• Enlargement of an Image by 2x2

Input
Output
5
Image Interpolation Methods

• Nearest Neighbor
• Bilinear
• Bi-Cubic
• Spline

6
Motivations 1 Pixels Correlation

• Normalized histograms of Lena gray Levels
• 256x256 -solid and 512x512-dashed

7
Motivations 2 Image Compression Results
Compression rates in bits/sample
8
Approaching the problem
9
Approaching the problem
Near Lossless Compression Scheme
10
Lossless Compression predictors
11
Lossless Compression Context modeling

• The error value is being subtracted from the
  average error in a given context

Horizontal edge
Vertical edge
12
Outline

• Black and White image interpolation
• Motivations
• Concepts
• Flow
• Results
• 1D Signal interpolation
• CCD Demosaicing
• Structure
• Methods Overveiw
• Components correlation
• Statistical extension
• Results
• Summary

13
Image Regions

• In edges regions an average prediction will
  result in a smoothness effect.
• The edge must be preserved.
• The edges exist in the input image and the same
  distribution is assumed in the large image.

14
Image Regions

• In case of a horizontal edge

15
Pixels fitting
From Lena 256x256
16
Image Regions

• In each region a different weighted sum is valid
  for the prediction

• The coefficients
• are learned from the input image

17
Outline

• Black and White image interpolation
• Motivations
• Concepts
• Flow
• Results
• 1D Signal interpolation
• CCD Demosaicing
• Structure
• Methods Overveiw
• Components correlation
• Statistical extension
• Results
• Summary

18
Step 1 Coefficients calculation

• Scanning the Input Image
• for the x type pixel we determine its
  permutation from its four neighbors and save its
  value and its neighbors values in VMx
• Modeling only the regions
• with significant changes
• in gray levels
• Same treatment for the type pixels

19
Step 1 Coefficients calculation

• For each permutation we find the four
  coefficients using the Least Square solution

• Same technique for the coefficients

20
Step 2 x type Reconstruction

• Scanning the sparse Image, for each pixel we
• determine its matching
• permutation (coefficients)
• from its four neighbors and predict its value
• using

21
Step 2 type Reconstruction

• The Input is Ix, for each pixel
  we find its matching permutation
  (coefficients) and calculate
  its prediction by

22
Experiments - Lena

• The 4 coefficients in 24 cases of x-type

a2
a1
a4
a3
Errors

• Lena size 512x512
• Lena size 256x256

23
Experiments 1 - BW images (128x128-gt256x256)
Original
Bilinear
Nearest neighbor (Input)
Proposed
Bi-Cubic Spline
Bi-Cubic
24
Experiments 2 - BW images (128x128-gt256x256)
Original
Bilinear
Nearest neighbor (Input)
Bi-Cubic Spline
Bi-Cubic
25
Outline

• Black and White image interpolation
• Motivations
• Concepts
• Flow
• Results
• 1D Signal interpolation
• CCD Demosaicing
• Structure
• Methods Overveiw
• Components correlation
• Statistical extension
• Results
• Summary

26
One Dimension Interpolation
Interpolating yd, by using NR. Its adjacent
samples serve as the four neighbors for the
coefficients calculation.
27
Synthetic Test Signal

• y1sin(r.(53.sin(2.(r0.7)))).sin(7.(r0.9))
  
• t11,2..N1
• r(t1OS1)/100
• N12400
• f11
• Ts2
• OS13000
• L2

28
1D Interpolation results 1
29
1D Interpolation results 2
Voice signal the word Diskette
30
Outline

• Black and White image interpolation
• Motivations
• Concepts
• Flow
• Results
• 1D Signal interpolation
• CCD Demosaicing
• Structure
• Methods Overveiw
• Components correlation
• Statistical extension
• Results
• Summary

31
CCD structure
32
CCD Demosaicing Methods

• Bilinear
• Kimmel - gradient based function and hues
  R/G,B/G.
• Gunturk data consistency and similarity between
  the high-frequency components.
• Muresan - interpolates R-G,B-G.

• Not Linear
• Changing the Input

33
Simple Method

• Treating each color component as individual BW
  image

Original
Bilinear
Proposed
34
Simple Method 2 Aliasing Effect
Original
Bilinear
Simple Method
35
Components method

• Using all colors neighbors for the green
  reconstruction
• Reconstruct the difference of the colors
  components Hues (R-G, B-G, R-B). Processing
  smoother signals.

36
Statistical extension

• Separating each case to sub-regions for better
  characterization.
• Using the mean and the standard deviation of each
  neighbors set for the division (size invariant).
  
• Each Sub-region will have its own coefficients
  better representative of the region

37
Case Study

• Maximal Size Region

38
Case Study 2

• 1 Region
• 14 Sub-Regions
• 98 Sub-Regions
• 140 Sub-Regions
• 196 Sub-Regions

39
Results 1 (384x256)
Original
Bi-Linear
Gunturk
Optimal Numeric Values s 2 divisions E 7
divisions
Optimal recovery
Kimmel
Neighbors Rule
40
Results 2 (384x256)
Original
Bi-Linear
Gunturk
Optimal recovery
Kimmel
Neighbors Rule
41
Outline

• Black and White image interpolation
• Motivations
• Concepts
• Flow
• Results
• 1D Signal interpolation
• CCD Demosaicing
• Structure
• Methods Overveiw
• Components correlation
• Statistical extension
• Results
• Summary

42
Summary

• A new reconstruction method was presented for 1D
  signals, BW images and CCD demosaicing using the
  correlation between low and high resolution
  versions.
• A non linear Localize scheme was developed to
  overcome the artificial effects caused from under
  sampling.
• The new method showed better performance over the
  traditional scheme in terms of MSE in 1D
  interpolation.
• Satisfying results achieved in BW interpolation
  and CCD demosaicing, compared to other known
  techniques.

43
Back Up
44
Comparison Simple vs. Components
45
Mean and STD histograms
Mean
STD
Green