MRFBased EdgeDirected Interpolation

1
MRF-Based Edge-Directed Interpolation

• Min Li and Truong Nguyen
• Digital Video Processing lab.,
• http//videoprocessing.ucsd.edu
• ECE Dept.,
• University of California, San Diego

2
Outline

• The spatial interpolation problem applications
  and challenges
• MRF-based edge-directed interpolation method
• Concepts in MRF models
• Two-dimensional Discontinuity-Adaptive Smoothness
  (DAS) constraint
• The proposed MRF model
• Applications in spatial interpolation
• Simulation results
• Conclusions and future work

3
The Spatial Interpolation Problem
Definition
4
Flow Diagram
MRF model
Formulation of the 2-D DAS constraint
U(?)
MAP
The application in spatial interpolation
Implementation Simulation results
5
MRF Model
In MRF model, an image is regarded as a 2-D
random field on a 2-D lattice. Furthermore, in
this random field, there are
Stan Z. Li, 2001 S.Z. Li, Markov Random Field
Modeling in Image Analysis, Springer-Verlag,2001.
GemanGeman, 1984 S. Geman and D.
Geman,Stochastic Relaxation, Gibbs
distribution, and the Bayesian restoration of
images,vol.6,no.6,pp. 721741,Nov. 1984.
6
Concepts in an MRF Model
Cliques
Neighborhood structure
Potential and energy functions
7
MAP-MRF Formulation
The interpolation problem can be stated as the
interpolation of the high resolution image given
the available low resolution data. The Maximum A
Posteriori (MAP) solution corresponds to the
minimal energy state of the high resolution image
if it is modeled as an MRF.
8
1-DDAS
Discontinuity features are related to large
intensity variations, of which the potentials are
bounded.
Stan Z. Li,1995 S. Z. Li,On
discontinuity-adaptive smoothness priors in
computer vision, IEEE Trans. on Pattern and
Machine Intelligence vol.17,no.6, pp.576586,June
1995.
9
Formulation of 2-D DAS Constraint
bounded energy in each direction
where
direction weights, between 0 and 1
10
2-D DAS Direction Weights Calculation
11
An Example of Direction Weights
magnitude
Weights of the central Pixel is calculated
row
col.
Edge pixel
Weights in sixteen discrete directions
12
Implementation
A Interpolation initialization (bilinear, spline,
etc.)
B Candidate set propose
Pixel from low res. image
7x7 local window (16 discrete directions)
Pixel to be interpolated
Example pixel
D Iteratively, I. New candidate propose II.
Local energy change calculation. III.
Updating the interpolation results based on
energy difference.
C Local energy calculation I. under 2-D DAS
constraint. II. Using direction weights.
13
Implementation Monte Carlo Markov Chain search

• Two configurations, ?1 and ?2, are the same
  except for a single pixel (i, j).
• The global probability of each is
• p(?1 )exp-U(?1 )/T/Z and
  p(?2)exp-U(?2)/T/Z,
• then,
• p(?1)/p(?2) exp (U(?1)-U(?2))/T exp
  ?U/T.
• The updating rule is to accept the new state with
• probability Pcmin(1, p(?1)/p(?2)).
• Update the probability of the pixel candidates
  with Pc .

To calculate ?U, ?UE1(i, j)-E2(i, j)
14
Implementation
10 iterations
initial state
4 iterations
20 iterations
40 iterations
original
15
Interpolation Results
Original
Proposed, PSNR 34.1dB
NEDI, PSNR 33.8dB
Bicubic, PSNR 32.5dB
16
Interpolation Results Zoom-in Comparison
Original
Bicubic
Proposed
NEDI
17
Conclusions and Future Work
Conclusions

• MRF-based edge-directed image interpolation.
• MRF-MAP formulation of the spatial interpolation
  problem.
• Formulation of the 2D DAS constraint.
• Imposing the 2D DAS constraint to images via MRF
  model
• Applications in spatial interpolation.
• Obtaining sharp and consistent reconstructed
  edges

Future Work

• Other applications in content-adaptive
  post-processing.
• Stronger local content-adaptive property.

18

• End.
• Authors email m3li_at_ucsd.edu

19
Single Pass Implementation

• Iterations in optimization are removed.
• Major complexity is with the learning of the
  model parameters.
• Sliding window method.
• In addition, two other options to lower the
  complexity.
• Size-limited candidate set.
• Discrimination of edge and non-edge pixels.

Freeman, 2002 W. T. Freeman, et al.,
Example-based super-resolution, IEEE
Transaction on Computer Graphics and Application,
vol. 22, no. 2, pp. 5665, Apr., 2002.
20
Traditional Interpolation Methods

• Polynomial-based interpolation methods such as
  bilinear, bicubic and spline.
• Polynomial-based interpolation followed by edge
  sharpening or enhancements.
• Edge detection followed by edge-directed
  interpolation.
• Edge-directed interpolation based on local
  correlation formulation Li, 2001.

Challenges

• Its challenging to interpolate sharp and
  consistent edges.
• Its difficult to detect natural edges (position,
  thickness, etc).

Li, 2001 X. Li and M. T. Orchard, New
edge-directed interpolation, IEEE Trans. on
Image Processing, vol.10,no.10, pp.
15211526,2001.
21
Ideal and Natural Edges and Object Boundaries
Natural edges
Explicit edge detectors
Ideal step edges
? works with ideal step edges. ? has difficulties
with natural edges. (e.g. edge position and
thickness, corners and crossings)
22
Direction Weights
23
Implementation Model Parameters
24
2-D DAS Direction Weights Calculation
Choose window W
Calculate PIVs
Derive weights
Flow diagram
Take one direction (k, q) as an example to show
the calculation
1) Window W has adaptive size
2) PIVs are calculated according to
3) Weights are derived as
,
or
25
Implementation Model Parameters
T can be constant (in Metropolis method
Metropolis, 1953) or gradually decreases (in
Simulated Annealing method GemanGeman, 1984).
One updating equation of T can be
Metropolis, 1953 N. Metropolis, et al.,
Equation of state calculations by fast
computing machines, J. Chem. Phys.,vol. 21, pp.
10871092, 1953.