Graph Spectral Image Smoothing

1
Graph Spectral Image Smoothing
GBR 2007
Fan Zhang, Edwin R. Hancock

• Computer Vision and Pattern Recognition Group
• Department of Computer Science
• University of York, UK

2
Overview

• Literature and Motivation
• Method
• Graph representation of an image
• Diffusion on a graph
• Numerical implementation
• Relationship to other methods (e.g. anisotropic
  diffusion, low-pass filtering, normalised cut)
• Experiments
• Conclusion

3
Literature

• General diffusion-based partial differential
  equation (PDE)
• If D1, isotropic linear diffusion (Gaussian
  filter).
• If D is a scalar function, inhomogeneous
  diffusion, e.g. Perona-Malik diffusion
• If D is a diffusion tensor, anisotropic
  diffusion,
• e.g. (Weickert,
  etc).

4
Motivation Why Graph Diffusion?

• Assumption for continuous PDEs
• An image is a continuous function on R2
• Consider discretisation for numerical
  implementation.
• Weakness
• Noisy images may not be sufficiently smooth to
  give reliable derivatives.
• Fast, accurate, and stable implementation is
  difficult to achieve.
• Solution diffusion on graphs
• An image is a smooth function on a graph.
• Purely combinatorial operators that require no
  discretisation are used.

5
Aim in this paper

• Explore if we can use graph spectral methods to
  solve the diffusion equation commencing from a
  discrete setting.

6
Steps

• Set-up diffusion process as problem involving
  weighted graph, where anisotropic smoothing is
  modelled by an edge-weight matrix.
• Diffusion is heat-flow on the associated
  graph-structure.
• Nodes are pixels, and diffusion of grey-scale
  information is along edges with time.
• Solution is given by exponentiating the spectrum
  of the associated Laplacian matrix.

7
Graph Representation of Images

• An image is represented using a weighted graph
  .
• The nodes V are the pixels. An edge
  is formed between two nodes vi and vj . The
  weight of an edge, , is denoted by
  .

8
Graph Edge Weight

• Characterise each pixel by a window
  of neighbors instead of using a single pixel
  alone.
• The similarity between nodes vi and vj is
  measured by the Gaussian weighted Euclidean
  distance between windows, i.e.
• Thus, edge weight is computed by

9
Laplacian of a graph

• Weighted graph G (V, E,W) with node-set V,
  edge-set E, adjacency weight matrix W and degree
  matrix D.
• Diagonal degree matrix
• Laplacian
• Normalised Laplacian

10
Laplacian spectrum

• Spectral decomposition of Laplacian

Eigenvalues
Eigenvectors
11
Graph Heat Kernel

• Heat equation on graph
• Solution of heat equation is heat kernel
• When t tends to 0,
• When t large,

12
Graph Heat Kernel

• Heat equation on graph
• Solution of heat equation is heat kernel
• When t tends to 0,
• When t large,

13
Graph Heat Kernel

• Heat equation on graph
• Solution of heat equation is heat kernel
• When t tends to 0,
• When t large,

14
Graph Heat Kernel

• Heat equation on graph
• Solution of heat equation is heat kernel
• When t tends to 0,
• When t large,

15
Lazy random walk on graph

• Moves between nodes with probability ,
  remains static with probability
• Transition probability matrix
• Let , then after N-steps of walk

16
Lazy random walk on graph

• Moves between nodes with probability ,
  remains static with probability
• Transition probability matrix
• Let , then after N-steps of walk

17
Continuous time random walk

• Let we the state probability vector at time
  t. Probability of visiting ith node after time t
  is
• State-vector is solution of the differential
  equation
• Solution given by heat-kernel

18
Continuous time random walk

• Let we the state probability vector at time
  t. Probability of visiting ith node after time t
  is
• State-vector is solution of the differential
  equation
• Solution given by heat-kernel

19
we are going to use the random walk to model
image smoothing via anisotropic
diffusion. Transition probability is small when
there is strong evidence of edge-structure, and
small in uniform regions. Heat-kernel is used to
smooth pixel values, and this means pixel values
are related to state probabilities of random walk.
20
Anisotropic diffusion as heat flow on a graph

• Vertices pixel values
• Edge weight c.f. thermal conductivity
• Diffusion-equation for pixel values

Pixel values stored as vector of stacked image
columns. Connectivity structure encoded by
Laplacian.
21
Graph spectral image smoothing

• Solution of diffusion equation
• Small t behaviour

22
Graph spectral image smoothing

• Solution of diffusion equation
• Small t behaviour

Heat kernel weights the averaging of grey-scale
values
23
Graph spectral image smoothing

• Solution of diffusion equation
• Small t behaviour

Effect of original pixel value decreases with
time. Weighted average of neighbouring values.
24
Meaning
What does this mean? - heat kernel weights
initial pixel values and smooths image. -
pixel values are strongly influenced by
neighbours when they are connected by edges of
large weight (small difference in grey-scale
value). - time plays the role of scale,
the longer the process is run the greater the
smoothing.
25
Numerical Implementation

• Laplacian L is very large, e.g.
  .
• It is not tractable to calculate the heat kernel
  by matrix exponential.
• Solution Krylov subspace projection technique
• Idea approximate by
  an element of Krylov space
• Approximation scheme
• first column of identity
  matrix
• orthonormal basis of Krylov space
• Hessenberg matrix resulting from Lanczos
  process

26
Relation to Anisotropic Diffusion

• An image is a 2D manifold M embedded in R3, i.e.

27
Relation to Signal Processing

• An extension of Fourier analysis to images
  defined on graphs.
• Decomposition of the image into a linear
  combination of eigenvectors
• Attenuate the terms associating with high
  eigenvalues. Thus, Graph heat kernel can be
  regarded as a low-pass filter kernel.

28
Relation to Spectral Clustering

• Large t behavior of the heat kernel is governed
  by the second eigenvector of the graph Laplacian
  
  Normalised cut (Shi and Malik).
• The algorithm projects the noisy image onto the
  space spanned by the first few eigenvectors.
  Laplacian eigenmap (Belkin and Niyogi).
• The second or first few eigenvectors of the
  Laplacian encode the segment-structure of the
  image.

29
Results
It takes 36 seconds to process a 256 square
image.
30
(a) Noisy Lenna (b) Zoomed portion (c) Our graph
smoothing (d)
Regularised Perona-Malik diffusion (RPM) (e)
Nonlinear complex ramp-preserving diffusion
(NCRD) (f) Coherence-enhancing diffusion (CED)
(g) Total-variation denoising (TV) (h) Wavelet
filtering (WAVELET)
31
(No Transcript)
32
Root-Mean-Square Error comparison
33
Root-Mean-Square Error comparison
34
(No Transcript)
35
(No Transcript)
36
Conclusion

• Graph representation of images is a natural,
  discrete and effective way for image processing.
• Diffusion on graphs can be efficiently used for
  image smoothing. The diffusion is determined by
  spectra of the graphs.
• Graph smoothing can be readily solved using
  Krylov subspace technique.
• Graph-spectral smoothing has close relationships
  with the continuous anisotropic diffusion,
  low-pass filtering and spectral clustering.