Signal- und Bildverarbeitung, 323.014 Image Analysis and Processing Arjan Kuijper 23.11.2006

1
Signal- und Bildverarbeitung, 323.014 Image
Analysis and ProcessingArjan Kuijper23.11.2006

• Johann Radon Institute for Computational and
  Applied Mathematics (RICAM) Austrian Academy of
  Sciences Altenbergerstraße 56A-4040 Linz,
  Austria
• arjan.kuijper_at_oeaw.ac.at

2
Last week

• The diffusion can be made locally adaptive to
  image structure. Three mathematical approaches
  are discussed
• PDE-based nonlinear diffusion, where the
  luminance function evolves as the divergence of
  some flow.
• Evolution of the isophotes as an example of
  curve-evolution
• Variational methods, minimizing an energy
  functional defined on the image.
• The nonlinear PDE's involve local image
  derivatives, and cannot be solved analytically.
• Adaptive smoothing requires geometric reasoning
  to define the influence on the diffusivity
  coefficient.
• The simplest equation is the equation proposed by
  Perona Malik, where the variable conduction is
  a function of the local edge strength. Strong
  gradient magnitudes prevent the blurring locally,
  the effect is edge preserving smoothing.
• The Perona Malik equation leads to deblurring
  (enhancing edges) for edges larger than the
  turnover point k, and blurs smaller edges.

3
Today

• Total Variation
• Rudin Osher Fatemi (ROF) Model
• Denoising
• Edge preserving
• Energy minimizing
• Bounded variationTaken from

4

• Let the observed intensity function u0(x, y)
  denote the pixel values of a noisy image for x,
  y? W. Let u(x, y) denote the desired clean image,
  sowith n additive white (0,?)noise.
• The constrained minimization problem is
  Min(HI1,I2) Min (H - l1I1 - l2I2)

5

• We arrive at the Euler-Lagrange equations 0 dH
  - l1dI1 - l2dI2
• Integrating the expression over W gives l1 0.
  So the average intensity is kept.

6

• The solution procedure uses a parabolic equation
  with time as an evolution parameter, or
  equivalently, the gradient descent method. This
  means that we solve

7

• We must compute l(t). We merely multiply the
  equation in W by (u - u0) and integrate by parts
  over W. We then have

8

• The numerical method in two spatial dimensions is
  as follows

9

• The numerical approximation is

10

• and ln is defined discreetly via
• A step size restriction is imposed for stability

11
Results
12
(No Transcript)
13

• Original
• Noisy
• Wiener
• TV

14

• Do it slightly sloppy
• demo

15

• Write
• Then I(u) is a constant of motion I(u0)I(ut)
• Then the pde converges to a minimum on the
  manifold given by the constraints

16
Some extensions

• A blurred noisy imageu0 (Au)(x,y)
  n(x,y)where A is a compact operator on L2.
• Multiplicative noise blurringu0 (Au)(x,y)
  n(x,y)u0 (Au)(x,y) u(x,y) n(x,y)
• Smarter functional

17
Recent Developments in Total Variation Image
Restoration

• T. Chan, S. Esedoglu, F. Park, A. YipHandbook of
  Mathematical Models in Computer VisionSince
  their introduction in a classic paper by Rudin,
  Osher and Fatemi, total variation minimizing
  models have become one of the most popular and
  successful methodology for image restoration.
  More recently, there has been a resurgence of
  interest and exciting new developments, some
  extending the applicabilities to inpainting,
  blind deconvolution and vector-valued images,
  while others offer improvements in better
  preservation of contrast, geometry and textures,
  in ameliorating the stair casing effect, and in
  exploiting the multi-scale nature of the models.
  In addition, new computational methods have
  been proposed with improved computational speed
  and robustness.

18
Properties

• Total variation based image restoration models
  were first introduced by Rudin, Osher, and Fatemi
  (ROF) in their pioneering work on edge preserving
  image denoising.
• It is one of the earliest and best known examples
  of PDE based edge preserving denoising.
• It was designed with the explicit goal of
  preserving sharp discontinuities (edges) in
  images while removing noise and other unwanted
  fine scale detail.
• Being convex, the ROF model is one of the
  simplest variational models having this most
  desirable property.
• The revolutionary aspect of this model is its
  regularization term that allows for
  discontinuities but at the same time disfavors
  oscillations.

19
Properties

• The constraint of the optimization forces the
  minimization to take place over images that are
  consistent with this known noise level.
• The objective functional itself is called the
  total variation (TV) of the function u(x) for
  smooth images it is equivalent to the L1 norm of
  the derivative, and hence is some measure of the
  amount of oscillation found in the function u(x).

20
Remark

• The step fromtois not trivial!

21
BV

• The space of functions with bounded variation
  (BV) is an ideal choice for minimizers to the ROF
  model since BV provides regularity of solutions
  but also allows sharp discontinuities (edges).
  Many other spaces like the Sobolev space W1,1 do
  not allow edges.
• Before defining the space BV, we formally state
  the definition of TV aswhere
  and is a bounded open set.
• We can now define the space BV as
• Thus, BV functions amount to L1 functions with
  bounded TV semi-norm.

22
TV Contours

• Why does this work?Ignoring the constraints we
  get
• The TV norm of f can be obtained by integrating
  along all contours of f c for all values of c.
  
• Thus, one can view TV as controlling both the
  size of the jumps in an image and the geometry of
  the isophotes (level sets).

23
Caveats

• While using TV-norm as regularization can reduce
  oscillations and regularize the geometry of level
  sets without penalizing discontinuities, it
  possesses some properties which may be
  undesirable under some circumstances.
• Loss of contrast The total variation of a
  function, defined on a bounded domain, is
  decreased if we re-scale it around its mean value
  in such a way that the difference between the
  maximum and minimum value (contrast) is reduced.
• Loss of geometry In addition to loss of
  contrast, the TV of a function may be decreased
  by reducing the length of each level set.
• Staircasing This refers to the phenomenon that
  the denoised image may look blocky (piecewise
  constant).
• Loss of Texture Although highly effective for
  denoising, the TV norm cannot preserve delicate
  small scale features like texture.

24
Summary

• Total variation minimizing models have become one
  of the most popular and successful methodology
  for image restoration.
• ROF is one of the earliest and best known
  examples of PDE based edge preserving denoising.
• It was designed with the explicit goal of
  preserving sharp discontinuities (edges) in
  images while removing noise and other unwanted
  fine scale detail.
• However, it has some drawbacks as shown in the
  previous slides

25
Next week

• Non-linear diffusionMean curvature motion
• Curve evolution
• Denoising
• Edge preserving
• Implementation
• Isophote vs. image implementation