Multiscale Analysis of Images

1
Multiscale Analysis of Images

• Gilad Lerman
• Math 5467
• (stealing slides from
• Gonzalez Woods, and Efros)

2
The Multiscale Nature of Images
3
Recall Gaussian pre-filtering
G 1/8
G 1/4
Gaussian 1/2

• Solution filter the image, then subsample
• Filter size should double for each ½ size
  reduction.

4
Subsampling with Gaussian pre-filtering
G 1/4
G 1/8
Gaussian 1/2

• Solution filter the image, then subsample
• Filter size should double for each ½ size
  reduction.

5
Image Pyramids

• Known as a Gaussian Pyramid Burt and Adelson,
  1983
• In computer graphics, a mip map Williams, 1983
• A precursor to wavelet transform

6
A bar in the big images is a hair on the zebras
nose in smaller images, a stripe in the
smallest, the animals nose
Figure from David Forsyth
7
Gaussian pyramid construction
filter mask

• Repeat
• Filter
• Subsample
• Until minimum resolution reached
• can specify desired number of levels (e.g.,
  3-level pyramid)
• The whole pyramid is only 4/3 the size of the
  original image!

8
What does blurring take away? (recall)
original
9
What does blurring take away? (recall)
smoothed (5x5 Gaussian)
10
High-Pass filter
smoothed original
11
Band-pass filtering
Gaussian Pyramid (low-pass images)

• Laplacian Pyramid (subband images)
• Created from Gaussian pyramid by subtraction

12
Laplacian Pyramid
Need this!
Original image

• How can we reconstruct (collapse) this pyramid
  into the original image?

13
(No Transcript)
14
(No Transcript)
15
Image resampling (interpolation)

• So far, we considered only power-of-two
  subsampling
• What about arbitrary scale reduction?
• How can we increase the size of the image?

d 1 in this example
1
2
3
4
5

• Recall how a digital image is formed
• It is a discrete point-sampling of a continuous
  function
• If we could somehow reconstruct the original
  function, any new image could be generated, at
  any resolution and scale

16
Image resampling

• So far, we considered only power-of-two
  subsampling
• What about arbitrary scale reduction?
• How can we increase the size of the image?

d 1 in this example
1
2
3
4
5

• Recall how a digital image is formed
• It is a discrete point-sampling of a continuous
  function
• If we could somehow reconstruct the original
  function, any new image could be generated, at
  any resolution and scale

17
Image resampling

• So what to do if we dont know

18
Resampling filters

• What does the 2D version of this hat function
  look like?

performs linear interpolation
(tent function) performs bilinear interpolation

• Better filters give better resampled images
• Bicubic is common choice
• Why not use a Gaussian?
• What if we dont want whole f, but just one
  sample?

19
Bilinear interpolation

• Smapling at f(x,y)

20
Applications (more efficient with wavelets)

• Coding
• High frequency coefficients need fewer bits, so
  one can
• collapse a compressed Laplacian pyramid
• Denoising/Restoration
• Setting most coefficients in Laplacian
  pyramid to zero
• Image Blending

21
Application Pyramid Blending
Left pyramid
Right pyramid
blend
22
Image Blending
23
Feathering
Ileft
Iright
?right
?left
Encoding transparency I(x,y) (aR, aG, aB, a)
Iblend ?left Ileft ?right Iright
24
Affect of Window Size
left
right
25
Affect of Window Size
26
Good Window Size
Optimal Window smooth but not ghosted Burt
and Adelson (83) Choose by pyramids
27
Pyramid Blending
28
Pyramid Blending (Color)
29
Laplacian Pyramid Blending

• General Approach
• Build Laplacian pyramids LA and LB from images A
  and B
• Build a Gaussian pyramid GR from selected region
  R (black white corresponding images)
• Form a combined pyramid LS from LA and LB using
  nodes of GR as weights
• LS(i,j) GR(I,j,)LA(I,j) (1-GR(I,j))LB(I,j)
• Collapse the LS pyramid to get the final blended
  image

30
laplacian level 0
left pyramid
right pyramid
blended pyramid
31
Blending Regions
32
Simplification Two-band Blending

• Brown Lowe, 2003
• Only use two bands high freq. and low freq.
• Blends low freq. smoothly
• Blend high freq. with no smoothing use binary
  mask

Can be explored in a project