Multiscale Analysis of Images

1
Multiscale Analysis of Images

• Gilad Lerman
• Math 5467
• (stealing slides from
• Gonzalez Woods, and Efros)

2
The Multiscale Nature of Images
3
Recall Gaussian pre-filtering
G 1/8
G 1/4
Gaussian 1/2

• Solution filter the image, then subsample
• Filter size should double for each ½ size
  reduction.

4
Subsampling with Gaussian pre-filtering
G 1/4
G 1/8
Gaussian 1/2

• Solution filter the image, then subsample
• Filter size should double for each ½ size
  reduction.

5
Image Pyramids

• Known as a Gaussian Pyramid Burt and Adelson,
  1983
• In computer graphics, a mip map Williams, 1983
• A precursor to wavelet transform

6
A bar in the big images is a hair on the zebras
nose in smaller images, a stripe in the
smallest, the animals nose
Figure from David Forsyth
7
Gaussian pyramid construction
filter mask

• Repeat
• Filter
• Subsample
• Until minimum resolution reached
• can specify desired number of levels (e.g.,
  3-level pyramid)
• The whole pyramid is only 4/3 the size of the
  original image!

8
What does blurring take away? (recall)
original
9
What does blurring take away? (recall)
smoothed (5x5 Gaussian)
10
High-Pass filter
smoothed original
11
Band-pass filtering
Gaussian Pyramid (low-pass images)

• Laplacian Pyramid (subband images)
• Created from Gaussian pyramid by subtraction

12
Laplacian Pyramid
Need this!
Original image

• How can we reconstruct (collapse) this pyramid
  into the original image?

13
(No Transcript)
14
(No Transcript)
15
Image resampling (interpolation)

• So far, we considered only power-of-two
  subsampling
• What about arbitrary scale reduction?
• How can we increase the size of the image?

d 1 in this example
1
2
3
4
5

• Recall how a digital image is formed
• It is a discrete point-sampling of a continuous
  function
• If we could somehow reconstruct the original
  function, any new image could be generated, at
  any resolution and scale

16
Image resampling

• So far, we considered only power-of-two
  subsampling
• What about arbitrary scale reduction?
• How can we increase the size of the image?

d 1 in this example
1
2
3
4
5

• Recall how a digital image is formed
• It is a discrete point-sampling of a continuous
  function
• If we could somehow reconstruct the original
  function, any new image could be generated, at
  any resolution and scale

17
Image resampling

• So what to do if we dont know

18
Resampling filters

• What does the 2D version of this hat function
  look like?

performs linear interpolation
(tent function) performs bilinear interpolation

• Better filters give better resampled images
• Bicubic is common choice
• Why not use a Gaussian?
• What if we dont want whole f, but just one
  sample?

19
Bilinear interpolation

• Smapling at f(x,y)

20
What happens in the frequency domain?

• Laplacian Pyramid gives rise to different
  frequency bands
• Alternatively can obtain similar pyramids
  according to frequency bands

21
Example of Frequency-bands pyramid

• Taken from Rajashekar and Simoncelli
  (http//www.cns.nyu.edu/ftp/lcv/rajashekar08a.pdf)

22
Applications (more efficient with wavelets)

• Coding
• High frequency coefficients need fewer bits, so
  one can
• collapse a compressed Laplacian pyramid
• Denoising/Restoration
• Setting most coefficients in Laplacian
  pyramid to zero
• Image Blending

23
Denoising demonstration (motivation)
Observation spatial correlation in
noise-free image (and not the noisy
image) Conclusion Noise is more evident in
the high-frequency components, than the
low- frequency components (smooth components)

• Taken from Rajashekar and Simoncelli
  (http//www.cns.nyu.edu/ftp/lcv/rajashekar08a.pdf)

24
Denoising demonstration (idea)
Partition into sub-bands of frequencies (top
high freq., bottom low) y-given image
(vectorized), - estimated denoised image
(vectorized) The middle graph intensity
thresholding. No thresholding for smoothest
component and keeping only higher intensities
for higher components (the filter is learned
empirically) In practice, can have blurring and
not competitive with NLM, BM3d, dictionary
learning

• Taken from Rajashekar and Simoncelli
  (http//www.cns.nyu.edu/ftp/lcv/rajashekar08a.pdf)

25
Application Pyramid Blending
Left pyramid
Right pyramid
blend
26
Image Blending
27
Feathering
Ileft
Iright
?right
?left
Encoding transparency I(x,y) (aR, aG, aB, a)
Iblend ?left Ileft ?right Iright
28
Affect of Window Size
left
right
29
Affect of Window Size
30
Good Window Size
Optimal Window smooth but not ghosted Burt
and Adelson (83) Choose by pyramids
31
Pyramid Blending
32
Pyramid Blending (Color)
33
Laplacian Pyramid Blending

• General Approach
• Build Laplacian pyramids LA and LB from images A
  and B
• Build a Gaussian pyramid GR from selected region
  R (black white corresponding images)
• Form a combined pyramid LS from LA and LB using
  nodes of GR as weights
• LS(i,j) GR(I,j,)LA(I,j) (1-GR(I,j))LB(I,j)
• Collapse the LS pyramid to get the final blended
  image

34
laplacian level 0
left pyramid
right pyramid
blended pyramid
35
Blending Regions
36
Simplification Two-band Blending

• Brown Lowe, 2003
• Only use two bands high freq. and low freq.
• Blends low freq. smoothly
• Blend high freq. with no smoothing use binary
  mask

Can be explored in a project