Image Processing Basic Concepts

1
Image Processing Basic Concepts
???????? ?????
2
Contents

• Images
• sampling and resolution
• Manipulation
• Filtering, geometrical transformations
• compression
• GIF, JPEG, JPEG2000

3
Sampling and resolution

• An input device (e.g. camera or scanner) will
  sample (measure) the colours in a scene at a
  number of finite points on a 2D rectangle.
• Resolution can refer to the number of points
  sampled (e.g. 640 by 480) or the size of the dots
  (e.g. 300 dpi).
• The pixel-depth is related to the number of
  quantisation levels used for each colour, e.g.
  24-bit colour

4
Image Manipulation

• Why would we want to manipulate an image?
• Deficiencies in the image
• Focus blur, motion blur, red-eye, poor lighting,
  noise, ...
• Special effects required
• Sepia, painting styles, combining images, ...
• What methods are available?
• Pixel level processing
• Statistical processing
• Group of pixel processing
• Geometrical transformations

5
Image Manipulation

• Pixel level changes
• Brightness add an equal value to the R, G and B
  values of each pixel

6
Image Manipulation

• Pixel level changes
• Contrast Multiply the RGB values by some value
  and reset overall brightness

7
Image Manipulation

• Pixel level changes
• Colour balance Vary the R, G and B brightnesses
  independently

8
Image Manipulation

• Pixel level changes
• Colour manipulation
• Grey scale average RGB weighted to human
  perceptual system approx. 0.4R0.4G0.2B
• "Greying out" (e.g. disabled button) Blend
  pixel values with grey e.g. R' (R200)/2, G'
  (G200)/2, etc

9
Image Manipulation

• Statistical Processing
• Histogram equalisation (automatically adjust
  contrast)
• Create a histogram H with one bin for each grey
  scale allowed e.g. for G grey scales
• for each pixel (x, y), Himage(x,y)
• Create a cummulative histogram Hc
• Hc0H0
• for each grey scale g from 1 to G-1, Hcg
  Hcg-1Hc
• Hcg is now increasing and HcG-1 equals the
  number of pixels in the image
• Rescale Hc to the range of grey scales i.e.
• HcgG/(widthheight)
• Remap image grey scales
• for each pixel (x, y), image(x,y)
  Hcimage(x,y)

10
Image Manipulation

• Histogram equalisation
• example image with 90 pixels and 10 grey scales
  0-9
• histogram 0, 0, 0, 10, 20, 30, 20, 10, 0, 0
• Hc 0, 0, 0, 10, 30, 60, 80, 90, 90, 90
• Hc' 0, 0, 0, 1, 3, 6, 8, 9, 9, 9
• new histogram 0, 10, 0, 20, 0, 0, 30, 0, 20,
  10
• Pushes intensities apart
• dark pixels get darker
• light pixels get lighter

11
Image Manipulation

• Pixel group processing
• e.g. Convolution
• New pixel values are a weighted sum of
  neighbouring pixel's original values
• A filter specifies the weights in the sum
• Can often use separable 1-D filters for
  efficiency.
• filter(i,j)filter_x(i)filter_y(j)
• Different (positive and negative) filter
  coefficients (weights) have different effects
  e.g.
• Values like 1, 4, 6, 4, 1/16 will blur the
  pixels
• Values like -1, -3, 3, 1/8 will perform edge
  detection

12
Image Manipulation

• Pixel group processing changes
• Blurring
• Use a low-pass filter, e.g. 1,4,6,4,1/16
  applied along rows then columns.

13
Image Manipulation

• Pixel group processing changes
• Edge enhancement using unsharp masking.
• Subtracting the blurred version from the original
  leaves just the "edges"
• Add these edges back to the original to bring out
  the areas of contrast.

14
Image Manipulation

• Pixel group processing changes
• Edge detecting filters
• Filters such as 1, 3, -3, -1 can be applied
  either horizontally or vertically (usually after
  smoothing) to locate the intensity changes (edges)

Horizontal edges ( 127)
Vertical edges ( 127)
15
Image Manipulation

• Pixel level changes
• Combined effects
• e.g. embossed a original b (127edges at
  angle q)
• e.g. b, 3b, 127a, -3b , b

16
Image Manipulation

• Pixel level changes
• Art effects e.g. charcoal sketch (looks like an
  edge detector)
• Also paint strokes that perform local,
  directional blending of colours for pointillism
  etc..

17
Image Manipulation

• Geometrical transformations
• Map each pixel (x,y) to some other position
  (x',y')
• newImage(x,y) oldImage(x',y')
• Uses backward coordinate mapping, can you see
  why?
• would usually sample the from oldImage at
  non-integer position (x', y') using bilinear
  interpolation.
• Many simple effects e.g.
• Shearing x' xay, y' y
• Rotation x' xsin(T)ycos(T), y'
  xcos(T)-ysin(T)
• More complex effects
• Interpolate translations of points across image
• Free-form deformations, thin-plate splines etc

18
Image Manipulation

• Geometrical transformations
• Rotation x' xsin(T)ycos(T), y'
  xcos(T)-ysin(T)

19
Image Manipulation

• Geometrical transformations
• Using interpolated user-specified translations

20
Image Compression

• Image compression is required for storage and
  transmission
• Lossless compression methods
• No data is lost in the compression
• Suitable to all kinds of data e.g. text
• Lossy compression methods
• Data is thrown away in compression cycle
• Choose data which the human visual system is
  insensitive to.
• e.g. small high frequency components
• File formats GIF, JPEG, JPEG2000

21
Image Compression

• Repetition supression
• e.g. Run-length encoding
• aaaaaabbbbbbccddddd... a6b6ccd5...
• Statistical encoding
• e.g. Huffman encoding
• Use short binary strings for common characters
• Use longer binary strings for uncommon characters
• aaadaabbbaacaaabbaacaabaab 8bits each
  268208bits
• a 0 b 10, c 110 , d 111
• 000111001010100011000010100011000100010
  39bits

22
GIF image compression

• GIF images use a mixture
• Restricted colours (only 256 different colours)
• Run-length encoding
• Staistical encoding (LZW algorithm)
• Therefore GIF is lossless for images of less than
  256 colours
• i.e. they can be reconstructed exactly

23
Lossy Image Compression

• Human eye is fairly insensitive to certain kinds
  of image information
• Large objects generally more important than fine
  detail, textures etc
• Quite different to audio compression
• Intensity more important than hue
• Can quantise colours more coarsely

24
JPEG Image Compression

• Algorithm overview
• Transform and code each 8x8 block independently
• Perform Discrete Cosine Transform (DCT) on each
  block
• Differentially quantise block's DCT values
• Run length encode in zig-zag path
• Statistical encode resulting string

25
JPEG Image Compression

• The Discrete Cosine Transform (DCT)
• Separates the images high and low frequency
  components
• Related to the Fourier Transform
• The DCT itself is reversible i.e. lossless

26
JPEG Image Compression
A visual map of the DCT

• Each pixel in the DCT block is a weighted sum of
  the pixels in the input block
• This diagram shows which weights are applied for
  each pixel in the DCT.

27
Differential Quantisation

• Quantise higher frequency components with fewer
  levels
• Human eye is relatively insensitive to high
  frequency components
• This is where data is thrown away
• More coarsely quantised frequency components
  require fewer bits to store
• Varying the values in the quantisation matrix
  allows different compression levels
• Trade off quality for small file size.

28
Zig-zag encoding

• The lowest frequency component is at (0,0) and
  highest at (N,N)
• Use zig-zag path to encode block
• Use run-length encoding on resulting string
• tends to group 0 coefficients
• Use Huffman encoding on run-length encoded string

29
JPEG compression

• Additional steps
• Some additional steps are performed to squeeze
  more compression out of the data
• Colour
• the image is first converted to YUV colour space
  and the Y (luminance) is coded with higher
  quality than the 2 colour channels (U and V).
• The colour channels are also often "down sampled"
  i.e. reduced by a factor of 2 along rows and/or
  columns
• Predictive compression
• The first element in the DCT block is essentially
  the brightness of the block. These values are
  coded separately using predictive compression to
  remove redundancy between blocks

30
JPEG decompression

• Decode strings
• Reverse Huffman and run-length encoding
• zig-zag to reconstruct N by N block
• "Dequantise" block values
• e.g. if quantised to 4 levels and decoded to 256
  levels then multiply value by 64.
• Inverse DCT
• Very similar to DCT

31
JPEG 2000

• Completely different algorithm to standard JPEG
• Uses EZW compression
• Based on wavelet theory
• Doesn't involve blocking of the data like
  standard JPEG
• Blocking artefacts are common in over-compressed
  JPEG images.

32
JPEG 2000

• Wavelet transforms
• The Fourier Transform (FT) converts a signal (or
  image) into its component frequencies
• Looses spatial information e.g. doesn't tell us
  where the high frequencies are located in the
  image
• DCT similar to an FT applied to each block
• Retains some spatial information (i.e. the
  location of the block), but looses frequency
  correlations between blocks
• Wavelet Transform (WT) a (smooth) trade off
  between frequency and spatial representation

33
Wavelet transforms

• Filter horizontally with two filters
• even pixels low-pass filter
• odd pixels high-pass wavelet
• Group low pass filtered components to left and
  high pass filtered to right
• Repeat vertically
• Repeat recursively on low-pass image

...
34
Exploiting subband correlation

• Subband correlation
• Although the wavelet transform decorrelates image
  information within a subband, there is still a
  high degree of correlation between subbands.
• Early wavelet methods struggled to find a compact
  way to exploit this correlation

35
Exploiting subband correlation

• Zero-trees
• EZW encodes zeros, rather than the data.
• A zero at a coarse scale is a good predictor of
  zeros at a finer scale.
• Can encode a lot of information in specifying
  just the root of a zero-tree.
• Similar to RL encoding

36
Successive quantisation

• The chances of finding a zero tree increase with
  a coarser quantisation
• More small values are set to zero.
• Successive quantisation
• Quantise the (remaining) image data using
  successively finer quantisation steps.
• This leaves a binary image to encode at each step
• Values only 0 or 1
• Encode using zero trees
• Zero trees completely specify a binary image
• Successive zero-trees encode less significant bits

37
Successive quantisation

• Successive quantisation algorithm
• 1. Choose an high quantisation step Q (half the
  max value)
• 2. Until finest quantisation step, repeat
• 2.1 Quantise wavelet coefficients using Q
• ie. if (w(x,y)ltQ) w'(x,y)0 else w'(x,y)1
• Creates a binary image of the most significant
  wavelet coefficients
• 2.2 Encode binary image (w') using zero-trees and
  output
• (This is the highest order bits of the remaining
  data)
• 2.3 Subtract the encoded data from the remaining
  data
• w(x,y)w(x,y) w'(x,y)Q
• 2.4 half the quantisation step size
• QQ/2
• 3. End

38
JPEG 2000

• Hence the wavelet transform is encoded as a
  series of zero-trees representing successively
  less significant bits of the data
• The binary information is embedded in the
  zero-trees.
• Hence the name Embedded Zero-tree Wavelet (EZW)
  compression
• The actual system is actually more complex, but
  this gives the basic idea

39
Summary

• Sampling and resolution
• Image Manipulation
• pixel level, statistical and pixel group
• Image Compression
• GIF Run-length and LZW compression
• JPEG DCT, zig-zag encoding, differential
  quantisation
• JPEG 2000 wavelet transforms, zero-trees,
  successive quantisation.