Colour Image Processing

1
Colour Image Processing

• Allan Hanbury
• PRIP, Vienna University of TechnologyFavoritenstr
  aße 9/1832A-1040 Vienna, Austriahanbury_at_prip.tuw
  ien.ac.athttp//www.prip.tuwien.ac.at/hanbury

2
Contents

• Introduction
• Processing Vectorial Images
• Alternative Colour Spaces
• 3D-polar Coordinate Colour Spaces
• Processing and Analysing Colour Images

3
Contents

• Introduction
• Processing Vectorial Images
• Alternative Colour Spaces
• 3D-polar Coordinate Colour Spaces
• Processing and Analysing Colour Images

4
Physical Background

• Visible light a narrow band of electromagnetic
  radiation ? 380nm (blue) - 780nm (red)
• Wavelength Each physically distinct colour
  corresponds to at least one wavelength in this
  band.
• Pure Colours Pure or monochromatic colours do
  not exist in nature.

5

• Spectrum Intensity asa function of wavelength.
• The colour of an object is the product of the
  spectrum of the incident light with the light
  absorption and/or reflection properties of the
  object.

From http//fuse.pha.jhu.edu/wpb/spectroscopy/bas
ics.html
6
Human colour perception

• The human eye does not perceive individual light
  wavelengths.
• It contains three types of colour receptor
  (cones) which integrate over parts of the
  spectrum

From http//math.ucr.edu/home/baez/physics/General
/BlueSky/blue_sky.html
7

• It is therefore possible to characterise a
  psycho-visual colour by specifying the amounts of
  three primary colours red, green and blue, mixed
  together.
• This leads to the standard RGB space used in
  television, computer monitors, etc.
• We specify the levels of R, G and B in the range
  0, 1, but they can easily be extended to other
  ranges (8-bit integers for example).

8
Problems with Processing Colour Images

• When processing colour images, the following
  problems (amongst others) have to be dealt with
• The images are vectorial ? 3 numbers are
  associated with each pixel.
• The colours recorded by a camera are heavily
  dependent on the lighting conditions.

9
Lighting conditions

• The lighting conditions of the scene have a large
  effect on the colours recorded.

Image taken lit by a flash.
Image taken lit by a tungsten lamp.
10

• The following four images of the same scene were
  acquired under different lighting conditions

11
Dealing with Lighting Changes

• Knowing just the RGB values is not enough to know
  everything about the image.
• The R, G and B primaries used by different
  devices are usually different.
• For scientific work, the camera and lighting
  should be calibrated.
• For multimedia applications, this is more
  difficult to organise
• Algorithms exist for estimating the illumination
  colour.

12
Contents

• Introduction
• Processing Vectorial Images
• Alternative Colour Spaces
• 3D-polar Coordinate Colour Spaces
• Processing and Analysing Colour Images

13
Processing Vectorial Images

• A vectorial image has a vector at each pixel. For
  colour images, these vectors each have 3
  components.
• Vectorial images with larger numbers of
  components also exist, e.g. in satellite imagery.
• There are two ways one can process vectorial
  images
• Marginal processing.
• Vectorial processing.

Greyscale
Colour
f (x, y) 0,1,, N
f (x, y) 0,,N, 0,,N, 0,,N
14

Marginal Processing

• Each channel is processed separately

15
Vectorial Processing

• The colour triplets are processed as single units

16
The Problem of False Colours
False Colours !!
17
The Problem of False Colours
18
Contents

• Introduction
• Processing Vectorial Images
• Alternative Colour Spaces
• 3D-polar Coordinate Colour Spaces
• Processing and Analysing Colour Images

19
Alternative Colour Spaces

• Various other colour representations can be
  calculated from the RGB representation.
• This can be done for
• Decorrelating the colour channels
• principal components.
• Bringing colour information to the fore
• Hue, saturation and brightness.
• Perceptual uniformity
• CIELuv, CIELab,

20
Processing Strategy
21
Colour spaces

• RGB (CIE), RnGnBn (TV - National Television
  Standard Comittee)
• XYZ (CIE)
• UVW (UCS de la CIE), UVW (UCS modified by the
  CIE)
• YUV, YIQ, YCbCr
• YDbDr
• DSH, HSV, HLS, IHS
• Munsel colour space (cylindrical representation)
• CIELuv
• CIELab
• SMPTE-C RGB
• YES (Xerox)
• Kodak Photo CD, YCC, YPbPr, ...

22
Contents

• Introduction
• Processing Vectorial Images
• Alternative Colour Spaces
• 3D-polar Coordinate Colour Spaces
• Processing and Analysing Colour Images

23
3D-polar Coordinate Colour Spaces

• These spaces use a cylindrical (3D-polar)
  coordinate system to encode the following three
  psycho-visual coordinates
• Hue (dominant colour seen)
• Wavelength of the pure colour observed in the
  signal.
• Distinguishes red, yellow, green, etc.
• More the 400 hues can be seen by the human eye.
• Saturation (degree of dilution)
• Inverse of the quantity of white present in the
  signal. A pure colour has 100 saturation, the
  white and grey have 0 saturation.
• Distinguishes red from pink, marine blue from
  royal blue, etc.
• About 20 saturation levels are visible per hue.
• Brightness
• Amount of light emitted.
• Distinguishes the greylevels.
• The human eye perceives about 100 levels.

24
3D-polar Coordinate Colour Spaces

• The transformation of the RGB colour space to a
  hue, saturation and brightness colour space is
  essentially a conversion from a set of
  rectangular coordinates to a set of (3D-polar)
  cylindrical coordinates.

H
25
Idea Behind the Transformation

• In the RGB space, the vectors R, G, B specify
  the amount of each red, green and blue primary in
  the colour.
• For convenience, we take R, G, B ? 0, 1
• The valid coordinates form the RGB cube 0, 1 ?
  0, 1 ? 0, 1.
• The basic idea behind the transformation to the
  hue, saturation and brightness coordinate system
  is to place a new axis between 0, 0, 0 and 1,
  1, 1, and to specify the colours in 3D-polar
  coordinates based on this axis.
• This new axis passes through all the grey points
  (R G B), so we call it the achromatic axis.

26
Basic Problem
Achromatic axis

• Many of the spaces were originally developed for
  easy numerical specification of colours in
  computer graphics applications.
• Due to its brightness function, the natural
  shape of the HSV colour gamut is a cone.
• However, with this shape, there are many
  coordinates which are not valid.
• In order to avoid costly verification of the
  validity of specified coordinates, these gamuts
  were often artificially expanded into cylinders.
• These cylindrical spaces have often been
  unwittingly carried over into image processing
  applications.

Conic HSV
H0
H180
Cylindrical HSV
27
Standard RGB ? HSV Transform

• max sup(R, G, B) min inf(R, G, B)
• L max
• If Ht lt 0, Ht Ht 6
• H Ht ? 60

28
Conic HLS

• The same problem occurs for the HLS
  transformation.
• Here the double cone has been artificially
  expanded at both ends.

H0
H180
Cylindrical HLS
29
Standard RGB ? HLS Transform

• max sup(R, G, B) min inf(R, G, B)
• If Ht lt 0, Ht Ht 6
• H Ht ? 60

R, G and B are between 0 and 1.
30
Removal of the Brightness Dependence of the
Saturation (1)

• HSV model
• But the HSV brightness LHSV max(R, G, B)
• So the saturation with the brightness-dependence
  removed is

31
Removal of the Brightness Dependence of the
Saturation (2)

• HLS model
• where the brightness

32

• The brightness dependence is removed by using
• to give

33

• Consider the colour image below. Not all the
  pixels which appear white have RGB coordinates of
  exactly 1, 1, 1 (similar for black pixels).

Le chanteur, Joan Mirò (bottom half inverted)
34

• The advantage of this measure of saturation is
  visible on the example image.

Le chanteur, Joan Mirò (bottom half inverted)
Proposed Saturation
35
Advantages of the IHLS space

• This is the Improved HLS space.
• It has the following advantages
• The saturation is low for black and white pixels.
• The brightness is independent of the saturation
  (can be shown mathematically). This means that
  you are free to choose any brightness, luminance
  or lightness function.
• The saturation values can be easily compared.
  This is important for mathematical morphology
  operators.

36
Transformation into the IHLS space

• The full transformation is implemented
  efficiently as

Or your own favourite brightness expression.
Trigonometric version is more accurate.
37
Contents

• Introduction
• Processing Vectorial Images
• Alternative Colour Spaces
• 3D-polar Coordinate Colour Spaces
• Processing and Analysing Colour Images

38
Processing and Analysing Colour Images

• Some applications which take advantage of the
  good properties of the IHLS space are presented.
• The following are discussed
• Hue statistics.
• Mathematical morphology on colour images.

39
Basic Idea
40
Processing the Hue Component
Colour image The virgin, P. Serra (16th
century)
41
Hue Statistics

• For the brightness and the saturation, one can
  use standard statistical methods for calculating
  the mean, standard deviation, etc.
• For data ai (i 1, 2, , n) distributed on the
  unit circle, the mean direction is that of the
  resultant vector obtained by adding unit vectors
  with directions ai.
• A measure of the variation in the directions of
  the data is given by the length of this vector
  divided by n (the mean length), which has the
  following characteristics
• range 01
• Values close to 1 ? the data is less spread out.

42
Mean

• Given n values of the hue Hi
• The mean direction H is calculated as follows
  
  (1)
  
  (2)
• The mean length R is
  
  (3)

43
Hue Mean which Takes the Saturation Into Account

• The previous formulation is standard in the texts
  on circular statistics, but it ignores the fact
  that not all hues have the same importance.
• We take this into account by weighting the length
  of each hue vector by the associated saturation
  value.

n 3
44

• Let Si be the saturation associated with hue Hi.
• We replace equation (1) by
• To calculate the saturation-weighted mean
  direction HS, we replace A and B by AS and BS
  in equation (2).
• Equation (3) becomes
• Note that RS remains a measure of the angular
  dispersion, and does not give information on the
  mean of the saturation.

45
Example
Colour Image
Hue histogram
Hue
Saturation histogram
Saturation
46
Colour statistics histograms

• Histograms are a useful tool for greyscale image
  analysis.
• We suggest a similar tool for use with colour
  images.
• Calculated as follows
• The luminance is quantised into N1 levels
• For each level l 0, 1,, N, we calculate
• The saturation-weighted hue mean.
• The associated mean length.
• We have two histograms as a function of
  luminance.
• For visualisation, we create one histogram by
• Setting the heights of the bars equal to the mean
  length.
• Setting the colour of the bars based on the hue
  mean.

47
Example (1)
Colour Image
Hue
Saturation
Luminance
48
Example (2)
Colour Image
Hue
Saturation
Luminance
49
Mathematical Morphology in a 3D-Polar Coordinate
Colour Space

• One can easily order the brightness and
  saturation values.
• On the other hand, the hue is defined on the
  circle, for which there is no obvious order
  (blue larger than red?, green smaller than red?
  ).
• Applying morphological operators to the
  brightness and saturation is therefore easy.
• We just have to be sure that we order the vectors
  and not the components (i.e. marginal order), so
  as to avoid introducing false colours.
• For this purpose we use the lexicographical
  order.
• Applying mathematical morphology to the hue is
  trickier, and wont be discussed in this talk.

1
1
0
0
L
S
H
0
50
Lexicographical Order

• This is the order in which words are arranged in
  a dictionary.

Aardvark Abacus Abandon . . . Borough Bough . .
51
Lexicographical Order for Vectors

• For example, given two vectors x (x1, x2, x3)
  and y (y1, y2, y3)
• The lexicographical order is a total vector
  order. This means
• There are no pairs of vectors for which the order
  is unknown.
• The maximum and minimum of a set of vectors is
  always part of the set ? no false colours!
• The disadvantage is that one of the vector
  components has to play a dominant role.

52
Angular Distance
a ? a?

• To enable us to work on the circle, we begin by
  defining an angular distance.
• Given a circle C with centre o
• We choose an arbitrary origin a0 on the circle.
  The points ai are then described by their
  curvilinear coordinate between 0 and 2p starting
  at a0.
• Given two points a and a?, the size of the acute
  angle aoa? is

a?
a
o
53
Lexicographical Order with Brightness at the Top
Level (1)

• We define
• and

54
Lexicographical Order with Brightness at the Top
Level (2)

• H0 is an arbitrary parameter. It does not have a
  major effect on the results, as, being at the
  third level, it is almost never taken into
  account.
• The erosion is defined in the standard way as
  follows e f (x) f (y) f (y) inf f (z)
  , z ? Bx and the dilation d f (x) f (y)
  f (y) sup f (z) , z ? Bx

55
Example
Brightness at the top level
Original Image (293 ? 418)
SE Square of size 5 ? 5
56
Lexicographical Order with Saturation at the Top
Level

• We define
• and

57
Example
Saturation at the top level
Original image (293 ? 418)
SE Square of size 5 ? 5
58
Colour Top-Hat

• We can define a top-hat operator for colour
  images similar to the one used for greyscale
  images.
• We simply calculate the Euclidean distance DE
  between each corresponding pixel of the original
  image I and the result of one of the colour
  opening g or closing ? operators
• Opening top-hat
• Closing top-hat

59
Example
We wish to extract the greyish lines in between
the mosaic tiles
60
Summary

• A 3D-polar coordinate representation of the RGB
  colour space can be very useful in image
  processing and analysis.
• One should be careful with how the saturation is
  defined.
• Applications demonstrated are
• Colour statistics.
• Colour morphology.
• Further applications include
• Colour histograms.
• 2-dimensional brightness-saturation histograms
  (have been used for segmentation).