Color and Brightness Constancy

1
Color and Brightness Constancy

• Jim Rehg
• CS 4495/7495 Computer Vision
• Lecture 25 26
• Wed Oct 18, 2002

2
Outline

• Human color inference
• Lands Retinex
• Dichromatic reflectance model
• Finite dimensional linear models
• Color constancy algorithm

3
Human Color Constancy

• Distinguish between
• Color constancy, which refers to hue and
  saturation
• Lightness constancy, which refers to gray-level.
• Humans can perceive
• Color a surface would have under white light
  (surface color)
• Color of the reflected light (limited ability to
  separate surface color from measured color)
• Color of illuminant (even more limited)

4
Spatial Arrangement and Color Perception
5
Spatial Arrangement and Color Perception
6
Spatial Arrangement and Color Perception
7
Lands Mondrian Experiments

• The (by-now) familiar phenomena Squares of color
  with the same color radiance yield very different
  color perceptions

Photometer 1.0, 0.3, 0.3
Photometer 1.0, 0.3, 0.3
Colored light
White light
Audience Red
Audience Blue
8
Basic Model for Lightness Constancy

• Modeling assumptions for camera
• Planar frontal scene
• Lambertian reflectance
• Linear camera response
• Camera model
• Modeling assumptions for scene
• Albedo is piecewise constant
• Exception ripening fruit
• Illumination is slowly-varying
• Exception shadow boundaries

9
Algorithm Components

• The goal is to determine what the surfaces in the
  image would look like under white light.
• A process that compares the brightness of patchs
  across their common boundaries and computes
  relative brightness.
• A process that establishes an absolute reference
  for lightness (e.g. brightest point is white)

10
1-D Lightness Retinex
Threshold gradient image to find surface (patch)
boundaries
11
1-D Lightness Retinex
Integration to recover surface lightness (unknown
constant)
12
Extension to 2-D

• Spatial issues
• Integration becomes much harder
• Integrate along many sample paths (random walk)
• Loopy propagation
• Recover of absolute lightness/color reference
• Brightest patch is white
• Average reflectance across scene is known
• Gamut is known
• Specularities can be detected
• Known reference (color chart, skin color, etc.)

13
Color Retinex
Images courtesy John McCann
14
Finding Specularities

• Dielectric materials
• Specularly reflected light has the color of the
  source
• Reflected light has two components, we see their
  sum
• Diffuse (body reflection)
• Specular (highlight)
• Specularities produce a Skewed-T in the color
  histogram of the object.

15
Skewed-T in Histogram

• A Physical Approach to Color Image Understanding
  Klinker, Shafer, and Kanade. IJCV 1990

16
Skewed-T in Histogram
17
Recent Application to Stereo
Synthetic scene
Motion of camera causes highlight location to
change. This cue can be combined with histogram
analysis.
18
Recent Application to Stereo
Real scene
19
Finite Dimensional Linear Models
20
Obtaining the illuminant from specularities

• Assume that a specularity has been identified,
  and material is dielectric.
• Then in the specularity, we have

• Assuming
• we know the sensitivities and the illuminant
  basis functions
• there are no more illuminant basis functions than
  receptors
• This linear system yields the illuminant
  coefficients.

21
Obtaining the illuminant from average color
assumptions

• Assume the spatial average reflectance is known
• We can measure the spatial average of the
  receptor response to get

• Assuming
• g_ijk are known
• average reflectance is known
• there are not more receptor types than illuminant
  basis functions
• We can recover the illuminant coefficients from
  this linear system

22
Normalizing the Gamut

• The gamut (collection of all pixel values in
  image) contains information about the light
  source
• It is usually impossible to obtain extreme color
  readings (255,0,0) under white light
• The convex hull of the gamut constrains
  illuminant
• Gamut mapping algorithm (Forsyth 90)
• Obtain convex hull W of pixels under white light
• Obtain convex hull G of input image
• The mapping M(G) must have property