Title: Color constancy at a pixel [Finlayson et al. CIC8, 2000]
1(No Transcript)
2Color constancy at a pixel Finlayson et al.
CIC8, 2000
Idea plot log(R/G) vs. log(B/G)
3Log(R/G)
Log(B/G)
Log(B/G)
Log(R/G)
For every patch, the direction from light color
change is about the same!
4Why all linear and same direction?
The image formation equation
color
shading
intensity
light SPD
k1..3
reflectance
sensor
Now lets make some assumptions
5Assumption 1 Light is Planckian (or some other
1D assumption)
Wiens approximation of a Planckian source
Note 1D parameter T temperature light
color.
P100
6Assumption 2 Narrow band sensors
The Sony Camera has fairly narrow band
sensitivities
Using spectral sharpening, we can make almost
all sensor sets have this property. Finlayson,
Drew, Funt
SONY DXC-930
7Modified Image Formation
The kth response
Substituting Narrow-band and Planckian Assumptions
Take logs
Response light intensity surface
light color
8Implications
We have k equations of the form
is common to all equations and can be removed
by simple differencing at this pixel
This results in k-1 independent equations of the
form
light color term
reflectance term
9 Implications
(1) If there are 3 sensors we have two
independent equations of this form
(2) For a single surface viewed under different
colored lights the log chromaticities must fall
on a line
(3) Different surfaces induce lines with the
same orientation
The log chromaticities of 7 surfaces viewed under
10 lights
10 One degree of freedom is invariantto light
change
Luminance
1D invariant
Project to 1D
Gray?
11More formally
form ratios
and define
define vectors
?
line in 2D ?
12What is this good for?
With certain restrictions, from a 3-band color
image we can derive a 1-D grayscale image which
is - illuminant invariant - and so, shadow free
13Then use edge info. to integrate back without
shadows ECCV02 Finlayson, Hordley, and Drew
14Other tasks Tracking, etc.
Tracking result for moving hand under lamp
light. Jiang and Drew, 2003
15But problem doesnt always remove all shadows
Depends on camera sensors ?
16How do we find light color change direction?
Sony DXC-930 camera
(Use robust line-finder)
Mean-subtracted log-chromaticity
17Problem invariant image isnt invariant across
illuminants
18Gets worse Kodak DCS420 camera is much less sharp
19How to proceed? Try spectral sharpening, since
wish to make sensors more narrowband. Or just
optimize directly, making invariant image more
invariant.
E.g. optimize color-matching functions
20Invariant image for patches ? apply optimized
sensors to any image
Before optimization of sensors
After optimization of sensors
21How to optimize?
Firstly, lets use a linear matrixing transform,
taking 31 x 3 sensor matrix Q to a new sensor set
sensors
colors
?
?3 x 3
Should we sharpen to get M?
22Should we sharpen to get M?
Sharpening flattening both work
23So need to use a term to steer away from a
rank-reduced M
Optimize on the (correlation coefficient)2 ? R 2
and encourage high effective rank
are singular values of M
Initialize with data-driven spectral sharpening
matrix.
24So optimize M
E.g., color-matching functions R2 goes from 0.43
to 0.94
25HP912 camera
R2 0.86 ? 0.93
entropy 5.856 ? 5.590 bits/pixel
26Real image
entropy 5.295 ? 4.939 bits/pixel with an M
27Thanks!