Color constancy at a pixel [Finlayson et al. CIC8, 2000]

1
(No Transcript)
2
Color constancy at a pixel Finlayson et al.
CIC8, 2000
Idea plot log(R/G) vs. log(B/G)
3
Log(R/G)
Log(B/G)
Log(B/G)
Log(R/G)
For every patch, the direction from light color
change is about the same!
4
Why all linear and same direction?
The image formation equation
color
shading
intensity
light SPD
k1..3
reflectance
sensor
Now lets make some assumptions
5
Assumption 1 Light is Planckian (or some other
1D assumption)
Wiens approximation of a Planckian source
Note 1D parameter T temperature light
color.
P100
6
Assumption 2 Narrow band sensors
The Sony Camera has fairly narrow band
sensitivities
Using spectral sharpening, we can make almost
all sensor sets have this property. Finlayson,
Drew, Funt
SONY DXC-930
7
Modified Image Formation
The kth response
Substituting Narrow-band and Planckian Assumptions
Take logs
Response light intensity surface
light color
8
Implications
We have k equations of the form
is common to all equations and can be removed
by simple differencing at this pixel
This results in k-1 independent equations of the
form
light color term
reflectance term
9
Implications
(1) If there are 3 sensors we have two
independent equations of this form
(2) For a single surface viewed under different
colored lights the log chromaticities must fall
on a line
(3) Different surfaces induce lines with the
same orientation
The log chromaticities of 7 surfaces viewed under
10 lights
10
One degree of freedom is invariantto light
change
Luminance
1D invariant
Project to 1D
Gray?
11
More formally
form ratios
and define
define vectors
?
line in 2D ?
12
What is this good for?
With certain restrictions, from a 3-band color
image we can derive a 1-D grayscale image which
is - illuminant invariant - and so, shadow free
13
Then use edge info. to integrate back without
shadows ECCV02 Finlayson, Hordley, and Drew
14
Other tasks Tracking, etc.




Tracking result for moving hand under lamp
light. Jiang and Drew, 2003
15
But problem doesnt always remove all shadows
Depends on camera sensors ?
16
How do we find light color change direction?
Sony DXC-930 camera
(Use robust line-finder)
Mean-subtracted log-chromaticity
17
Problem invariant image isnt invariant across
illuminants
18
Gets worse Kodak DCS420 camera is much less sharp
19
How to proceed? Try spectral sharpening, since
wish to make sensors more narrowband. Or just
optimize directly, making invariant image more
invariant.
E.g. optimize color-matching functions
20
Invariant image for patches ? apply optimized
sensors to any image
Before optimization of sensors
After optimization of sensors
21
How to optimize?
Firstly, lets use a linear matrixing transform,
taking 31 x 3 sensor matrix Q to a new sensor set
sensors
colors
?
?3 x 3
Should we sharpen to get M?
22
Should we sharpen to get M?
Sharpening flattening both work
23
So need to use a term to steer away from a
rank-reduced M
Optimize on the (correlation coefficient)2 ? R 2
and encourage high effective rank
are singular values of M
Initialize with data-driven spectral sharpening
matrix.
24
So optimize M
E.g., color-matching functions R2 goes from 0.43
to 0.94
25
HP912 camera
R2 0.86 ? 0.93
entropy 5.856 ? 5.590 bits/pixel
26
Real image
entropy 5.295 ? 4.939 bits/pixel with an M
27
Thanks!