Automatic Compensation for Camera Settings for Images Taken

1
Automatic Compensation for Camera Settings for
Images Taken under Different Illuminants
Cheng Lu and Mark S. Drew Simon Fraser
University clu, mark_at_cs.sfu.ca
2
Flash/No-flash Imagery What About Camera
Settings?
(or, more generally, pairs of images with two
different illuminants).

• Growing body of research on combining
  flash/no-flash
• image pairs to carry out tasks in
• Computer Vision and in
• Color Science

3
One use Removing Shadows using Flash/Noflash
Image Edges Lu, Drew, Finlayson, ICME 2006

4
(No Transcript)
5
(No Transcript)
6
(
)
7
But need to ensure that
-
really gives just the image under pure-flash
lighting.
If settings are different, wont work, without
compensation!
8
Strategy

• Wish to compensate for
• exposure time,
• ISO,
• aperture,
• focal length,
• white balance.
• Can use a 2nd-order masking model (i.e.,
  polynomial) on such parameters
• How do we know how to compensate?
• Make shadow disappear for difference of adjusted
  images, by matrixing,
• Map pairs of settings to matrix via masking
  model.

9
Strategy, contd

• Simplify matrix Adjust magnitude in each color
  channel so as to eliminate shadow in
• (with-flash) (no-flash),
• over large set of image pairs.
• Train polynomial model.
• Apply polynomial model to new image pairs.

10
Assumptions

• Additivity and proportionality of (transformed)
  camera parameters
• 2nd order polynomial model ? 9 parameters.
• (Compare CMY overprinting

)
11
Example of image pairs
Ambient light (A)
Scaled to max255
No scaling
Ambient flash (Both, B)
12
Now subtract
No, see shadow in pure-flash image!
So use in-shadow, out-of-shadow regions to obtain
3 color-channel multipliers ?
13
? We need 3-vector of scaling coefficients A ?A?
so boxes match, in difference image.
Call in-shadow region s, out-of-shadow ns
14

• Now what is M A ?A? as a function of camera
  settings?
• ? use polynomial model (like for printers) --
  uses logs and assumes additivity and
  proportionality of values.

Parameters
15

• Training
• 1. Fix focal length, use tripod.
• 2. Use auto setting and acquire actual
  settings used from stored image meta-data.
• 3. Use EV (exposure value) same for all shutter
  speed/aperture combinations that give same
  exposure. In APEX system (Additive Photographic
  Exposure System), EVAVTV
• AVApertureValuelog2f2, TVTimeValue-log2t
• 4. ISO automatic

5. White balance ? encapsulate the effect of
white balancing by using use the mean value for
each RGB channel in the masking model.
16
6. Ok, we generate values in M A ?A? by
selecting in/out-of-shadow areas by hand. What
model should we use for mapping settings to M? ?
Use logs of ratios, in 2nd-order model
9 parameters a1,a2,a3,b1,b2,b3,c1,c2 c3, so use
least-squares.
Then apply same model to new image pair.
17
3N x 1
3N x 9
9 x 1
??
18
Experiments
125 training image pairs 125 tests using
take-one-out re-calc. of M re-compute 9
params, predict M, apply.

• 5 lighting sources
• Direct sunlight, cloudy daylight, a tungsten
  light lamp and incandescent lamp, and xenon flash
  light.
• Images captured in 5 situations

19
Ambient images
Ambientflash images
Pure flash images
Success! no shadows
20
Thanks! To Natural Sciences and Engineering
Research Council of Canada