CSE 291 Final Project: Adaptive MultiSpectral Differencing

1
CSE 291 Final ProjectAdaptive Multi-Spectral
Differencing

• Andrew Cosand
• UCSD CVRR

2
Differencing

• Detect changes in a sequence of images.
• Pixels of reference image are subtracted from the
  current image to determine how different they
  are.
• Pixels with exceed some difference threshold are
  assumed to correspond to different objects in the
  images.

3
Differencing
Reference Image Current Image Difference
4
Problems

• Differences other than the object of interest may
  show up.
• Pixel noise
• Moving background objects (trees, water)
• Lighting changes
• Camera movement (small)
• Shadows Reflections

5
Pixel Noise
6
Solutions

• Variations can be included in a background model.
• Reference frame may use eg Gaussian mixture
  models to characterize pixels
• Reference frame can be updated at different
  rates. Very slow basically detects changes from
  when the system was started, very fast detects
  changes from the previous frame.

7
Camera Movement
8
Solutions

• Very small camera movements can be modeled in the
  background similar to pixel noise or moving
  background objects
• Other segmentation methods can be used to
  identify and track objects in the scene
• Camera motion can be identified and corrected
  (Optical flow, correspondence)

9
Shadows
Detected Difference
Good
Bad
Shadow
10
Solutions

• Color Space Conversion
• Transform data into more useful form, eg
  normalized chromaticity or Hue Saturation
  Intensity colorspace, which separates color and
  intensity for robust detection in the presence of
  shadows.

11
HSI

• Hue angle determines color
• Saturation determines how colorful or washed
  out
• Intensity determines brightness

12
HIS Colorspace Detection

• Shadows simply decrease intensity without
  effecting hue
• Hue differencing is therefore quite robust to the
  presence of shadows
• Great
• But.

13
Hue Determination

• To decide what color a pixel is, it must first
  have a color
• Conversion
• Normalize R,G,B s.t. 0 ? r,g,b ? 1
• h acos (r-g)(r-b)
• 2(r-g)2 (r-b)(g-b)1/2
• Very sensitive when r ? g ? b

14
Hue Differencing
Hue Noise Causes False Detects
15
Idea

• Since hue information is unreliable for grayish
  pixels, ignore hue difference results at these
  pixels and use intensity instead.
• Need some weighting function which determines how
  to do this.

16
Previous Solution

• Francois and Medioni used a saturation threshold
  to ignore hue information for gray pixels
• Works well
• Requires threshold to be set

17
Goal

• Want a weighting function which will specify a
  combination of hue and intensity differencing.
• Intensity should receive more weight when hue is
  unreliable
• Hue should receive more weight when it can be
  reliably determined
• Hope to find some underlying relationship

18
Implementation

• Using Euclidian distance to gray line as a color
  measure
• Saturation is somewhat tricky (a la Matlab)
• Ideal system would determine weighting function
  based on training data, similar to backpropogation

19
Backpropogation

• Outputs are weighted combinations of inputs
• Determine errors at outputs
• Determine how much each input was responsible for
  the error
• Adjust each weight accordingly

20
Current Algorithm

• Examines each pixel, changes weight in proportion
  to the error
• For pixels which should have detected, weight is
  increased proportionally to 1-detection
• For pixels which should NOT have detected, weight
  is DECREASED proportionally to detection

21
Insights

• Examination of hue errors shows a definite
  correlation to coloration

22
Results Weighting Functions
23
Lack of Colorful Data
24
Results Combined Detection
25
Problems

• Correlation can vary widely from image to image.
• Weights are noisy, skewed by lack of colorful
  data
• Probably needs more data processing
• No good model determined yet

26
Conclusion

• System shows definite promise
• Model still needs to be determined and adaptively
  fit

27
Shadow Supression
28
References

• A.R.J. Francois, G.G. Medioni, Adaptive Color
  Background Modeling for Real-Time Segmentation of
  Video Streams
• A. Prati, I. Mikic, M. Trivedi, R. Cucchiara,
  Detecting Moving Shadows Formulation, Algorithms
  and Evaluation
• T. Horprasert, D. Harwood, L.S. Davis, A
  statistical approach for real-time robust
  background subtraction and shadow detection