Skin ColorBased Video Segmentation under TimeVarying Illumination

1
Skin Color-Based Video Segmentation under
Time-Varying Illumination

• By Lenoid Sigal, Stan Sclaroff, Vassilis Athitsos
• Presented by Ryan Lustig
• 02/07/05 ECE 285

2
Agenda

• Overview
• Approach
• Initialization
• Learning
• Prediction and Tracking
• Results
• Movies
• Live Video

3
Overview

• System that addresses the following
• Smooth but relatively fast changing illumination
• Its effects on skin-color appearance
• Concerned with 3 conditions
• Time-varying illumination
• Multiple sources, with time-varying illumination
• Single- or multiple-colored sources

4
Overview (Cont.)

• Works on purely color data
• Uses predictive histogram adaptation to model
  color distribution over time

5
Initialization

• Computed histograms using skin and nonskin images
• Obtained conditional probability densities
• P(rgbfg) P(rgbbg)

6
Initialization (Cont.)

• Use probabilities to compute classification
  boundary threshold, K.
• P(fg), probability of a pixel being skin
• Chose K such that 85 correct classification and
  25 false alarm rate K .0673

7
Learning Color Space

• A new sampled histogram is created from the
  sequence of frames
• Due to possible sampling problems, a 3x3x3
  Gaussian kernel is used for smoothing
• Each video frame is converted from RGB to HSV

8
Learning Motion of Distribution

• Assume
• Skin-color distribution evolves as a whole
• Global distributions are affine
• Use 3 affine transformations
• Translation, Scaling and Rotation
• TH, TS, TV, SH, SS, SV, ?, f

9
Learning Estimating Motion Param.

• Translation (TH, TS, TV)
• Extracted from the mean of HSV skin-color
  distribution histogram
• Scaling (SH, SS, SV)
• Extracted from the standard deviation of HSV
  skin-color distribution
• Rotation (?, f)
• , where e1,t is the
  eigenvector with largest eigenvalue of covariance
  matrix at time t

10
Learning Estimating Motion Param.
11
Learning Dynamical Distribution Model

• Motion model used for predictions is a
  second-order discrete time Markov model
• X is an 8-dimensional parameter vector
• A0 and A1 govern the deterministic part of motion
  model
• B governs the stochastic part of motion model

12
Learning the Parameters

• The parameters, (A0, A1, B, X) are estimated
  using Maximum Likelihood Estimation (MLE)
• Able to learn the parameters in a minimum of 4
  frames less than 1 second

13
Learning Histogram Adaptation

• Updates to the histograms are made via
• H(p) is predicted by the second-order Markov
  model
• a is adaptation coefficient
• Computed offline

14
Prediction and Tracking

• Using the previously estimated Markov model, the
  new parameters can be predicted
• Warp color vectors and resample the new
  distribution
• New samples used to predict histogram for next
  frame

15
Prediction and Tracking (Cont.)
16
Results
17
Results (Cont.)

• Skin classification rates dynamic histograms were
  as good or better than static in all but one case
• Background classification was comparable

18
Results (Cont.)

• Performance degradation in some sequences due to
  skin-like color patches in background of initial
  frames

19
Results Live Video
20
Final Thoughts

• System works well, but highly dependent upon
  initialization phase
• Real-time?
• CPU System used?