Real Time Video Segmentation

1
Real Time Video Segmentation

• Feng Xie

2
Motivation

• Video compositing layering
• Video Avatar
• Object Recognition
• Video understanding
• Video Surveillence

3
Overview Comparison

• Sample based segmentation
• Simple and easy to implement and accelerate
• insensitive to object or scene space complexity
• sensitive to lighting and color changes
• Model based segmentation
• more robust against light and color changes by
  exploiting model and motion continuity
• complexity increases with scene complexity

4
Real Time Human Tracking

• C.Wren, etc, "PFinderReal-Time Tracking of the
  Human Body", PAMI, 1997, pp 780-785

Segmentation using Mahalanobis distance
5
Adaptive background elimination

• A. Francois and G. Medioni on Adaptive Color
  Background Modeling for Real-Time Segmentation of
  Video Streams

6
Video Avatar for Lecture of the Future

• Real time motion can be sudden and jerky
• Lots of lighting changes due to highly emmisive
  and specular surfaces
• Shadows, reflections and occlusion

7
Prototype Implementation

• Color Differencing
• Hue Differencing
• (for removing shadows)
• Mophological Filtering
• removing noisy outliers and patch up holes
• Connected Component
• removing big pieces of outliers.

8
Results

• System works well for room when
• lighting condition is fairly stable
• the foreground human is well lit
• most of parts of the human is well contrasted
  with the background room.

9
Results

• System fails when
• lighting condition is highly variant
• foregound human is dark
• major part of the human has the same color as the
  background.

10
Video Demos

• Feng
• Segmented Feng
• James
• Segmented James
• Milton
• Segmented Milton