Segmentation of Vehicles in Traffic Video

1
Segmentation of Vehicles in Traffic Video

• Tun-Yu Chiang
• Wilson Lau

2
Introduction

• Motivation
• In CA alone, gt400 road monitoring cameras, plans
  to install more
• Reduce video bit-rate by object coding
• Collect traffic flow data and/or surveillance
  e.g. count vehicles passing on highway, draw
  attention to abnormal driver behavior
• Two different approaches to segmentation
• Motion Segmentation
• Gaussian Mixture Model

3
Motion Segmentation

• Segment regions with a coherent motion
• Coherent motion similar parameters in motion
  model
• Steps
• Estimate dense optic flow field
• Iterate between motion parameter estimation and
  region segmentation
• Segmentation by k-means clustering of motion
  parameters

Translational model ? use motion vectors directly
as parameters
4
Optic Flow Estimation

• Optic Flow (or spatio-temporal constraint)
  equation
• Ix vx Iy vy It 0
• where Ix , Iy , It are the spatial and temporal
  derivatives
• Problems
• Under-constrained add smoothness constraint
  assume flow field constant over 5x5 neighbourhood
  window
• ? weighted LS solution
• Small flow assumption often not valid
• e.g. at 1 pixel/frame, object will take 10 secs
  (300 frames_at_30fps)
• to move across width of 300 pixels
• ? multi-scale approach

5
Multi-scale Optic Flow Estimation

• Iteratively Gaussian filter and sub-sample by 2
  to get pyramid of lower resolution images
• Project and interpolate LS solution from higher
  level which then serve as initial estimates for
  current level
• Use estimates to pre-warp one frame to satisfy
  small motion assumption
• LS solution at each level refines previous
  estimates
• Problem Error propagation
• ? temporal smoothing essential at higher levels

Level 2
1 pixel/frame
Level 1
2 pixels/frame
4 pixels/frame
Level 0 (original resolution)
6
Results Optic flow field estimation
7
Results Optical flow field estimation

• Smoothing of motion vectors across motion
  (object) boundaries due to
• Smoothness constraint added (5x5 window) to solve
  optic flow equation
• Further exacerbated by multi-scale approach
• Occlusions, other assumption violations (e.g.
  constant intensity)
• ? noisy motion estimates

8
Segmentation

• Extract regions of interest by thresholding
  magnitude of motion vectors
• For each connected region, perform k-means
  clustering using feature vector
• Color intensities give information on object
  boundaries to counter the smoothing of motion
  vectors across edges in optic flow estimate
• Remove small, isolated regions

9
Segmentation Results

• Simple translational motion model adequate
• Camera motion
• Unable to segment car in background
• 2-pixel border at level 2 of image pyramid (5x5
  neighbourhood window) translates to a 8-pixel
  border region at full resolution

10
Segmentation Results

• Unsatisfactory segmentation when optic flow
  estimate is noisy
• Further work on
• Adding temporal continuity constraint for objects
• Improving optic flow estimation e.g. Total Least
  Squares
• Assess reliability of each motion vector estimate
  and incorporate into segmentation

11
Gaussian Background Mixture Model

• Per-pixel model
• Each pixel is modeled as sum of K weighted
  Gaussians. K 35
• The weights reflects the frequency the Gaussian
  is identified as part of background
• Model updated adaptively with learning rate
  and new observation

12
Segmentation Algorithm

• Matching Criterion
• If no match found pixel is foreground
• If match found background is average of high
  ranking Gaussians. Foreground is average of
  low ranking Gaussians

• Update Formula
• Update weights
• Update Gaussian
• Match found
• No Match found
• Replace least possible Gaussian with new
  observation.

13
Segmentation Result 1

• Background disappearing electrical pole,
  blurring in the trees
• lane marks appear in both foreground/background

14
Segmentation Result 2
Cleaner background beginning of original
sequence is purely background, so background
model was built faster.
15
Segmentation Result 3
Smaller global motion in original sequence
Cleaner foreground and background.
16
Parameters matter

• affects how fast the background model
  incorporates new observation
• K affects how sharp the detail regions appears

17
Artifacts Global Motion

• Constant small motion caused by hand-held camera
• Blurring of background
• Lane marks (vertical motion) and electrical pole
  (horizontal motion)

18
Global Motion Compensation

• We used Phase Correlation Motion Estimation
• Block-based method
• Computationally inexpensive comparing to block
  matching

19
Segmentation After Compensation
20
Segmentation After Compensation

• Corrects artifacts before mentioned
• Still have problems residue disappears slower
  even with same learning rate

21
Q A
22
Mixture model fails when

• Constant repetitive motion (jittering)
• High contrast between neighborhood values (edge
  regions)
• The object would appear in both foreground and
  background

23
Phase Correlation Motion Estimation

• Use block-based Phase Correlation Function (PCF)
  to estimate translation vectors.

24
Introduction
25
Our Experiment

• Obtain test data
• We shoot our own test sequences at intersection
  of Page Mill Rd. and I-280.
• Only translational motions included in the
  sequences
• Segmentation
• Tun-Yu experimented on Gaussian mixture model
• Wilson experimented on motion segmentation