Neeraj K. Kanhere

1
Vehicle Segmentation and Tracking in the Presence
of Occlusions
Neeraj K. Kanhere Dr. Stanley T. Birchfield Dr.
Wayne A. Sarasua
Clemson University
2
Introduction
Traffic parameters such as volume, speeds,
turning counts and classification are
fundamental for

• Transportation planning

• Traffic impact of land use

• Transportation engineering applications (e.g.
  signal timing)

• Intelligent Transportation Systems (ITS)

3
Why computer vision?

• Different types of sensors can be used to gather
  data
• Radar or laser based sensors
• Inductive loop detectors
• Video Camera (with Computer Vision techniques)

• No traffic disruption for installation and
  maintenance

• Covers wide area with a single camera

• Provides rich visual information for manual
  inspection

4
Why tracking?
Current systems use localized detection within
the detection zones which is prone to errors
when camera placement in not ideal.

• Tracking enables prediction of a vehicles
  location in consecutive frames
• Can provide more accurate estimates of traffic
  volumes and speeds
• Potential to count turn-movements at
  intersections
• Detect traffic incidents

5
Related research

• Region/Contour Based
• Computationally efficient
• Good results when vehicles are well separated
• 3D Model Based
• Large number of vehicle models needed
• Limited experimental results
• Markov Random Field
• Good results on low angle sequences
• Accuracy drops by 50 when sequence is processed
  in true order

• Feature Tracking Based
• Handles partial occlusions
• Good accuracy when sufficient features are
  tracked from entry region to exit region

6
Factors to be considered
High-angle
Mid-angle

• More depth variation
• Occlusions
• A difficult problem

• Planar motion assumption
• Well-separated vehicles
• Relatively easy

7
Overview of the approach
Offline Calibration
Background model
Frame-Block 1
Feature Tracking
Frame-Block 2
Frame-Block 3
Counts, Speeds and Classification
Block Correspondence and Post Processing
8

Background model and calibration

• Adaptive time domain median filtering for
  background
• Calibration provides mapping from scene to image
• Use scene features to estimate correspondences
• Lane widths
• Truck heights
• Approximate calibration is good enough for counts

9
Processing a frame-block
Overlap
Block n
Block n1
features in block
frames in block
10
Frame differencing

• Partially occluded vehicles appear as single blob
• Effectively segments well-separated vehicles
• Goal is to get filled connected components

11
Estimation using single frame
camera
vehicle
Road plane
12
Selecting stable features

• Shadows, partial occlusions will result into
  wrong estimates
• Planar motion assumption is violated more for
  features higher up
• Select stable features, which are closer to road
• Use stable features to re-estimate world
  coordinates of other features

13
Estimation using motion

• Estimate coordinates with respect to each stable
  feature
• Choose coordinates which minimized weighted sum
  of euclidean distance and trajectory error

• Rigid body under translation
• Estimate coordinates with respect to each stable
  feature
• Select the coordinates minimizing weighted sum of
  Euclidean distance and trajectory error

• P Feature with unknown coordinates
• Q Stable feature
• R Backprojection on road
• H Backprojection at maximum height
• 0 First frame of the block
• t Last frame of the block
• ? Translation of corresponding point

14
Affinity matrix

• Each element represents the similarity between
  corresponding features
• Three quantities contribute to the affinity
  matrix
• Euclidean distance (AD), Trajectory Error (AE)
  and Background- Content (AB)

• Normalized Cut is used for segmentation (Shi,
  Malik)
• Number of Cuts is not known

15
Incremental normalized cuts

• We apply normalized cut to initial A with
  increasing number of cuts
• For each successive cut, segmented groups are
  analyzed till valid groups are found
• Valid group meets dimensional criteria
• Elements corresponding to valid groups are
  removed from A and process repeated starting from
  single cut

Avoids specifying a threshold for the number of
cuts
16
Correspondence over blocks

• Formulated as a problem of finding maximum weight
  graph
• Nodes represent segmented groups
• Edge weights represent number features common
  over two blocks

17
Results
18
Results
19
Conclusion

• A novel approach based on feature point tracking
• Key part of the technique is estimation of 3-D
  coordinates
• Results demonstrate the ability to correctly
  segment vehicles even under severe partial
  occlusions
• Vehicle count, speeds and classification (car or
  heavy vehicle) data can be easily obtained for
  tracked vehicles

Future Work

• Robust block-correspondence
• Tracking vehicles at intersections
• Automatic calibration by detecting lane markings
• Explicit shadow suppression

20
Questions ?
21
Thank you !