Tracking Video Objects in Cluttered Background

1
Tracking Video Objects in Cluttered Background

• Andrea Cavallaro, Member, IEEE, Olivier Steiger,
  Member, IEEE, and Touradj Ebrahimi, Member, IEEE

2
Outline

• Introduction
• Related works
• Hybrid video object tracking
• Experimental results
• Conclusion

3
Introduction(1/2)

• Goal Tracking multiple video objects in
  cluttered background
• Using object and region information to solve the
  problem
• Appearance and disappearance of objects,
  splitting and partial occlusions are resolved
• Video object extraction
• Video object segmentation
• Identifying objects in the scene and separating
  them from the background.
• Video object tracking
• Following video objects in the scene and at
  updating their two-dimensional (2-D) shape from
  frame to frame.

4
Introduction(2/2)

• Establish stable track for dynamic scene
• ? Effective track management
• Track initiation
• Track update
• Track termination
• Temporal variation of the 2-D shape of video
  objects
• Nonrigid objects
• Occlusion
• Splitting
• Appearance and disappearance of objects

5
Related works

• Object tracking methods can be classified into
  five groups model-based, appearance-based,
  contour- and mesh-based, feature-based, and
  hybrid methods.
• Model-based
• Exploit the a priori knowledge of the shape of
  typical objects in a given scene
• Computationally expensive
• Need for object models with detailed geometry for
  all objects that could be found in the scene
• Lack of generality.

6
Related works

• Appearance-based
• Relies on information provided by the entire
  region
• Cannot usually cope with complex deformation.
• Contour- and mesh-based
• Track only the contour of the object.
• Computational complexity is high
• Large nonrigid movements cannot be handled by the
  method
• Active contour models (snakes) 810
• Feature-based
• Uses features of a video object to track parts of
  the object.
• Stable tracks for the features under analysis
  even in case of partial occlusion of the object
• The problem is how to group the features to
  determine which of them belong to the same object

7
Related works

• Hybrid methods
• A hybrid between a region-based and a
  feature-based technique
• Track video objects based on interactions between
  different levels of the hierarchy.
• Higher computational complexity
• This paper uses a hybrid object tracking
  algorithm
• No need for computationally expensive motion
  models.
• It can cope with deformation and complex motion

8
Hybrid video object tracking

• Object partition of frame n
• Object partition is generated by the method
  presented in 16.
• Region partition of objects in frame n
• Take spatiotemporal properties into account and
  extracts homogeneous regions.17

9
Region partition

• Feature space used here is composed of spatial
  and temporal features.
• color features, texture ,displacement vectors

10
Region descriptor

• Create a tracking mechanism for deformable
  objects.
• Each region is represented by a region
  descriptor.
• The region descriptor summarizes the value of the
  features in the corresponding region.
• Next, the tracking mechanism operates on region
  descriptors.

11
Region tracking

• Region tracking is based on a flexible procedure
• Projects the region descriptors from the current
  frame onto the next frame,
• Refines the region partition so as to naturally
  create the updated 2-D topology
• The region descriptor is defined as
• is the number of features in frame
• represent the position of the
  region descriptor,
• is motion vector

12

• In the specific implementation, Ki(n) 8 , and
• represents the
  mean value of the three color components in the
  corresponding region,
• is the mean value of the texture
  feature
• The position predicted through motion
  compensation
• Predicted region descriptor

13
Multilevel Region-Object Tracking

• The joint region-object tracking mechanism is
  organized in two major steps
• Object partition validation
• Results in a tentative correspondence
• Data association
• generates the final correspondence.

14
Object Partition Validation

• Initializes the tracking process and improves the
  accuracy of the object partition
• Before initializing the tracking procedure
• video object is decomposed into a set of regions
• Each region is characterized by its region
  descriptor
• Track initiation
• Each region descriptor is associated to the
  corresponding object
• Takes place at the beginning of the tracking
  process and every time a new video object appears

15
Object Partition Validation

• After the initialization
• Region descriptors are projected into the next
  frame.
• The predicted region is defined as
• After the projection
• A bottom-up feedback from the region partition
  refines the topology of the object partition.
• The feedback generates a tentative correspondence
  by labeling the object partition
  according to the predicted region partition

16
Object Partition Validation

• Once all the pixels in the next object partition
  are associated to the projected regions, we have
  a prediction as follows
• This procedure is straightforward in case each
  set of connected pixels receives projected region
  descriptors, and receives them from one object
  only
• ?In reality, multiple simultaneous objects may
  occlude each other
• and therefore be included in the same set of
  connected pixels.
• ?The bottom-up interaction is used to improve the
  object labeling in these cases

17
bottom-up interaction

• A new object
• A connected set of pixels in does not
  get any region descriptor from the projection
  mechanism.
• The detection of a new object triggers a track
  initiation
• An occlusion
• A connected set of pixels in receives
  projected region descriptors from several objects

18
bottom-up interaction

• A splitting
• Two different disconnected sets of pixels get
  region descriptors projected from the same video
  object.
• The predicted partition may not cover all the
  pixels
• If a connected component of receives
  region descriptors from one object only
• Unclassified pixels are assigned to that object.
• If a connected set of receives region
  descriptors from several objects
• The unclassified pixels are assigned to the
  closest projected region.

19
Data Association

• Data association validates the track of each
  region descriptor, and this step is particularly
  important when faced with track management
  issues.
• To verify the correctness of the tentative
  correspondence obtained with region descriptors
  projection, we consider the proximity between
  region descriptors in and in

20
Data Association

• The proximity is computed by measuring the
  distance in the feature space between the region
  descriptors
• Before distance computation,using preselection to
  eliminate the computation
• The Mahalanobis distance
• The distance computeed within each category
• A tentative correspondence between the pth region
  description frame and the qth region descriptor
  in frame n1 is confirmed if

21
Experimental results
22
Experimental results
23
Experimental results
24
Experimental results
25
Conclusion

• We presented an automatic tracking algorithm
  based on interactions between video objects and
  their regions
• Using region descriptor to track the
  corresponding video object
• Future works
• Simpler region segmentation algorithm
• Initialization of a track when groups of objects
  enter the scene and total occlusions
• The data association step may operate on a longer
  temporal window