IBM Smart Surveillance System S3 Sales and Technical Training

1
Emerging Topics in Video Surveillance
Rogerio Feris IBM TJ Watson Research
Center rsferis_at_us.ibm.com http//rogerioferis.com
2
Outline

• Video Surveillance in Crowded Scenarios
• Online Learning Self-adaptation in
  Surveillance
• Other Recent Topics

3
Simple Scenarios

• Few Objects Background Subtraction Tracking
  High-level Event/Alert Detection

• Current systems work well

4
Crowded Scenarios

• Many objects, occlusions, shadows, etc.

Object Segmentation, Tracking and Event Analysis
in crowded scenarios Open Problem!
5
Parts-based Detectors
Pedro et al, A discriminatively trained,
multiscale, deformable part model, CVPR08

• Occlusion Handling

• Root filter (low-res) Parts filters (high-res)

6
Parts-based Detectors

• Score of a window score of root score of
  parts
• Score of Parts Appearance Geometry
• Efficient localization of parts through Dynamic
  Programming
• SVM Classification (Structured prediction)

7
Detecting Pedestrians in Crowds
Leibe et al, Pedestrian Detection in Crowded
Scenes, CVPR05

• Combination of different models bag of
  features, segmentation, and chamfer matching

8
Tracking in Crowds
Andriluka et al, People-tracking-by-detection
and people-detection-by-tracking, CVPR08

• Extends Leibe et al, CVPR05 to
  temporal-domain and person articulation (parts)
  estimation

• Click for Video Demo

9
Crowd Segmentation
Dong et al, Fast Crowd Segmentation Using Shape
Indexing, ICCV07
10
Crowd Analysis
Ali Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR07
11
Online Learning
12
Offline Adaboost Learning

• Adaboost ensembles many weak classifiers into
  one single strong classifier

• Initialize sample weights
• For each cycle
• Find a classifier/rectangle feature that performs
  well on the weighted samples
• Increase weights of misclassified examples
• Return a weighted combination of classifiers

13
Offline Adaboost Learning
Major Problems

• Large number of examples required to train a
  robust classifier
• time consuming to label data
• slow training (may take several days)
• No Adaptation to particular surveillance
  scenarios

14
Learning from Small Sets

• Choice of Features (Levi Weiss, CVPR04)
• Co-Training (Levin Viola, ICCV2003)

• Online Adaptation
• Online Boosting (Oza01, Javed05, Bischof06,
  Pham07)

15
Online Boosting Oza,2001

• Train a generic strong classifier (set of weak
  classifiers, of weak classifiers fixed) on a
  small training set.
• Online Process
• Given one single example with known label
• Slide the example over each weak classifier
• When the weak classifier receives the example
• update the weak classifier online
• update the weight of the example and pass to the
  next weak classifier

16
Online Boosting Oza,2001
17
Online Boosting Oza,2001
18
Online Boosting Car and People Detection Omar
Javed, CVPR05

• Train a generic strong classifier (set of weak
  classifiers, of weak classifiers fixed) on a
  small training set.
• While running the classifier on unlabeled data,
  if an example is confidently predicted by a
  subset of weak classifiers ? use it for online
  learning
• Co-training framework
• BGS used for efficiency, for using more
  expensive features, and for balancing the number
  of positive and negative examples

19
Online Boosting Car and People Detection Omar
Javed, CVPR05
20
More Recent Work

• Helmut Hurst, Online Boosting and Vision,
  CVPR06

• Bo Wu Nevatia, Improving Part-based Object
  Detection by Unsupervised, Online Boosting,
  CVPR07

• Pham Cham, Online Learning Asymmetric Boosted
  Classifiers for Object Detection, CVPR07

• Huang et al.,Incremental Learning of Boosted
  Face Detector, ICCV07 Boosting Adaptation

• IEEE Online Learning for Classification Workshop
  (CVPR08)

21
Other Recent Topics
22
High-Resolution Imagery Kopf et al, Capturing
and Viewing Gigapixel Images, SIGGRAPH07

• How can we make use of high-resolution in video
  analytics?

• Much more info e.g., in face reco skin
  texture, iris, etc.

23
Next Generation Neural Networks Hinton, Reducing
the dimensionality of data with neural networks,
Science 2006

• New algorithm for learning deep belief nets
• State-of-the art results in MNIST digit dataset
  (better than SVMs)
• Youtube talk at Google http//www.youtube.com/w
  atch?vAyzOUbkUf3M
• Matlab Code http//www.cs.toronto.edu/hinton/

24
Learning with lots of data

• How can we recognize thousands of products in a
  retail store for loss prevention?
• 80 Million Tiny Images (http//www.cs.nyu.edu/f
  ergus/)

Surveillance with Moving Cameras

• Cameras in vehicles, or even wearable cameras.
  New challenges object detection, etc.
• Leibe et al, Dynamic 3D Scene Analysis from a
  Moving Vehicle, CVPR 2007

25
Many more recent topics

• Check for papers in recent computer vision
  conferences (like CVPR, ICCV, and ECCV) and also
  specialized workshops/conferences such as AVSS
  and PETS