CS6534 Guided Studies Crowd Flow Analysis

1
CS6534 Guided StudiesCrowd Flow Analysis

• Supervised by Dr. Hau-San WONG
• Prepared by Kam-fung YU
• (51150118)

2
Background challenge

• Video Surveillance System are widely used for
  monitoring
• Performance is good as the number of object for
  detection is small (Spatial variation is small)
  and
• The change over time is small (Temporal variation
  is small)
• BUT
• A challenge for crowd flow
• The number of objects is in the order of 102103
• The change of the scene is very fast

3
Related works

• Based on Tracking of Individuals
• Shape and Color Model of Individuals
• Trajectories of Points
• Boundary Contour
• xt Slices of Spatio-temporal Video Volume
• People Counting in the Crowd

4
Shape color model of individuals
3D human model

• Models human shape by using 3D model
• Data-driven Markov chain Monte Carlo (DDMCMC)
• Iterate an optimized solution

T. Zhao et. al., Bayesian Human Segmentation in
Crowded Situations, IEEE CVPR03, 2003.
5
Trajectories of points

• Similar method with Shape Color Model
• Use some simple feature, such as corner of an
  object, to extract points probabilistically
• Clustering the points into independently moving
  entities, cluster

Shape and Color Model
Trajectories of Points
G. Brostow et. al., Unsupervised Bayesian
Detetcion of Independent Motion in Crowds, IEEE
CVPR, 2006.
6
Boundary contour

• Use of low-interest points to detect the object
  clustering
• Select by the high temporal and spatial
  discontinuity
• Outline the object by joining edges

Clustered object
Sample Scene
P. Tu et. al., Crowd Segmentation through
Emergent Labeling, In ECCV Workshop SMVP, 2004
7
Xt sclies of spatio-temporal video volume

• Scan interesting lines over a certain frames,
  xt-slice
• Use the Hough transform to detect movement in the
  xt slices

5 corresponding xt slices
Sample Scene with 5 lines
Hough transform
P. Reisman, Crowd Detection in Video Sequences,
IEEE Intelligent Vehicles Symposium, 2004.
8
People counting in the crowd

• Clustering of some feature points by their motion
• Estimate the number of people by the number of
  cluster

A result of clustering on two video scene
V. Rabaud et. al., Counting Crowded Moving
Objects, IEEE CVPR, 2006
9
Limitations on tracking of individuals

• Involes Iteration
• Convergence Decease as Number of Objects Increase
• Large Computational Time
• High of Computational Power
• Difficulty to implement on Real Time Monitoring
  System

10
Our approach

• Proposed by Saad Ali Mubarak Shah in 2007
• Individual Flow ? Global Optical Flow
• Tracking Individuals ? Measuring Global
  Quantities
• Using Fluid Dynamics to treat the problem
• Global QuantitiesFinite Time Lyapunov Exponents
  Field (FTLE), Lagrangian Coherent Structures(LCS)
• Expect a Higher Faster Algorithm in Performance

11
Algorithmic outline
12
Optical flow

• 16x16 size block
• Displacement vector x
• p frames for 1 mean field
• q mean field for 1 block mean field

S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007
13
Algorithmic outline
14
Flow map

• Launch a set of particles over the optical flow
  field
• Solve a flow map for a time period T p?q frames
• Interpolation a cubic velocity equation by 4th
  order Runge-Kutta-Fehlberg algorithm (RK4)
• ??x, ?y are used to record the x and y coordinate
  at each initial position launched after time

Flow map of x-particle
Flow map of y-particle
S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007
15
Algorithmic outline
16
Ftle Field

• Compute the four spatial derivates
• Plug into the Cauchy-Green deformation tensor
• The largest finite time Lyapunov exponent with
  the maxmum eigenvalue ?max of the tensor and the
  period Tp?q frames

FTLE Field Plot
S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007
17
Algorithmic outline
18
lcs
FTLE Field Plot
S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007
19
segmentation

• This process involves two stages
• Cut spatially into different region by the ridges
  in FTLE
• Use the Lyapunov divergence to decided two
  segment merge or not

1st stage
2nd stage
S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007
20
Algorithmic outline
21
Flow instability detection

• Flow instability is defined as the change in the
  number of flow segments with respect to time

New segment
Current segment
S. Ali, M. Shah, A Lagrangian Particle Dynamics
Approach for Crowd Flow Segmentation and
Stability Analysis, CVPR May, 2007
22
Capabilities

• Capable for monitoring thousands of objects
  simultaneously
• Get rid of number of people constrain
• Capable for monitoring flow in any orientation
• Obtain same result under any rotation
• Capable for new segment detection over time
• Locate the increase or the decrease of segments
  over time

23
Potentials

• Potential for flow control or city design
• Making immediate decision for crowd flow
• Facilitate on the planning of city streets,
  traffic flow, overhead, bridges and passageways
• Potential for flow pattern recognition
• Extraction of various flow pattern
• Flow pattern solution space construction for a
  given static scenery
• Flow pattern bases finding

24
limitations

• Limitation on crowd density
• Degraded as crowd density is low
• Worse at only a number of objects
• Limitation on a large number of many-fold
  dynamics flow
• Too many segments (too noisy) on the scene
• Hard to merge segment
• Limitation on a rapid unstable flow
• Hard to retrieval information from rapid changing
  flow
• Too slow to capture the information

25
Further suggestions

• Find out the critical crowd density for an
  acceptable performance
• Finding out a method that can undergo
  segmentation under a noisy domain
• Designing a rapid flow capturing algorithm
• Finding out the possible flow patterns on given
  static scenery
• Find out the flow patterns solution space and
  bases

26
conclusion

• In this guided study, we studied about various
  kinds of methods and the Lagrangian Dynamics in
  solving the crowd segmentation problem.
• We also realized the capabilities, potentials and
  limitations .
• We finally suggested some possible direction for
  future studies.

27
references

• Z.N. Li, M.S. Drew, Fundamentals of Multimedia,
  NJ Pearson Education Hall, 2004
• P.E. Mattison, Practical Digital Video with
  programming examples in C, NY John Wiley Sons
  Inc, 1994
• L. Perko, Differential Equations and Dynamical
  Systems 3rd Ed., NY Springer, 2001
• Intel Corporation, Open Source Computer Vision
  Library, Reference Manual, USA Intel
  Corporation, 2001
• S. Ali, M. Shah, A Lagrangian Particle Dynamics
  Approach for Crowd Flow Segmentation and
  Stability Analysis, CVPR May, 2007
• S. C. Shadden, Lagrangian Coherent Structures
  Analysis of time-dependent dynamical systems
  using finite-time Lyapunov exponents,Available
  Online http//www.cds.caltech.edu/shawn/LCS-tut
  orial/, Last update 15th April, 2005
• P. Reisman, Crowd Detection in Video Sequences,
  IEEE Intelligent Vehicles Symposium, 2004.
• T. Zhao et. al., Bayesian Human Segmentation in
  Crowded Situations, IEEE CVPR03, 2003.
• P. Tu et. al., Crowd Segmentation through
  Emergent Labeling, In ECCV Workshop SMVP, 2004.
• G. Brostow et. al., Unsupervised Bayesian
  Detetcion of Independent Motion in Crowds, IEEE
  CVPR, 2006.
• D. Yang et. al., Counting People in Crowds with
  a Real-Time Network of Simple Image Sensors,
  ICCV, 2003.
• V. Rabaud et. al., Counting Crowded Moving
  Objects, IEEE CVPR, 2006.
• E. Rosten and T. Drummond, Machine learning for
  high-speed corner detection, Europe Conference
  on Computer Vision, May 2006.
• C. Tomasi and T. Kanade, Detection and tracking
  of point features, Technical Report
  CMU-CS-91-132, Carnegie Mellon University, April
  1991.
• S. Ali, Crowd Flow Segmentation Stability
  Analysis, Available Online http//www.cs.ucf.ed
  u/sali/Projects/CrowdSegmentation/index.html ,
  Last visited 30th Nov, 2008

28
Thank you

• Department of Computer Science
• City University of Hong Kong