Motion Detection And Analysis

1
Motion Detection And Analysis

• Michael Knowles
• Tuesday 13th January 2004

2
Introduction

• Brief Discussion on Motion Analysis and its
  applications
• Static Scene Object Tracking
• Motion Compensation for Moving-Camera Sequences

3
Applications of Motion Tracking

• Control Applications
• Object Avoidance
• Automatic Guidance
• Head Tracking for Video Conferencing
• Surveillance/Monitoring Applications
• Security Cameras
• Traffic Monitoring
• People Counting

4
Two Approaches

• Optical Flow
• Compute motion within region or the frame as a
  whole
• Object-based Tracking
• Detect objects within a scene
• Track object across a number of frames

5
My Work

• Started by tracking moving objects in a static
  scene
• Develop a statistical model of the background
• Mark all regions that do not conform to the model
  as moving object

6
My Work

• Now working on object detection and
  classification from a moving camera
• Current focus is motion compensated background
  filtering
• Determine motion of background and apply to the
  model.

7
Static Scene Object Detection and Tracking

• Model the background and subtract to obtain
  object mask
• Filter to remove noise
• Group adjacent pixels to obtain objects
• Track objects between frames to develop
  trajectories

8
Background Modelling
9
Background Model
10
After Background Filtering
11
Background Filtering

• My algorithm based on
• Learning Patterns of Activity using Real-Time
  Tracking C. Stauffer and W.E.L. Grimson. IEEE
  Trans. On Pattern Analysis and Machine
  Intelligence. August 2000
• The history of each pixel is modelled by a
  sequence of Gaussian distributions

12
Multi-dimensional Gaussian Distributions

• Described mathematically as
• More easily visualised as
• (2-Dimensional)

13
Simplifying.

• Calculating the full Gaussian for every pixel in
  frame is very, very slow
• Therefore I use a linear approximation

14
How do we use this to represent a pixel?

• Stauffer and Grimson suggest using a static
  number of Gaussians for each pixel
• This was found to be inefficient so the number
  of Gaussians used to represent each pixel is
  variable

15
Weights

• Each Gaussian carries a weight value
• This weight is a measure of how well the Gaussian
  represents the history of the pixel
• If a pixel is found to match a Gaussian then the
  weight is increased and vice-versa
• If the weight drops below a threshold then that
  Gaussian is eliminated

16
Matching

• Each incoming pixel value must be checked against
  all the Gaussians at that location
• If a match is found then the value of that
  Gaussian is updated
• If there is no match then a new Gaussian is
  created with a low weight

17
Updating

• If a Gaussian matches a pixel, then the value of
  that Gaussian is updated using the current value
• The rate of learning is greater in the early
  stages when the model is being formed

18
Colour Spaces

• If RGB is used then the background filtering is
  sensitive to shadows
• The use of a colour space that separates
  intensity information from chromatic information
  overcomes this
• For this reason the YUV colour space is used

19
Colour Spaces

• Background and Frame
• Channel Differences

20
Isolate Objects

• Groups of object pixels must be grouped to form
  objects
• A connected components algorithm is used
• The result is a list of objects and their
  position and size

21
Track objects

• Objects are tracked from frame to frame using
• Location
• Direction of motion
• Size
• Colour

22
The Story So Far

• Basic principle of background filtering
• Stages necessary in maintaining a background
  model
• How it is applied to tracking

23
Moving Camera Sequences

• Basic Idea is the same as before
• Detect and track objects moving within a scene
• BUT this time the camera is not stationary, so
  everything is moving

24
Motion Segmentation

• Use a motion estimation algorithm on the whole
  frame
• Iteratively apply the same algorithm to areas
  that do not conform to this motion to find all
  motions present
• Problem this is very, very slow

25
Motion Compensated Background Filtering

• Basic Principle
• Develop and maintain background model as
  previously
• Determine global motion and use this to update
  the model between frames

26
Advantages

• Only one motion model has to be found
• This is therefore much faster
• Estimating motion for small regions can be
  unreliable
• Not as easy as it sounds though..

27
Motion Models

• Trying to determine the exact optical flow at
  every point in the frame would be ridiculously
  slow
• Therefore we try to fit a parametric model to the
  motion

28
Affine Motion Model

• The affine model describes the vector at each
  point in the image
• Need to find values for the parameters that best
  fit the motion present

29
Minimisation of Error Function

• If we are to find the optimum parameters we need
  an error function to minimise
• But this is not in a form that is easy to
  minimise

30
Gradient-based Formulation

• Applying Taylor expansion to the error function
• Much easier to work with

31
Gradient-descent Minimisation

• If we know how the error changes with respect to
  the parameters, we can home in on the minimum
  error
• Various methods built on this principle

32
Applying Gradient Descent

• We need
• Using the chain rule

33
Robust Estimation

• What about points that do not belong to the
  motion we are estimating?
• These will pull the solution away from the true
  one

34
Robust Estimators

• Robust estimators decrease the effect of outliers
  on estimation

35
Error w.r.t. parameters

• The complete function is

36
Aside Influence Function

• It can be seen that the first derivative of the
  robust estimator is used in the minimisation

37
Pyramid Approach

• Trying to estimate the parameters form scratch at
  full scale can be wasteful
• Therefore a pyramid of resolutions or Gaussian
  pyramid is used
• The principle is to estimate the parameters on a
  smaller scale and refine until full scale is
  reached

38
Pyramid of Resolutions

• Each level in the pyramid is half the scale of
  the one below i.e. a quarter of the area

39

• Out pops the solution.
• When combined with a suitable gradient based
  minimisation scheme

40
Problems with this approach

• Resampling the background model
• Model cannot be too complex
• Resampling will bring in errors
• Motion model is only an estimate of what is
  really happening
• Can lead to false object detection particularly
  close to boundaries

41
Background Model Design

• The background model needs to be robust to these
  problems
• We need some way to differentiate between genuine
  object detections and false ones from motion
  model and background model errors

42
My Approach

• Rather than updating model values with current,
  matched values replace them
• In this way resampling errors are not allowed to
  accumulate

43
Aside Real Time

• The ability to process a sequence in real-time is
  dependent on THREE key factors
• The speed of the algorithm
• The frame rate required
• The number of pixels

44
Recap

• Introduction to motion analysis
• Principles of background modelling
• Example of a static scene tracker
• Discussion of motion estimation
• Shortcomings when applied to background filtering