Motion Tracking

1
Motion Tracking
2
Introduction

• Finding how objects have moved in an image
  sequence
• Movement in space
• Movement in image plane
• Camera options
• Static camera, moving objects
• Moving camera, moving objects

3
Contents

• Acquiring targets
• Image differencing
• Moving edge detector
• Following targets
• Matching
• Minimum path curvature
• Model based methods
• Kalman filtering
• Condensation
• Hidden Markov Model

4
Camera calibration revisited

• Image to camera co-ordinate transformation
• Intrinsic parameters
• Camera to world co-ordinate transformation
• Extrinsic parameters

5
Camera to Image Co-ordinates Distortionless
Camera

• If
• no distortions
• uniform sampling
• Co-ordinates linearly related
• offset and scale

6
Camera to Image Co-ordinates Distorting Camera

• Periphery is distorted
• k2 0 is good enough

7
Pinhole Camera
f
Z
Optical centre
Object
Image
Image and centre, object and centre are similar
triangles.
8
Camera and World Co-ordinates
translate and rotate
9
System architecture
Start
End
10
Target acquisition

• Finding a target to follow
• Differencing
• Moving edge detector

11
Change and Moving Object Detection

• Simplest method of detecting change
• Compute differences between
• Live and background images
• Adjacent images in a sequence

12
Image Differencing

• Differences due to
• Moving object overlying static background
• Moving object overlying another moving object
• Moving object overlying same moving object
• Random fluctuation of image data

13
(No Transcript)
14
Demo

• Inter frame differencing
• Difference from a background
• pFinder

15
Background image

• Detecting true differences required an accurate
  background
• Lighting changes?
• Camera movement?

16
Background image updates

• Periodically modify whole background
• Will include changes in new background
• Systematically incorporate non-changed portions
  of image into background

17
Critique

• Can identify changes in the image data
• But what do the changes mean?
• Need a second layer of processing
• To recognize changes
• Optical flow sidesteps this problem...

18
Target following

• Observing the positions of an object or objects
  in a time sequence of images.
• Object matching
• Minimum path curvature
• Model based methods

19
Matching

• Locate objects in each image
• Match objects between images
• Use methods of previous lectures

20
Minimum path curvature

• Observations of two objects in three images

21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
Which is best solution?

• One with overall straightest paths
• For each solution
• For each path
• Compute total curvature
• Sum

26
Model based tracking

• Mathematical model of objects motions
• position, velocity (speed, direction),
  acceleration
• Can predict objects positions

27
System overview
Motion Model
Predict Location
Update?
Verify Location
28
Simple Motion Model

• Newtons laws
• s position
• u velocity
• a acceleration
• all vector quantities
• measured in image co-ordinates

29
Prediction

• Can predict position at time t knowing
• Position
• Velocity
• Acceleration
• At t0

30
Uncertainty

• If some error in a - Da or u - Du
• Then error in predicted position - Ds

31
Verification

• Is the object at the predicted location?
• Matching
• How to decide if object is found
• Search area
• Where to look for object

32
Object Matching

• Compare
• A small bitmap derived from the object vs.
• Small regions of the image
• Matching?
• Measure differences

33
Search Area Why? and Where?

• Uncertainty in knowledge of model parameters
• Limited accuracy of measurement
• Values might change between measurements
• Define an area in which object could be
• Centred on predicted location, s ? Ds

34
Update the Model?

• Is the object at the predicted location?
• Yes
• No change to model
• No
• Model needs updating
• Kalman filter is a solution

35
Kalman filter

• Mathematically rigorous methods of using
  uncertain measurements to update a model

36
Kalman filter notation

• Relates
• Measurements yk
• e.g. positions
• System state xk
• Position, velocity of object, etc
• Observation matrix Hk
• Relates system state to measurements
• Evolution matrix Ak
• Relates state of system between epochs
• Measurement noise nk
• Process noise vk

37
Mathematically
How do observations relate to model?
38
Prediction of System State

• Relates system states at epochs k and k1

39
Prediction of Observation

• From predicted system state, estimate what
  observation should occur

40
Updating the Model

• Predict/estimate a measurement
• Make a measurement
• Predict state of model
• How does the new measurement contribute to
  updating the model?

41
Updating the Model

• G is Kalman Gain
• Derived from A, H, v, n.

42
Example

• Tracking two corners of a minimum bounding box
• Matching using colour
• Image differencing to locate target

43
Condensation Tracking

• So far considered single motions
• What if movements change?
• Bouncing ball
• Human movements
• Use multiple models
• plus a model selection mechanism

44
Selection and Tracking

• Occur simultaneously
• Maintain
• A distribution of likely object positions plus
  weights
• Predict
• Select N samples, predict locations
• Verify
• Match predicted locations against image
• Update distributions

45
Tracking Using Hidden Markov Models

• Hidden Markov model describes
• States occupied by a system
• Possible transitions between states
• Probabilities of transitions
• How to recognise a transition

46
Optic Flow

• Assume each pixel moves but does not change
  intensity
• Pixel at location (x,y) in image 1 is pixel at
  (xDx,yDy) in image 2.
• Optic flow associates displacement vector with
  each pixel

47
(No Transcript)
48
Aperture Problem

• ?I/ ? x ? horizontal difference
• ? I/ ? y ? vertical difference
• ? I/ ? t ? difference between images
• One equation, two unknowns
• ? cannot solve equation
• Could solve for movement perpendicular to gradient

49
Solution

• Impose smoothness constraint
• Minimise total of sum of squares of velocity
  component gradients

50
l is a constant Iterate over a pair of frames,
or over a sequence
51
Critique

• Assumptions
• Linear variation of intensities
• Velocity changes smoothly
• These are invalid
• Especially at object boundaries

52
Area Based Methods

• Match small regions in image 1 with small regions
  in image 2
• Assume objects move but do not deform
• Same formulation as for optical flow, different
  areas involved.

53
Matching

• Compute (Dx, Dy) by finding (u,v) that minimises
• Then (u,v) (Dx, Dy)

54
Summary

• Target acquisition
• Image differencing
• Background model
• Target following
• Matching
• Minimum path curvature
• Model based methods
• Optic flow

55

• This telephone has too many shortcomings to be
  seriously considered as a means of communication.
  The device is of no value to us.
• Western Union internal memo, 1876