CS223b Problem Session 3: Programming Assignment 3: Motion Estimation

1
CS223b Problem Session 3Programming Assignment
3Motion Estimation

• Greg Corrado

2
Details

• Due Wednesday, February 15th, 1159PM
• Late days okay.
• Submit to cs233bsubmit_at_gmail.com
• Where is the data?
• http//cs233b.stanford.edu/homework/hw3/video
• Available as .zip, .tgz, .sit -gt .jpg
• (The .avi, .dv, and .mov versions are for
  amusement only.)

3
Some words of caution

• This assignment has never been successfully
  completed by anyone.
• This is a hard problem and absolute performance
  levels will be lower than on Assignment 1.
• You have a limited amount of time to get
  something working - if it seems like its going
  well you can extend it for the competition.

4
The big picture
ltmovie file heregt
5
Some quirks about the data

• Image duplication

6
Some quirks about the data

• Image duplication (my bad)

7
(No Transcript)
8
Some quirks about the data

• Image duplication (will be fixed tomorrow)

9
Some quirks about the data

• Image duplication (will be fixed tomorrow)
• Field aliasing / interlacing.

10
Mouse Teeth
11
About fields
12
About fields
13
About fields
14
What to do about it?

• (1) Nothing - maybe your algorithm doesnt care.
• (2) Blur/Downsample
• This is called field blending
• Ideally use more smoothing in Y than X
• (3) Analyze Fields separately, and combine the
  results at the end.

15
How do I approach this assignment?
16
Two possible approaches

• Motion then Cars
• Cars then Motion

17
Motion then Cars

• Build an optical flow map.
• Guess what the map should look like if there were
  no moving objects.
• Look for outliers.
• Cull the set of pixels down to those which are
  probably cars.

18
Cars then Motion

• Find pixels which are part of a car in the target
  frame.
• Then look back and guess if those pixels were in
  fact moving in an unusual way.

19
In either case

• Look for features and track them.
• (e.g. Corners/Nearest Neighbors or SIFT
  Features/Kalman Filters)
• -OR-
• Ignore features and go right for pure flow.
• (e.g. Luminance Constancy, Image Pyramids, Ego
  motion flow models)

20
We cant tell you which will work better.

• Not just because were mean-spirited.
• Which we are.
• But because we really dont know.
• Dan and I will both be writing our own solutions
  to this as you do. (Dan in OpenCV, Me in MATLAB.)

21
MATLAB use.

• I would not recommend using MATLAB for this
  assignment if you do not have access to the Image
  Processsing Toolbox
• In addition, the Video and Image Processing
  Blockset is pretty cool too if you like Simulink.
  
• Use the helpwin command to browse packages and
  functions
• Use the lookfor command to search for a keyword
  in all available help headers. (There is
  information beyond this in helpwin but its a
  good way to poke around.

22
Some Open CV Functions

• These are the OpenCV functions that are most
  relevant to this assignment-
• cvCalcOpticalFlowHS Calculate optical flow
  for two images-
• cvCalcGlobalOrientation Calculate global
  motion orientation of selected region-
• cvSegmentMotion Segments whole motion into
  separate moving parts-
• cvUpdateMotionHistory Keep track of motion
  history by feeding silhouette image

23
Some Open CV Functions

• OpenCV Secondary functions of note
• Secondary functions-
• cvCornerHarris Harris corner detector-
• cvGoodFeaturesToTrack Find the n strongest
  corners in the region of interest-
• cvLogPolar Remap image to log-polar space
  (could be used for perspective modeling)-
• cvMatchTemplate Compare template against
  overlapped image regions- RunningAvg Update a
  running average image that could be used for

24
Some Open CV Functions

• cvHoughLines2 Hough transform to find lines-
• cvThreshold Apply fixed-level threshold to
  image

25
Using other peoples code.

• Unrestricted. but
• Your write up MUST include
• The name of the tool.
• Who wrote it.
• Where you downloaded it from.
• What you used it to accomplish.

26
Good luck...