Stereoscopic%20Imaging%20for%20Slow-Moving%20Autonomous%20Vehicle

1
Stereoscopic Imaging for Slow-Moving Autonomous
Vehicle
Senior Project Progress Report Bradley
University ECE Department

• By Alex Norton
• Advisor Dr. Huggins
• February 28, 2012

2
Presentation Outline

• Review of Proposed Project
• Project Overview
• Original Proposed Schedule
• Tasks Completed
• Webcams setup
• Calibration mode software
• Remaining Tasks
• Run mode software
• Improve existing software
• Revised Schedule

3
Project Overview

• Two horizontally aligned, slightly offset cameras
  taking a pair of images at the same time
• By matching corresponding pixels between the two
  images, the distances to objects can be
  calculated using triangulation
• This depth information can be used to create a
  3-D image and terrain map

4
Original Proposed Schedule
Tentative Schedule for Spring 2012 Tentative Schedule for Spring 2012 Tentative Schedule for Spring 2012
Weeks Alex Norton Matthew Foster
1 Assemble camera setup Assemble camera setup
2 Configure calibration rig Ensure OpenCV runs correctly on lab computers
3 Begin writing OpenCV code for calibration mode Begin writing OpenCV code for run mode
4 Continue writing OpenCV code for calibration mode Continue writing OpenCV code for run mode
5 Continue writing OpenCV code for calibration mode Continue writing OpenCV code for run mode
6 Continue writing OpenCV code for calibration mode Continue writing OpenCV code for run mode
7 Test and debug calibration mode code Continue writing OpenCV code for run mode
8 Test and debug calibration mode code Continue writing OpenCV code for run mode
9 Test run mode code with calibrated cameras Test run mode code with calibrated cameras
10 Debug calibration mode code Debug run mode code
11 Debug calibration mode code Debug run mode code
12 Test and debug complete computer vision code Test and debug complete computer vision code
13 Test and debug complete computer vision code Test and debug complete computer vision code
14 Prepare for final presentation Prepare for final presentation
5
Tasks Completed

• Webcams setup
• Creates capture objects for both webcams
• Takes a set of images each time the enter key
  is pressed
• Displays the saved set of images in two windows
• Saves the images to a specified folder to use for
  further image processing

6
Webcams Setup Output
7
Necessity of Calibration

• Produces the rotation and translation matrices
  needed to rectify sets of images
• Rectification makes the stereo correspondence
  more accurate and more efficient
• Failing to calibrate the cameras is a possible
  reason for why past groups have failed to get
  accurate results

8
Calibration Mode Software

• Input is a list of sets of images of a
  chessboard, and the number of corners along the
  length and width of the chessboard
• Read in the left and right image pairs, find the
  chessboard corners, and set object and image
  points for the images where all the chessboards
  could be found
• Given this list of found points on the chessboard
  images, the code calls cvStereoCalibrate() to
  calibrate the cameras

9
Calibration Mode Software

• This calibration gives us the camera matrix M and
  the distortion vector D for the two cameras it
  also yields the rotation matrix R, the
  translation vector T, the essential matrix E, and
  the fundamental matrix F
• The accuracy of the calibration is assessed by
  checking how nearly the points in one image lie
  on the epipolar lines of the other image

10
Calibration Mode Software

• The code then moves on to computing the
  rectification maps using Bouguets method with
  cvStereoRectify()
• The rectified images are then computed using
  cvRemap()
• The disparity map is then computed by using
  cvFindStereoCorrespondenceBM()

11
Calibration Mode Software

• Two methods for stereo rectification
• Hartleys Method uses the fundamental matrix,
  does not require the cameras to be calibrated,
  produces more distorted images than Bouguets
  method
• Bougets Method uses the rotation and
  translation parameters from two calibrated
  cameras, also outputs the reprojection matrix Q
  used to project two dimensional points into three
  dimensions

12
Calibration Mode Software Matrices

• Rotation matrix R, Translation Vector T
  extrinsic matrices, put the right camera in the
  same plane as the left camera, which makes the
  two image planes coplanar
• Fundamental matrix F intrinsic matrix, relates
  the points on the image plane of one camera in
  pixels to the points on the image plane of the
  other camera in pixels

13
Calibration Mode Software Matrices

• Essential Matrix E intrinsic matrix, relates the
  physical location of the point P as seen by the
  left camera to the location of the same point as
  seen by the right camera
• Camera matrix M, distortion matrix D intrinsic
  matrices, calculated and used within the function

14
Calibration Mode Software
Example of bad chessboard image
15
Calibration Mode Software
Output when bad chessboard images are run through
the calibration software
16
Calibration Mode Software
Example of good chessboard image
17
Calibration Mode Software
Output when good chessboard images are run
through the calibration software
18
Remaining Tasks

• Use triangulation to determine distances to
  objects
• Calculate the error in the distance measurements
• Minimize the error in both the camera calibration
  and the distance measurements

19
Revised Schedule
Schedule for Spring 2012 Schedule for Spring 2012 Schedule for Spring 2012
Weeks Alex Norton Matthew Foster
7 Test and debug calibration mode code Test and debug calibration mode code
8 Test and debug calibration mode code Test and debug calibration mode code
9 Write OpenCV code for run mode Write OpenCV code for run mode
10 Write OpenCV code for run mode Write OpenCV code for run mode
11 Test and debug run mode code Test and debug run mode code
12 Test and debug run mode code Test and debug run mode code
13 Test and debug complete computer vision code Test and debug complete computer vision code
14 Test and debug complete computer vision code, prepare for final presentation Test and debug complete computer vision code, prepare for final presentation
20
Questions??