Static Environment Recognition Using Omnicamera from a Moving Vehicle

1
Static Environment Recognition Using Omni-camera
from a Moving Vehicle

• Teruko Yata Chuck Thorpe Frank Dellaert

Presented by Ty Russell ECE 285 Feb. 12, 2003
2
What are they trying to do?

• This research aims to develop a sensor system
  which will be able to recognize city area
  environments from a moving vehicle.

Why?

• A city area is a rather cluttered outdoor
  man-made environment.
• Past methods focus on indoor and highway
  environments which are visually simple.
• Another step towards a driver assistance system

3
Hardware
NavLab11

• 2000 Jeep Wrangler Sport
• Encoder data obtained from counting four 51
  slots/wheel rotation gears used with the
  anti-locking braking system
• Odometry, X and Y location and heading direction
  of vehicle in world coordinate

4
Hardware
Omni-directional camera ?

• 360 degrees view in a single
• frame
• Black and white
• Only top 180 degrees used, trimmed front of
  vehicle and sky
• Connected to a digitizer on a PC
• 1 degree in the view image corresponds to 2
  pixels in the fixed image

5
Strategy

• Pre-process of Omni-image
• Landmark Selection
• Template Tracking
• Angle Measurement
• Landmark Location Calculation on 2D Map

6
Pre-process of Omni-image

• Image is transformed from forward-looking 180
  degree into cylindrical image
• When there is no corresponding pixel, pixel is
  interpolated by values around pixel

7
Landmark Selection

• Vertical continuous edges are selected as
  landmarks to calculate optical flow
• Sobel filter used to find straight lines
• Continuous groups of pixels with a specified
  minimum of pixels make a landmark

8
Template Tracking

• Optical flow is calculated using templates of
  following frames
• Templates are extracted from original image based
  on rectangular area of landmark
• Initial template is calculated from estimated
  position of point being evaluated
• Odometry data for each image helps predict where
  the point should be

9
Template Tracking

• After selecting the landmark, it is tracked and
  the template is updated at each frame
• If average difference reaches a threshold,
  tracking stops for that landmark
• A new landmark is defined if there are no other
  landmarks around it

10
Angle Measurement and 2D Map

• Each tracked template gives bearing angle to
  create 2D map
• Bearing angle used for calculating location of
  landmarks
• From bearing angle, a straight line of sight to
  tracked landmarks is drawn at each position, and
  the crossing points are the resultant landmark
  location in 2D
• Extended Kalman filter for estimating these
  locations in 2D

11
Bearing Measurements

• Landmark X (x,y)
• As seen from vehicle V (a,b,c)
• c is heading of vehicle in radians
• Bearing z to landmark z h(x,y) n
• n is Gaussian noise with standard deviation s
• Jacobian H

12
Kalman Filter

• Z is a vector valued measurement with noise
  variance R, X is the state and (XO,P) is the mean
  covariance of the predicted state
• X XO ?X Replace non-linear measurement
  function by its linear approximation

13
Kalman Filter
14
Integrating Bearing Measurements

• These can be plugged in straightforwardly in (2)
  and (3) to do bearing update
• Initial estimated location is defined as line of
  sight crossing points using the first and second
  bearing information of tracked template.
• This estimation is repeated and appearance,
  template, and feedback of the estimated location
  is updated for the next template of the next image

15
Experiment

• The vehicle was driven about 15 meters in a
  parking lot
• Odometry data and 85 frames were acquired from
  the driving

16
Experiment
Original
Lines by Sobel filter
Detected Landmarks
17
Experiment
Tracked template (1st frame)
Tracked template (2nd frame)
18
Experiment
Tracked template (1st frame)
Tracked template (2nd frame)
19
Experiment
20
Experiment
21
Results

• Shadows or marks are sometime detected as
  objects.
• Buildings and cars were detected well, however,
  bicycle, human or other objects should be
  considered
• New setup has camera on corner of vehicle to get
  a 270 degree view