Landing a UAV on a Runway Using Image Registration

1
Landing a UAV on a Runway Using Image Registration

• Andrew Miller, Don Harper, Mubarak Shah
• University of Central Florida
• ICRA 2008

2
Overview

• System for landing a UAV on a runway
• Small RC airplane
• Only sensor is a fixed, forward-looking camera
• Finds the runway using SIFT registration
• Linear control system
• Experiments
• Microsoft Flight Simulator (no flight model)
• Partial implementation on a real UAV

3
Block Diagram
30 frames per second 720x480 pixels
RGB Downsample to 360x240
4
Main Steps

• Locate the runway in each video frame
• Estimate the attitude of the UAV
• Steer the UAV towards the runway maintaining the
  correct glideslope

5
1. Locate the Runway

• Base point and vanishing point
• (location and orientation)

6
Planar Homography

• The 3x3 planar homography matrix projects every
  point in the reference frame to the corresponding
  point in the incoming video frame

Reference Frame
7
Find the Homography usingSIFT and RANSAC

• SIFT Feature Matching
• 200-500 feature points, 100-200 matches
• Chosen greedily, least ambiguous first
• Planar homography between correspondences
• RANSAC to discard outliers

8
Stack of Reference Frames

• Prepare a reference frames from a video
• Annotate the runway and vanishing point
• Sample the frames (more samples at lower
  altitudes)
• Terrain features are important (not just the
  runway)

9
Stack of Reference Frames
10
Using the Stack

• Keep track of the current index
• Highest number of SIFT matches most similar
  viewpoint
• Only need to compare adjacent frames

11
2. Estimate the UAV Attitude

• 6 Degrees of Freedom
• Pitch, Bank, Heading, Elevation, Distance, Course
• Strategy
• Ignore Distance
• Find Pitch and Bank from the horizon line
  (x-axis)
• Find Elevation, Heading, Course from the runway

12
Intuitive Geometry

• Relationship between runway appearance and UAV
  attitude
• This is how human pilots land visually

13
Formal Geometry

• 3D Projection
• C Internal Calibration
• R External Calibration
• Small Angle Approximation
• Assume the UAV is flying smooth and level

14
2. Estimate the UAV Attitude

• Recover the orientation parameters
• Vanishing point of the runway
• Beginning of the runway

15
Find the Horizon

• Horizon estimation algorithm by Ettinger, et al.
• Based on Differing Color Distributions
• Used to recover two (pitch / bank)

16
3. Control the UAV

• Cascaded Linear Feedback Controller
• Two separate chains
• Two gains
• Proportional
• Integral
• Intuitive
• If UAV is too far right, steer left
• If UAV is too high, pitch down
• Bank angle is derivative of heading, heading is
  derivative of course
• Pitch is derivative of elevation

17
Autopilot GUI
18
Algorithm Performance

• Multiple stages
• Control loops run at 50 Hz
• Integrates smoothly even while input stays same
• Horizon detection runs at 10 Hz
• Pitch and bank are the most sensitive
• Runway detection runs at 2 Hz
• Elevation and course are the least sensitive

19
Simulator Results

• Microsoft Flight Simulator
• Simulator-in-the-loop (separate computer)
• ICRA08_1140_VI_fi.mp4
• Horizon 2007 09 Sep 11 Tue 08.13pm.avi

20
Simulator Results

• Error from earth curvature

21
Actual UAV Experiments

• Only using partial implementation
• Horizon stabilization
• Road following (no runway available)
• Only brief periods of autonomous control

22
Horizon Stabilization - Results
23
Conclusions

• Successful but imprecise landings
• Performance is applicable to Cessna
• Slower and more stable than actual UAVs
• Assumption of linear system is not applicable
  near the runway
• This is why the aircraft oscillates before
  landing
• Future work
• Incorporate flight model into controller design