MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD INFRARED CAMERA

1
MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO
LANDING USING AN ONBOARD INFRARED CAMERA

• Dr. Gerard Medioni
• Cheng Hua Pai
• Yu Ping Lin

2
Introduction

• Input Infrared runway sequence
• Goal Detect moving objects on runway

3
Approach

• We do it in two steps
• Stabilize the sequence
• Detect motion on the stabilized sequence

4
Flow chart of the system
Video In
Reference Frame
Runway Identification
Image Stabilization
Update Reference frame
Motion Detection
Yes
Locally Stabilized Image Sequence and
Homographies
Update Reference frame?
Blobs in motion
5
Stabilization

• Issues
• Planar region containing the runway
• Feature choice and matching
• Transformation between consecutive frames

6
Stabilization

• Approach
• Manually Label planar region
• SIFT provides sufficient and descriptive features
• RANSAC to estimate best transformation

7
Stabilization

• Result

Stabilized runway sequence
8
Adaptive Reference Frame

• Issues
• For longer sequence
• Small errors accumulate
• Big scale difference

Beginning of a Sequence
End of a Sequence
9
Adaptive Reference Frame

• When to change reference frame?
• Check the lower edge length ratio

10
Stabilization algorithm
Landing UAV image sequence
Manually labeled planar region
input

1. Use RANSAC to remove outliers and estimate
   homography

1. Extract SIFT features

1. Region of Interest

1. Update reference frame if necessary

1. Match features to previous frame to establish
   correspondence

1. Warp to the reference frame

output
Locally stabilized image sequence and
for all s
11
Adaptive Reference Frame

• Result

Original Sequence
Locally Stabilized Sequence
12
Detection module

• Issues
• Detection method
• Global intensity variation
• Noise
• Moire in the sequence
• Poor stabilization
• Local intensity variation
• Random noise

13
Detection

• Approach
• Use simple Gaussian background model
• ?t (1-?) (?t-1) ? (It)
• ?t2 (1-?) (?t-1) 2 ? (It- ?t)
• Foreground More than 4?t2 from mean

Foreground
Background
4?t2
4?t2
?t
Intensity distribution of an image
14
Global intensity variation

• Approach
• Compensate gain with affine transformation
  Yalcin 05

Before compensation
After compensation
15
Noise reduction

• Approach
• Moire in the sequence
• Compare 8 neighbouring background pixels
• Poor stabilization
• Restabilize with gradient map (also SIFT)

To Gradient
16
Noise reduction

• Approach
• Local intensity variation
• Intensity normalization on the foreground pixels
• Random noise
• Compare consecutive foreground masks

With random noise
Without random noise
17
Detailed flow chart of Motion Detection Module
18
Detection Result

• Result

Foreground mask
Locally Stabilized Sequence
19
Evaluation

• Tested on 150 synthesized and 18 real-world
  sequences
• Results (synthetic data)

Obj. size
20
Conclusion

• Detection affected by
• Object speed and size
• Threshold parameters
• Program limitation
• Moving objects fade in and out
• Bad result near the end of the sequence
• Future work
• More test on larger dataset
• Speed improvement

21
Reference

• W. G. Chris Stauffer. Adaptive background mixture
  models for real-time tracking. 1999 IEEE Computer
  Society Conference on Computer Vision and Pattern
  Recognition (CVPR'99), 22246, 1999.
• M. Fischler and R. Bolles. Random sample
  consensus A paradigm for model fitting with
  applications to image analysis and automated
  cartography. Communications of the ACM,
  24(6)381-395, 1981.
• C. Harris and M. Stephens. A combined corner and
  edge detector. Proceedings of The Fourth Alvey
  Vision Conference, (1)147-152, 1988.
• T. G. R. Kasturi, O. Camps and S. Devadiga.
  Detection of obstacles on runway using ego-motion
  compensation and tracking of signicant features.
  Proceedings 3rd IEEE Workshop on Applications of
  Computer Vision, 1996 (WACV'96), pages 168-173,
  1996.
• D. G. Lowe. Distinctive image features from scale
  invariant keypoints. International Journal of
  Computer Vision, 60(2)91-110, 2004.
• R. S. B. Sridhar and B. Hussien. Passive range
  estimation for rotor-craft low-altitude flight.
  Machine Vision and Applications, 6(1)10-24,
  1993.
• S. Sull and B. Sridhar. Runway obstacle detection
  by controlled spatiotemporal image Low disparity.
  IEEE Transactions on Robotics and Automation,
  15(3)537-547, 1999.
• R. C. H. Yalcin and M. Hebert. Background
  estimation under rapid gain change in thermal
  imagery. Second IEEE Workshop on Object Tracking
  and Classification in and Beyond the Visible
  Spectrum (OTCBVS'05), 2005.
• Q. Zheng and R. Chellappa. Motion detection in
  image sequences acquired from a moving platform.
  Proc. Int. Conf. Acoustics, Speech, and Signal
  Processing, Minneapolis, 5201-204, 1993.