Automated Rivet Inspection System for Aging Aircrafts

1
Automated Rivet Inspection System for Aging
Aircrafts

• Unsang Park, Lalita Udpa, George C. Stockman
• Computer Science and Engineering
• Michigan State University

2
Contents

• Nondestructive Inspection (NDI)
• Magneto-optic Imager in NDI
• Motion-based Filtering (MBF)
• Real-time implementation of MBF
• Automated rivet inspection system
• Rivet detection
• Rivet classification
• Results and conclusions
• Future work

3
Nondestructive Inspection for Aircrafts

• Detect subsurface defects

Seam
Rivet
Crack
4
Nondestructive Inspection for Aircrafts

• Increase service life of airplane
• Prevent disasters

Aloha Airlines B-737-200 lost part of its front
fuselage during a flight in Hawaii, 1985
5
Magneto-optic Imager (MOI)

• Eddy current excitation
• Magneto-optic sensing
• Imaging

6
Magneto-optic Imager (cont.)

• Produce real-time analog images of inspected part
• Images both surface breaking and subsurface
• cracks
• Easy to interpret with minimal training
• Applicable both on conducting samples as well as
• composites by tagging with ferromagnetic
• particles

7
Sample MOI Images
Crack along seam
Crack between two rivets, Radial crack on a rivet
Corrosion dome
8
Drawback of MOI images

• MOI image contains serpentine pattern
• noises due to the magnetic domain walls
• in magneto-optic sensor

Signals due to domain walls
Rivet
9
Motion-based Filtering (MBF)

• Additive Frame Subtraction

In-5
In-4
In-3
In-2
In-1
In
D1
D2
D3
D4
D5
?
Sn
10
Motion-based Filtering (cont.)
11
MB filtered images
Filtered
Original
12
MB filtered images (cont.)
Filtered
Original
13
Real-time implementation of MBF

• Experimental setup for proof of concept

Play on a Video player
Record to movie file
noise
Play on a PC monitor
14
Real-time implementation of MBF (cont.)

• Data transfer rate
• 4.6 Mbytes /sec ( 320?240 pixels ? 16 bit ? 30
  fps )
• Data are down sampled as the input images are
  dropped while an image is processed
• Diagram of real-time Motion-based Filtering

15
Real-time implementation of MBF (cont.)

• Optimizing MBF algorithm in C

16
Real-time implementation of MBF (cont.)

• RGB to Gray conversion
• Additive frame subtraction
• MAX(I1-I3,I2-I3) ? MAX(I1,I2) - I3
• Median filter

Normal Table Lookup Table build time
320240 16 bit image 2023 ms 12 ms 56 ms (1M bytes)
5 by 5 7 by 7
MATLAB 50 ms 88 ms
Modified Quick Sort (C) 200 250 ms 400 450 ms
Moving Median with Sorting (C) 100 150 ms 150 200 ms
Moving Median with Histogram (C) 20 25 ms 20 40 ms
17
Real-time implementation of MBF (cont.)

• Execution time of MBF algorithm in C

Before optimization Intel 2GHz, C After optimization Intel 2GHz, C
Capture 20 ms 20 ms
RGB to gray 20 23 ms 1 2 ms
Subtraction (x10) Max (x10) Threshold 1 2 ms (x10) 1 2 ms (x10) 12 ms 1 2 ms (x10) 1 2 ms
Median Filter (3x3) 200 250 ms 20 30 ms
Stretch 1 2 ms 1 2 ms
Total 262 337 ms 53 76 ms
18
Drawbacks of current MOI inspection

• No measure for quantitative interpretation
• Data interpretation is subjective
• Manual inspection by human operator (more than 10
  hours per airplane)
• Expensive labor cost
• Error due to fatigue

19
Automated MOI inspection system

• PRI Research and Development Corporation (PRI)
• Developing and improving magneto-optic imager
• (MOI)
• Michigan State University, ECE department
• Image processing algorithm for filtering and
• classification
• Boeing Phantom Works
• Self-guided, suction cup robot crawls over
  airplane
• skin

20
Automated MOI inspection (cont.)

• Currently focusing on radial cracks on rivets
• Quantification of defects in MOI images
• Implementing real-time rivet inspection algorithm

21
Rivet detection

• Hough transformation-based method
• Circular Hough transformation
• Morphological operation-based method

22
Rivet detection (cont.)
Hough transformation
Original
Morphological operation
23
Rivet classification

• Two-pass Hough transformation
• 1st pass Rivet detection
• 2nd pass Blob detection

good
bad
Original image
Filtered image - After 1st pass
After 2nd Pass
24
Rivet classification (cont.)

• Bayesian classifier
• Feature selection

Hough transformation
Morphological op.
Original image
25
Off-line test

• Training
• 10 normal, 10 defective rivet images
• Obtain mean and variance of feature f1
• Testing
• 222 rivet images including 66 defective rivet
  images

Two-pass Hough
Morph. - Bayes
Hough - Bayes
26
Experimental Results

• Accuracies of three algorithms

Inspection algorithm Rivet detection Rivet detection Rivet classification
Inspection algorithm False negative False positive Rivet classification
Two-pass Hough 1/242 0/242 90 (200/222)
Hough-Bayes 1/242 0/242 96 (214/222)
Morph.-Bayes 1/242 1/242 99 (220/222)
27
Conclusions

• MB filtered image is optimal in image processing
  for automated rivet inspection
• Morphological operation-based rivet detection is
• superior to Hough-based rivet detection both for
  
• execution time and accuracy
• Bayesian classifier is superior to Hough-based
• classifier
• Radial crack detection on rivets showed 99
• accuracy in off-line test

28
Future work

• Implement MBF and rivet inspection algorithms on
  the Digital Signal Processing (DSP) board
• Improve robustness of the algorithms with the
  feedback from field test
• Develop MOI inspection algorithms for other
  types of defects in aircrafts