Semi-automatic Range to Range Registration: A Feature-based Method

1
Semi-automatic Range to Range Registration A
Feature-based Method
Chao Chen Ioannis StamosComputer Science
DepartmentGraduate Center, Hunter CollegeThe
City University of New York
2
Motivation

• Goal highly accurate photo-realistic description
  of 3D world
• Applications urban planning, historical
  preservation, virtual reality

3D model
Photo-realistic 3D Model
Highly accurate
Range scans
Computational efficient
Minimum human interaction
2D Pictures
3
Contribution

• Related methods on range image registration
• Iterative Closest Point algorithm Besl McKay,
  Chen Medioni, Rusinkiewicz
• Require close initial registration
• Spin images Johnson Hebert
• More suitable for curved surfaces needs accurate
  normal
• Previous feature-based algorithm Stamos
  Leordeanu
• Exhaustively searching line pairs high
  complexity
• Symmetric structures require manual registration
• Our semi-automatic 3D registration system
• No rough pre-registration required
• Automated registration procedure
• Utilize global information to compute
  transformation
• ICP algorithm to optimize registration
• Context-sensitive user interface to
• Display registration result at each step
• Conveniently adjust translation and rotation

4
Outline

• Automated registration procedures
• Previous exhaustive search approach
• Improved automated registration
• Global stitching process to register all images
• User interface
• Experimental results
• Conclusions and future work

5
Exhaustive Search Approach

• Range image segmentation

Intersection line
Each Segmented Planar Area is shown with
different color for clarity
Range sensing direction
Segmented Planar Area Exterior and Interior
borders shown
Interior border
6
Exhaustive Search Approach

• One pair of correctly matched lines provides
  rotation

Line in left range image
Line in right range image
Plane normal in right image
Plane normal in left image
xright
xleft
zleft
yright
zright
yleft
Right range image coordinate system
Left range image coordinate system
7
Exhaustive Search Approach

• Two pairs of correctly matched lines provide
  exact translation

xright
xleft
zright
zleft
yright
yleft
Rotated left range image coordinate system
Right range image coordinate system
8
Exhaustive Search Approach

• Find the two pairs of corresponding lines that
  maximizes the total number of line matches
• Consider two corresponding line pairs
• Compute transformation
• Grade of computed transform total number of line
  matches
• Keep the transform with the highest grade
• At the end refine best transform using all
  matched lines

White lines (left scan) Blue lines
(right scan) Red/Green lines (matches)
9
Exhaustive Search Approach

• No initial registration needed
• High computational complexity
• Symmetry problem unsolved
• Improvements
• Extract object-based coordinate system
• Context-sensitive user interface

10
Framework of New Solution
Lines and planes from segmentation
Correct registration
Next pair of scans
Global stitching
Automated Registration
User Interactions
11
Line Clustering

• Line clustering
• Line directions
• Plane normals

Buildings local coordinate system
12
Rotation Estimation

• Rotation estimation

R x2 y2 z2 T x1 y1 z1
24 possible Rs?
13
Rotation Estimation

• Heuristic eliminate candidates based on
  observations
• Scanner moves on the ground plane y axis not
  change much
• Overlapping images from close by viewpoints
  smallest rotation candidate is chosen

14
Translation Estimation

• Translation estimation
• Left and right axes parallel accordingly after
  rotation
• Pick robust line pairs to estimate translation

R
15
Translation Estimation

• One pair of matched lines provides an estimated
  translation
• Two pairs with similar estimated translations
  provide translation candidate

16
Translation Estimation

• Two types of translation candidates

y1
y2
z2
z1
x2
d1
x1
d2
17
Translation Estimation

• Find the translation that maximizes the total
  number of line matches

• Cluster all estimated Ts, pick 10 most
  frequently appeared Ts
• For each T
• Find all matches, solve linear system to update
  RT
• Count matched line pairs again
• Choose the RT with the most number of matched
  line pairs
• Refine transformation with ICP

18
Registration System Flowchart
start
Read in all image pairs, form the transformation
graph
Read in one image pair
Y
Find pivot image, compute path
Last pair?
N
Automated registration
Compute transform from each image to pivot image
Display to user
1
2
3
Need manual adjustment
Rotation wrong by 90
correct
Global optimization
Display draggers
Display other rotations
save
Global stitching
Adjust R/T, optimize
Choose a new R, compute T
Display and save
exit
save
19
User Interface
Display window Points and lines of registered
two scans
20
User Interface
start
Read in all image pairs, form the transformation
graph
Read in one image pair
Y
Find pivot image, compute path
Last pair?
N
Automated registration
Compute transform from each image to pivot image
Display to user
3
1
2
Need manual adjustment
Rotation wrong by 90
correct
Global optimization
Display draggers
save
Display other rotations
Global stitching
Adjust R/T, optimize
Choose a new R, compute T
Display and save
exit
save
21
User Interface

• Rotation wrong by 90 degrees choose from other
  candidate rotations computed previously

22
User Interface
start
Read in all image pairs, form the transformation
graph
Read in one image pair
Y
Find pivot image, compute path
Last pair?
N
Automated registration
Compute transform from each image to pivot image
Display to user
1
2
3
Rotation wrong by 90
Need manual adjustment
correct
Global optimization
save
Display other rotations
Display draggers
Global stitching
Adjust R/T, optimize
Choose a new R, compute T
Display and save
exit
save
23
User Interface

• Adjusting rotation and translation based on the
  buildings coordinate system
• White draggers
• for translation
• Blue spheres
• for rotation

24
User Interface
Display window Points and lines of registered
two scans
Exhaustive Search Approach
ICP Optimization
25
Global Stitching
start
Read in all image pairs, form the transformation
graph
Read in one image pair
Y
Find pivot image, compute path
Last pair?
N
Automated registration
Compute transform from each image to pivot image
Display to user
1
2
3
Rotation wrong by 90
Need manual adjustment
correct
Global optimization
Display draggers
save
Display other rotations
Global stitching
Adjust R/T, optimize
Choose a new R, compute T
Display and save
exit
save
26
Global Stitching

• Global stitching for all images
• Pivot image the image with the most number of
  neighbors
• Transform composition along the strongest path
  from each image to the pivot image
• Further improvement to consider
• Global optimization to minimize registration error

27
Experimental Results

• Thomas Hunter building, Hunter College
• 14 scans, 15 pairs (13 automated, 2 manually
  adjusted)
• 1020 seconds per pair a few minutes for entire
  registration

28
Registration Results

• Shepard Hall, City College
• 20 scans, 24 pairs (9 automated, 8 R symmetry, 7
  adjust R/T)
• 2090 seconds per pair 1 hour for entire
  registration

video
29
Registration Results

• Great Hall (interior of Shepard Hall)
• 21 scans, 44 pairs (12 automated, 18 R symmetry,
  13 adjust R/T )
• 2090 seconds per pair 1.5 hour for entire
  registration

30
Registration Results

• Great Hall interior scene

31
Registration Results

• Great Hall interior scene

32
Registration Results

• Great Hall interior scene

33
Algorithm Performance
Time for automated registration
Average error of matching planes
Number of matching lines
Lines in two scans
34
Algorithm Performance
Thomas Hunter building (rectangular) Shepard Hall (intricate) Great Hall (much symmetry)
Number of scans Number of scans 14 20 21
Number of pairs Number of pairs 15 24 44
Automated / Rotation only / Manual Automated / Rotation only / Manual 13 / 2 / 0 9 / 8 / 7 12 / 18 / 13
Approximate number of lines per scan Approximate number of lines per scan 200 600 600
Average time to register a pair Average time to register a pair 10-20s 20-90s 20-90s
Approximate time for entire procedure Approximate time for entire procedure a few minutes 1 hour 1.5 hour
Average distance between matching planes Before ICP Optimization 21.77mm 51.72mm 17.59mm
Average distance between matching planes After ICP Optimization 1.77mm 3.23mm 7.26mm
35
Performance Comparison

• Twice as fast as previous exhaustive searching
  approach
• Suitable for structures with many lines, new
  approach more suitable for cubic-like shapes
• not as general as previous approach
• Accuracy
• Similar to previous method before applying ICP
  algorithm
• Greatly improved after applying ICP algorithm

36
Conclusions

• Semi-automatic registration system
• Automated 3D registration routines
• Context-sensitive user interface
• Fast computation, accurate registration
• Future work
• Global optimization
• Extract higher ordered curvatures from range data
  for faster and more accurate feature-based
  registration

37
Spin Image Representation Histogram of surface
points about a rotation around surface normal at
the sample Point at varying radii from the sample
point A method of measuring shape and curvature
with local support
38
Acknowledgement

• NSF CAREER IIS-01-21239
• NSF MRI/RUI EIA-0215962
• Conference committee and all audiences
• Contact us
• http//www.cs.hunter.cuny.edu/ioannis/Vision.htm
• Ioannis Stamos, istamos_at_hunter.cuny.edu
• Cecilia Chao Chen, cchen_at_gc.cuny.edu