Screen-Camera Calibration Using Gray Codes

1
Screen-Camera Calibration Using Gray Codes
Yannick Francken Chris Hermans Philippe Bekaert

• Hasselt University tUL IBBT
• Expertise Centre for Digital Media, Belgium
• firstname.lastname_at_uhasselt.be

2
Goal
Geometric calibration of a camera w.r.t. a screen
3
Motivation

• Vision based HCI
• 3D reconstruction

Chen et al., SPIE 2002
Gorodnichy et al., SPIE 2002
Francken et al., CVPR 2008
Nehab et al., CVPR 2008
4
Related Work

• Planar mirror

Funk and Yang, CRV 2007
Bonfort et al., ACCV 2006
5
Related Work

• Planar mirror
• Spherical mirror
• Corner reflections

?
?
?
?
Tarini et al., Graphical Models 2005
6
Related Work

• Planar mirror
• Spherical mirror
• Corner reflections
• Edge reflections

Francken et al., CRV 2007
7
Our Approach

• Planar mirror
• Spherical mirror
• Corner reflections
• Edge reflections
• Surface reflection
• Increased accuracy
• Less manual interventions
• Robust screen reflectiondetection

8
Concept

1. Mirror detection
2. Screen pixel labeling
3. 3D reconstruction

9
Mirror detection

1. Internal camera parameters K
2. Background subtraction
3. Edge extraction
4. Ellipse fitting
5. 2D ellipse to 3D sphere

10
Screen pixel labeling
Find screen-reflection correspondences
?
11
Screen pixel labeling
12
Screen pixel labeling
13
Screen pixel labeling
14
Screen pixel labeling
15
Screen pixel labeling
16
Screen pixel labeling
17
Screen pixel labeling

• Screen position as
• Intensities
• 2 patterns
• 4 patterns
• 6 patterns
• 8 patterns

18
Reflection mask
Find reflection mask
19
Reflection mask
Displayed patterns
Recorded images
Intensity differences
20
Reflection mask
Displayed patterns
Recorded images
Intensity differences
21
Reflection mask
Displayed patterns
Recorded images
Intensity differences
22
Reflection mask
Displayed patterns
Recorded images
Intensity differences
23
Reflection mask
Displayed patterns
Recorded images
Intensity differences
24
Reflection mask
Displayed patterns
Recorded images
Intensity differences
25
Reflection mask
Displayed patterns
Threshold
Inter- reflection
Direct reflection
Recorded images
Intensity differences
26
3D reconstruction

• Reflected rayintersections
• Plane estimation
• Grid estimation

Known parameters
27
3D reconstruction

• Reflected rayintersections
• Plane estimation
• Grid estimation

Solution Find 2D 2D similarity transform
Result 2D pixel u ? 3D location x
x M . u
28
Overview
3D sphere locations
Reflection mask
Labels




x M . u
29
Results

• Error as function of pattern refinement

• Accuracy
• Ground truth
• Francken et al., CRV 2007
• Our approach

Error 1.1-2.4 cm
Error 0.3-0.4 cm
30
Results

• Error as function of sphere combinations

31
Results

• Error as function of sphere combinations

32
Results

• Error as function of sphere combinations

33
Results

• Error as function of sphere combinations

34
Conclusion

• Screen-camera calibration using Gray codes
• Increased accuracy
• Less manual interventions
• Robust screen reflectiondetection

35
Future Work

• Gradient patterns
• Speed!
• Quality?
• Camera defocus
• Which patternsare robust?

Blurry reflections
Sharp sphere
36
Questions?
3D sphere locations
Reflection mask
Labels




x M . u
yannick.francken_at_uhasselt.be
http//research.edm.uhasselt.be/yfrancken
37
TODO bronnen, accuracy ERROR ipv scherm afstanden