Cata-Fisheye Camera for Panoramic Imaging

1
Cata-Fisheye Camera for Panoramic Imaging
Guru Krishnan Shree K. Nayar
Columbia University
IEEE Workshop on Applications of Computer Vision
January 2008, Copper Mountain, USA
Support ONR
2
Applications of Panoramic Cameras
Surveillance
flickr.com
Video Conferencing
sun.com
Navigation
mountpleasant.ie
3
Desired Field of View
550
3600
600
pointgrey.com
4
Previous Work in Panoramic Cameras
No Parallax
Large Parallax
Synchronization in time ? Photometric and
Geometric Calibration
Hemispherical FOV
Large Parallax
Aberrations
5
Our Approach
- Simple, Compact and Inexpensive
Design - High Quality Panoramic Images -
Parallax Only for Short Distances from the Camera
6
Fisheye Lens Camera
Cata-
Camera
Captured Image
7
Fisheye Lens Camera
Cata-
Camera
Captured Image
8
Concept
Mirror
Fisheye Lens
Detector
9
Concept
Mirror
Fisheye Lens
Detector
10
Concept
Mirror
Fisheye Lens
Detector
11
Concept
Fisheye Panorama
Mirror Panorama
12
Field Of View
13
Design
Upper Limit
Spherical Mirror
Dqo
Overlap
Lower Limit
Mirror Parameters
Fisheye Parameters
Radius ( rm )
Field of View ( qfl )
Width ( dm )
Position ( hm )
14
Design
qfu
Upper Limit
Mirror Upper Limit
P
Spherical Mirror
Overlap
hl
Lower Limit
qml
Mirror Parameters
Fisheye Parameters
Radius ( rm )
Field of View ( qfl )
Width ( dm )
Position ( hm )
15
Design
Position, hm Smallest Value qml Lower Limit
where, qn ½(p qfl qfu - Dqo)
16
Calibration

• Fisheye Projection Function r f(q)

( qf , ff ) ( qm , fm )
17
Calibration

• Initialize Mirror Position and Orientation

Fisheye Panorama (Overlap Region)

• Generate Fisheye and Mirror-Panoramas

• Find Corresponding Features using SIFT

Mirror Panorama (Overlap Region)
Ff(i) (qf(i) , ff(i))
18
Calibration
Panorama
One-Time Calibration
19
Image Resolution
Panorama Resolution Map
Non-Uniform Resolution
20
Image Resolution
Panorama Resolution Map
Non-Uniform Resolution
21
Parallax
Fisheye Viewpoint
Fisheye Lens
Small Parallax
22
Prototype

• Lumenera Lw570 Camera
• 5MP sensor
• Fujinon Fisheye Lens
• Vertical FOV 250 to -300
• Custom Spherical Mirror
• Radius 70 mm Width 50 mm

Panorama 3600 x 550 pixels _at_ 7fps No
Visible Parallax after 3 ft
23
Street Scene
24
Street Scene
1800
00
Left Half (1800 pixels)
250
550 pixels
-300
250
550 pixels
-300
Right Half (1800 pixels)
3600
1800
25
Street Scene
550 pixels
800 pixels
26
Video Conferencing
1800
00
Left Half (1800 pixels)
250
550 pixels
-300
250
550 pixels
-300
Right Half (1800 pixels)
3600
1800
27
Video Conferencing
550 pixels
800 pixels
28
Summary
29
Thank You Questions?
1800
00
Left Half (1800 pixels)
250
550 pixels
-300
250
550 pixels
-300
Right Half (1800 pixels)
3600
1800
30
Computing the Fisheye Panorama
f 0
qau
(qp , fp)
qal
Panorama
f(qp)cos(fp), f(qp)sin(fp)
(qp , fp)
31
Computing the Mirror Panorama
f 0
qau
qal
Panorama
(qc , fc)
f(qc)cos(fc), f(qc)sin(fc)
(qp , fp)
32
Computing the Mirror Panorama
(qc , fc)
(qp , fp)
f(qc)cos(fc), f(qc)sin(fc)
33
Prototype Performance Analysis
Optical Performance
Panorama Resolution