Multi-perspective Panoramas

1
Multi-perspective Panoramas

• Slides from a talk by Lihi Zelnik-Manor at
  ICCV07 3DRR workshop

2
Pictures capture memories
3
Panoramas
Registration Brown Lowe, ICCV05 Blending
Burt Adelson, Trans. Graphics,1983 Visuali
zation Kopf et al., SIGGRAPH, 2007
4
Bad panorama?
Output of Brown Lowe software
5
No geometrically consistent solution
6
Scientists solution to panoramasSingle center
of projection
No 3D!!!
Registration Brown Lowe, ICCV05 Blending
Burt Adelson, Trans. Graphics,1983 Visuali
zation Kopf et al., SIGGRAPH, 2007
7
From sphere to plane
Distortions are unavoidable
8
Distorted panoramas
Actual appearance
Output of Brown Lowe software
9
Objectives

• Better looking panoramas
• Let the camera move
• Any view
• Natural photographing

10
Stand on the shoulders of giants
Cartographers
Artists
11
Cartographic projections
12
Common panorama projections
Perspective
Stereographic
Cylindircal
13
Global Projections
Perspective
Stereographic
Cylindircal
14
Learn from the artists
Multiple view points
De Chirico Mystery and Melancholy of a Street,
1914
15
Renaissance painters solution
School of Athens, Raffaello Sanzio 1510
Give a separate treatment to different parts of
the scene!!
16
Personalized projections
School of Athens, Raffaello Sanzio 1510
Give a separate treatment to different parts of
the scene!!
17
Multiple planes of projection
Sharp discontinuities can often be well hidden
18
Single view
Our multi-view result
19
Single view
Our multi-view result
20
Single view
Our multi-view result
21
Applying personalized projections
Input images
Foreground
Background panorama
22
Single view
Our multi-view result
23
Single view
Our multi-view result
24
Objectives - revisited

• Better looking panoramas
• Let the camera move
• Any view
• Natural photographing

Multiple views can live together
25
Multi-view compositions
3D!!
David Hockney, Place Furstenberg, (1985)
26
Why multi-view?
Multiple viewpoints
Single viewpoint
Melissa Slemin, Place Furstenberg, 2003
David Hockney, Place Furstenberg, 1985
27
Multi-view panoramas
Single view
Multiview
Zomet et al. (PAMI03)
Requires video input
28
Long Imaging
Agarwala et al. (SIGGRAPH 2006)
29
Smooth Multi-View
Google maps
30
Whats wrong in the picture?
Google maps
31
Non-smooth
Google maps
32
The Chair
David Hockney (1985)
33
Joiners are popular
Flickr statistics (Aug07)
4,985 photos matching joiners.
4,007 photos matching Hockney.
41 groups about Hockney
Thousands of members
34
Main goals Automate joiners Generalize
panoramas to general image collections
35
Objectives

• For ArtistsReduce manual labor

Fully automatic
Manual 40min.
36
Objectives

• For ArtistsReduce manual labor
• For non-artistsGenerate pleasing-to-the-eye
  joiners

37
Objectives

• For ArtistsReduce manual labor
• For non-artistsGenerate pleasing-to-the-eye
  joiners
• For data explorationOrganize images spatially

38
Whats going on here?
39
A cacti garden
40
Principles
41
Principles

• Convey topology

Correct
Incorrect
42
Principles

• Convey topology
• A 2D layering of images

Blending blurry
Graph-cut cuts hood
Desired joiner
43
Principles

• Convey topology
• A 2D layering of images
• Dont distort images

rotate
scale
translate
44
Principles

• Convey topology
• A 2D layering of images
• Dont distort images
• Minimize inconsistencies

Bad
Good
45
Algorithm
46
Step 1 Feature matching
Brown Lowe, ICCV03
47
Step 2 Align
Large inconsistencies
Brown Lowe, ICCV03
48
Step 3 Order
Reduced inconsistencies
49
Ordering images

• Try all orders only for small datasets

50
Ordering images

• Try all orders only for small datasets
• complexity (mn)?m imagesn overlaps?
  acyclic orders

51
Ordering images

• Observations
• Typically each image overlaps with only a few
  others
• Many decisions can be taken locally

52
Ordering images

• Approximate solution
• Solve for each image independently
• Iterate over all images

53
Can we do better?
54
Step 4 Improve alignment
55
Iterate Align-Order-Importance
56
Iterative refinement
Initial
Final
57
Iterative refinement
Initial
Final
58
Iterative refinement
Initial
Final
59
What is this?
60
Thats me reading
61
Anza-Borrego
62
Tractor
63
Art reproduction
Paolo Uccello, 1436
64
Art reproduction
Paolo Uccello, 1436
Zelnik Perona, 2006
65
Art reproduction
Single view-point
Zelnik Perona, 2006
66
Manual by Photographer
67
Our automatic result
68
Failure?
69
GUI
70
The Impossible Bridge
71
Homage to David Hockney
72
Take home

• Incorrect geometries are possible and fun!
• Geometry is not enough, we need scene analysis
• A highly related work"Scene Collages and
  Flexible Camera Arrays,Y. Nomura, L. Zhang and
  S.K. Nayar,Eurographics Symposium on Rendering,
  Jun, 2007.

73
Thank You