Dynamic%20View%20Morphing

1
Dynamic View Morphing

• performs view interpolation of dynamic scenes

2
Expanded Theory

• orthography
• methods for finding camera-to-camera
  transformation
• virtual camera not restricted to line connecting
  original cameras
• weak rectification is sufficient for physical
  realism
• appearance of straight-line motion without
  camera-to-camera transformation

3
motion from time0 to time1, as seen through A
4
For Orthographic Projection
physically correct
straight-line motion
(because motion vectors aligned)
constant-velocity motion
(because motion vectors identical)
5
For Perspective Projection

• IF first make image planes parallel to
• motion of object, and
• each other
• THEN orthographic results apply
• condition above is weak rectification

6
camera views related by fundamental matrix F
7
time 1
time 0
camera views still related by same fundamental
matrix F
8
(No Transcript)
9
each object W has its own fundamental matrix FW
10
Camera-to-camera transformation

• denoted TAB
• once known, view interpolations portray constant
  velocity motion
• potential for model building

11
Finding TAB

• can be determined from fundamental matrices for
  two distinct objects
• can be determined from four conjugate directions
• can be approximated from two conjugate directions

12
Layering Static Objects

• improves sense of object rigidity

static table, walls, and floor object gets
broken into two pieces
13
Environment Map Morphing
14
Environment Map

• environment map or panoramic mosaic or
  plenoptic function all the light that reaches
  a given point in space at an instant in time

15
Environment Map Morphing

• View morphing of entire environment maps
• uncalibrated cameras
• sparse correspondences
• widely separated views
• In particular, view morphing with
• camera moving towards scene
• objects vanishing point in view

16
Interpolating Augmented Views
A
B
17
Benefits

• placing synthetic object over real object
• segmentation
• point correspondences
• camera-to-camera transformation
• added realism moving parts, shadows,
  transparency, dont morph synthetic object
• can also use real object views instead of a
  synthetic object

18
Benefits

• automation
• by matching edges, computer can place model
  automatically
• all previous benefits become automated
• scenario visualization
• combine synthetic objects with real scenes to
  create new scenarios

19
DONE
20
Layering Static Objects

• greatly improves sense of object solidity

static table, walls, and floor object gets
broken into two pieces
21
each object W has its own fundamental matrix FW
22
Environment Map Morphing

• view morphing for environment maps

A
time0.0
???
time0.4
B
time1.0
23
Analogous to View Morphing
View Morphing
Environment Map Morphing

• rectify image planes
• interpolate conjugate points
• use interpolated points to guide morphing
  algorithm

• rectify image cylinders
• interpolate conjugate points
• use interpolated points to guide morphing
  algorithm

24
locate conjugate points
view morphing
environment map morphing
rectify image planes
rectify image cylinders
interpolate conjugate points
Morph based on interpolated points
cylinder-based morph needed for environment maps
25
z 1 image plane
y2 z2 1 image cylinder
26
Environment Map Morphing

• (STEP 1) find fundamental matrix
• (STEP 2) strongly rectify the views

then notice that, for any point in space, camera
A and camera B will give the same y and z
coordinates
27
Environment Map Morphing

• (STEP 3) project environment map onto image
  cylinder (a.k.a pipe)
• (STEP 4) interpolate conjugate points and morph

this is the cylinder y2 z2 1
28
(No Transcript)
29

TBA
x
after applying TBA
A and B
30
Outline

• layering static scenes, improvement
• orthography
• generalization of math for view morphing
• making objects appear to follow line
• Tab and how to find

31
Underlying Mathematics

• weak rectification image planes parallel
• virtual movement not restricted to line

32
Orthography

• long-distance photography
• no prewarps needed! (physical correctness)
• straight-line motion by aligning directions

33
Preconditions/Output
34
Appearance of Straight-line Motion
35
(No Transcript)
36
Orthographic Projection
physically correct
straight-line motion
constant-velocity motion
A
B
37
(No Transcript)
38

x
TBA
A
B
A
A
B
39
t 1
t 0
B took this view
A took this view
after applying TBA
A and B
40
(No Transcript)
41
A
B
physically correct
straight-line motion
constant-velocity motion