todo

1
todo

• Fix up motivation slides
• (?) Duality
• Single effective viewpoint constraint
• Demonstrate equivalence
• (?) Explain IAC
• Linearization of projection
• Include reconstruction ambiguity animations,
  house
• Dimension argument, how to make intuitive
• Essential harmonic transform

• SFM motion section
• Self-calibration
• Multiple view geometry
• Stereo rectification
• Differential case
• Estimation algorithms
• References slide

2
Short Course onOmnidirectional
VisionInternational Conference on Computer
VisionOctober 10th, 2003
Dr. Christopher GeyerUniveristy of California,
Berkeley
Prof. Tomáš PajdlaCenter for Machine
PerceptionCzech Technical University
Prof. Kostas DaniilidisGRASP LabUniversity of
Pennsylvania
3
Outline

• Intro A tour of omnidirectional systems
• Part 1 Christopher Structure-from-motion with
  parabolic mirrors
• 10 minute break
• Part 3 Kostas Images as homogeneous spaces
• Part 4 Tomáš Panoramic and other non-central
  cameras
• Conclusion

4
Introduction

• In this course we will take a detailed look at
  omni-directional sensors for computer vision.
  Omnidirectional sensors come in many varieties,
  but by definition must have a wide field-of-view.

5
Introduction

• Q Why are perspective systems insufficient and
  why is field of view important?
• A Perspective systems are one imaging modality
  of many, we are interested in sensors better
  suited to specific tasks. Sensor modality should
  enter into design of computer vision
  systems

For example, perhaps for flight wide
field-of-view sensors are appropriate, and in
general useful for mobile robots.
6
Which one?
From the Page of Omnidirectional Vision
http//www.cis.upenn.edu/kostas/omni.html
7
(Poly-)Dioptric solutions
One to two fish-eye cameras or many synchornized
cameras
Pros - High resolution per viewing angle
Cons- Bandwidth- Multiple cameras
8
(Poly-)Dioptric solutions
One to two fish-eye cameras or many synchornized
cameras
Homebrewed polydioptric cameras are cheaper, but
require calibrating and synchronizing
commercial designs tend to be expensive
9
Catadioptric solutions
Usually single camera combined with convex mirror
Cons- Blindspot- Low resolution
Pros - Single image
10
Confused?
Confused?
Confused?
Confused?
Confused?
Confused?

• Q What kind of sensor should one use?
• A Depends on your application.
• 1. If you are primarily concerned with
• resolution surveillance (coverage)
• and can afford the bandwidth expense,you
  might stick with polydioptric solutions
• 2. If you are concerned with
• bandwidth servoing, SFM
• investigate catadioptric or single wideFOV
  dioptric solutions

11
Other myths and hesitations

• Myth Catadioptric images are by necessity highly
  distorted.
• Truth Actually no parabolic mirrors induce no
  distortion (perpendicular to the viewing
  direction).
• Myth Omnidirectional cameras are more
  complicated than perspective cameras, and harder
  to do SFM with.
• Truth Actually no parabolic mirrors are easy to
  model, calibrate and do SFM with.
• Truth Omnidirectional systems have lower
  resolution
• Tradeoff Balance resolution and field of view
  for your needs

12
Goals for this part of the courseDemystifying
catadioptric cameras

• Simplify Catadioptric projections can be
  described by simple, intuitive models
• Revelations Modeling catadioptric projections
  can actually give us insight into perspective
  cameras
• SFM
• To give a framework for studying
  structure-from-motion in parabolic cameras

13
Part IModeling centralcatadioptric cameras
14
Outline of Part I

1. Properties of arbitrary camera projections,
   caustics
2. The fixed viewpoint constraint
3. The central catadioptric projections
4. Models of their projections
5. A unifying model of central catadioptric
   projection
6. Consequences of the model
7. Application

15
When is a catadioptric camera equivalent up
to distortion to a perspective one?

h
For what kinds of mirrors can the image be
warped by h into a perspective image?
Suppose we are given a catadioptric image
16
Review The projection induced by a camera
The projection inducedby a camera is the
function from space to the image plane, e.g.
f
17
Review The projection induced by a camera
The projection inducedby a camera is the
function from space to the image plane, e.g.
The least restrictive assumption that can be
made about any cameramodel is that the
inverseimage of a point is a line in space
f-1(p)
18
Review The projection induced by a camera
For many cameras, all such lines do not
necessarily intersect in a single point
19
Some optics Caustics
For many cameras, all such lines do not
necessarily intersect in a single point
Their envelope is called a (dia-)caustic and
represents a locus of viewpoints
20
Review Central projections
If all the lines intersectin a single point,
thenthe system has a singleeffective viewpoint
andit is a central projection
21
Review Central projections
If all the lines intersectin a single point,
thenthe system has a singleeffective viewpoint
andit is a central projection
If a central projectiontakes any line in
spaceto a line in the plane,then it must be a
perspective projection
22
When is a catadioptric camera equivalent up to
distortion to a perspective one?
If the projection induced by a catadioptric
camera is at most a scene independent
distortion of a perspective projection, then it
must at least be a central projection
g
23
When is a catadioptric camera equivalent up to
distortion to a perspective one?
If the projection induced by a catadioptric
camera is at most a scene independent
distortion of a perspective projection, then it
must at least be a central projection
The lines in space along which the image is
constant intersect in a single effective
viewpoint
24
When is a catadioptric camera equivalent up to
distortion to a perspective one?
Question Which combinations of mirrors and
cameras give rise to a system with a single
effectiveviewpoint?
25
Central catadioptric solutions
parabolic mirror orthographic camera
hyperbolic mirror perspective camera
elliptic mirror perspective camera

• Theorem Simon Baker Shree Nayar, CVPR 1998
• A catadioptic camera has a single effective
  viewpoint if and only if the mirrors
  cross-section is a conic section

26
The fixed viewpoint constraint Baker
y
y f (x)
Suppose that the height of the mirror at x is f
(x) And the single effectiveviewpoint lies a
distance? from the camera focus
?
x
27
The fixed viewpoint constraint
The condition that the aray emanating from
thefocus is reflected in adirection incident
withthe mirror focus can bedescribed by an ODE
28
The fixed viewpoint constraint
The solutions to this ODEcan be shown to be
restrictedto conic sections, e.g.,
29
Modeling a parabolic projection
space point
image point
30
Modeling a hyperbolic projection
image point
space point
31
Modeling an elliptic projection
image point
space point
32
Questions about catadioptric projections

• Q What are the properties of the projections
  induced by these types of sensors?
• Q How can we extend a theory of
  structure-from-motion and self-calibration for
  uncalibrated catadioptric cameras?
• Note there is no difference for calibrated
  catadioptric cameras, since they can be warped to
  calibrated perspective images
• Q Are there simplified models for all
  catadioptric projections?

33
Abstracting catadioptric projections

• In each case the projection to one surface (the
  mirror) followed by a projection to another
  surface (the image plane).

34
Abstracting catadioptric projections

• In other words they can be written as the
  composition of two functions
• f? is a non-linear function (projection to a
  quadric) and g is a linear function (projection
  to a plane)

35
Abstracting catadioptric projections
(projective linear)
(non-linear)
36
Switcharoo...

• Can we commute the decomposition such that the
  non-linear projection becomes independent of the
  eccentricity?

(linear)
(linear)
(non-linear parameterdependent)
(non-linear butparameterindependent)
37
Decomposing catadioptric projections

• The general catadioptric projection can be
  written
• The result can be written in homogeneous
  coordinates

(non-linear butparameterindependent)
(homogeneous coordinates)
(linear projectivetransformation)
38
Alternative decomposition
f
f centrally projects to the sphere it yields a
homogeneous point whose fourth coordinateis the
distance of the space point
39
Alternative decomposition
g
?
g centrally projects to the image plane from a
point on the axis of the sphere, the height of
thepoint is determined by the eccentricity
?
40
Consequences of this model (1 of 5)
1. Central catadioptric projections and
perspective projections are represented in one
framework
41
Consequences of this model (2 of 5)
2. (a) The projection of a line in space to the
sphere is a great circle (b) The central
projection of a great circle to the image plane
is a conic section
(c) Since stereographic projection sends great
circles to circles in the image, the parabolic
projection of a line is a circle
42
Consequences of this model (3 of 5)
3. (a) The Jacobian fromthe viewing sphere to
theimage plane is easilycalculated
(b) The Jacobian for the para-bolic/stereographic
projection is proportional to a
rotationparabolic projection is locally
distortionless, i.e. conformal
43
Consequences of this model (4 of 5)
4. (a) Height function
is not one-to-one (b) Satisfies
reciprocal eccentricities map to
the same height (c) Elliptical and hyperbolic
mirrors are indistinguishablefrom the
projections they induce
44
Consequences of this model (5 of 5)

• Recall properties of
• perspective projection
• Domain (projective space)
• Range (the projective plane)
• a. Antipodal points have same image
• b. Equator projects to line at infinity

45
Consequences of this model (5 of 5)

• Recall properties of
• perspective projection
• Domain (projective space)
• Range (the projective plane)
• a. Antipodal points have same image
• b. Equator projects to line at infinity

5. Parabolic projection Domain
(exclude plane at ?) Range
(extd real plane) a. Antipodal points have
inverted imgs b. Equator projects to circle
propor-tional to focal length
46

• Modeling central catadiopric cameras
• Unifying model of central catadioptric cameras
• Line images are conics
• Conformality of stereographic projection
• Indistinguishability of elliptic and hyperbolic
  projections
• Inadequacy of projective plane for catadioptric
  systems

47
End of Part IQuestions?
48
Part IIFocus on Parabolic Mirrors
49
Outline of Part II

1. Point, circle and line image representation
2. The image of the absolute conic
3. Lorentz transformations and their conformality
4. Infinitessimal generators of Lorentz
   transformations
5. Comparison of Lorentz transformations and
   rotations
6. Complex representation estimation
7. Linearization of the parabolic projection

50
Projective geometryA framework for perspective
imaging
Linear projection formula because of the use of
homogeneous coordinates If
and then where and K
is the calibration matrix
X
p
z
y
(R,t)
x
51
Projective geometryA framework for perspective
imaging
Linear projection formula because of the use of
homogeneous coordinates If
and then where and K
is the calibration matrix
q
p
Where, for example, the line between two points
can be represented by the cross product of two
points Lines lie in the dual space
z
y
x
52
Projective geometryA framework for perspective
imaging

• Some things maybe we take for granted
• Representation of lines points
• Condition that a line coincide with a point
• Construction of the point coinciding with two
  lines
• Dually, construction of the line between two
  points
• Conditions for the coincidence of three lines
• Dually, conditions for the collinearity of three
  points
• Homographies and the invariance of the
  cross-ratio
• Absolute conic and all that jazz

53
Projective geometryA framework for perspective
imaging

• Over 2 decades this framework has been used to
  derive
• Multiview geometry multilinear constraints in
  multiple views
• E.g., fundamental matrices in two views
• Theories of self-calibration
• Simplifications for special motions
  homographies, etc.
• Q How can we extend these results to
  catadioptric cameras?
• A Like the perspective theory, start with a
  framework for the representation of features

54
But what about non-linearity?

• It seems an obstacle to this vision is the
  non-linearity of the projection equation
• Recall though that the perspective projection is
  also non-linear

(projection mapping induced by a parabolic
catadioptric camera)
55
Representation of circles
Start with a circle in the image planethis
sphere is not necessarily calibrated
56
Representation of circles
The inverse stereographic projection of a circle
is a circle
57
Representation of circles
Through this circle there passes a unique plane
all such planes are in 1-to-1 correspondence with
circles in the image plane
58
Representation of circles
This plane is in 1-to-1 correspondence with its
polethe vertex of the cone tangent to the
sphere at the circle
59
Representation of circles
This plane is in 1-to-1 correspondence with its
polethe vertex of the cone tangent to the
sphere at the circle
60
Representation of circles
The circles center is collinear with the
representation and the north pole, the radius
varies with position along the line
61
Representation of circles
Easy Solve for u, v and r
Given the position in space, determine the circle
center and radius
62
Representation of circles
imaginary locus, r imaginary
zero radius, r 0
Three cases a. inside sphereb. on spherec.
outside sphere
real locus, r gt 0
63
Partition of feature space
Point features and circles have point
representations in the same space recall in
projective plane, dual space reqd
64
Representation of image points
So now we have a repre-sentation of image
points For if then
65
Representation of image points
Recall that the sphere is the locus of points
which satisfy the equation
In projective space this is the set of points
lying on the quadratic surfacegiven by a
quadratic form
66
Partition of feature space
67
Coincidence condition
ax by c 0
or
p (x,y)
r
What is the condition thatp lies on a circle of
radius r and center (x,y)?
(u,v)
p (x,y)
68
Coincidence condition
We want with some algebra one finds
69
Angle of intersection
Hence, circles orthogonal iff
70
Meets and joins
Projective plane
Parabolic plane
Circle through two points not unique In space
there is only one line through two pointsWhy
isnt this true of their projection?
Contradiction?
71
Images of lines in space
All lines intersect the fronto-parallel
horizon(projection of the equator) antipodally
72
(No Transcript)
73
image center
twice focal length
focallength
74
(No Transcript)
75
Line image constraint
When taking into account image center and circle
center, the constraint becomes
76
Line image constraint
When taking into account image center and circle
center, the constraint becomes
(u,v)
(x,y)
and can be written as
where
? represents an imaginary circle
77
Line image constraint
78
Line image constraint
Implies calibration by fitting plane to circle
representations
79
Interpretation Absolute and calibrating conics
absolute conic
calibrating conic
80
Summary up to now

• We have a system of representation for
• Image features
• Real radii circles
• Imaginary radii circles
• Conditions for coincidence
• Formula for angle of intersection
• Condition that a circle be a line image,absolute
  conic

81
Summary up to now

• We have a system of representation for
• Image features
• Real radii circles
• Imaginary radii circles
• Conditions for coincidence
• Formula for angle of intersection
• Condition that a circle be a line image,absolute
  conic

Questions?
82
Uncalibrated cameras
83
Why should it be linear?
Choose an arbitrary circle and find its inverse
stereographic image
84
Why should it be linear?
Translate the circle find the inversestereograph
ic image of the translated circle
85
Why should it be linear?
The translation in the plane induces a
trans-formation of the sphere which preserves
planes
86
Uncalibrated cameras
This argument applies to scaling, rotation and
translation Thus a similarity transformation in
the plane induces some projective linear
transformation A of circle space
It also sends any point satisfying to some
point satisfying
87
Sphere preserving transformations
where
The set of all such matrices is closed under
matrix multi-plication, inversion and contains
the identity it is a group
88
Lorentz and orthogonal groups
where
The set of all such matrices is closed under
matrix multi-plication, inversion and contains
the identity it is a group
Lorentz group
Orthogonal group (in 4-dimensions)
89
The Lorentz group
90
The Lorentz group
91
The Lorentz Lie group
Suppose we have a curve satisfying
A(0) I
92
The Lorentz Lie group
Differentiate both sides of
to obtain
Implying
93
The Lorentz Lie group
94
The Lorentz Lie group
For any matrix Lie group, a local one-to-one map
from its Lie algebra back to the Lie group is
given by the exponential map.
exp
95
Infinitessimal generators of the Lorentz group
Rotations aboutthe x-axis
y-axis
z-axis
Generated by skew-symmetric matrices
96
Infinitessimal generators of the Lorentz group
Translations alongthe x-axis
Scaling aboutthe origin
y-axis
Generated by
97
Lorentz group consistently transforms circle
space
One last property of Lorentz transformationsis
that they transform representations of
circlesconsistent with the transformations of
image points
98
Lorentz group consistently transforms circle
space
Questions?
One last property of Lorentz transformationsis
that they transform representations of
circlesconsistent with the transformations of
image points
99
Inverting the projection
With insight into properties of parabolic
projections, lets reconsider the problem of
inverting an uncalibrated projection
Recall that we can decompose the parabolic
projection as
n
s is stereographic projection n is projection
to the sphere
s
100
Inverting the projection
With insight into properties of parabolic
projections, lets reconsider the problem of
inverting an uncalibrated projection
Recall that we can decompose the parabolic
projection as
n
s
k is a calibration transformation
k
101
Inverting the projection
However we now know that there exists some
projective linear k such that s ? k k ? s
102
Inverting the projection
s-1(x)
s-1(k(x))
x
k(x)
We have the points in the plane and their
inverse stereographic images
103
Inverting the projection
Problem obtain s-1(x) as a linear
transformation of s-1(k(x)).
s-1(x)
s-1(k(x))
x
k(x)
We have the points in the plane and their
inverse stereographic images
104
Inverting the projection
Problem obtain s-1(x) as a linear
transformation of s-1(k(x)).
Knowns k(x) Unknowns x, k Non-linear in
unknown s-1(x) s-1(k-1(k(x)))
s-1(x)
s-1(k(x))
x
k(x)
We have the points in the plane and their
inverse stereographic images
105
Inverting the projection
s-1(x)
Ks-1(x) s-1(k(x))
x
k(x)
k and K commute about s-1
106
Inverting the projection
s-1(x) K-1 s-1(k(x))
Ks-1(x) s-1(k(x))
x
k(x)
Therefore the calibrated point is a linear
transformation of the lifting of the uncalibrated
point
107
Inverting the projection
s-1(x) K-1 s-1(k(x))
Ks-1(x) s-1(k(x))
Linear in unknown s-1(x) K-1 s-1(k(x))
x
k(x)
Therefore the calibrated point is a linear
transformation of the lifting of the uncalibrated
point
108
Linearization of the inverse projection
PK-1 s-1(k(x))
x
k(x)
Then the ray (in P2), as a function of the
uncalibrated image point is, is a linear
transformation of the lifting
109
Linearization of the inverse projection
PK-1 s-1(k(x)) is equivalent to the perspective
projection of the space point
PK-1 s-1(k(x))
x
k(x)
Then the ray (in P2), as a function of the
uncalibrated image point is, is a linear
transformation of the lifting
110
Linearization of the inverse projection
K-1 is and unknown but linear transformation(and
can be absorbed into linear constraints) s-1(x)
is a non-linear but known transformation
PK-1 s-1(k(x))
x
k(x)
Then the ray (in P2), as a function of the
uncalibrated image point is, is a linear
transformation of the lifting
111
uncalibrated rays
We do not claim that there is a linear
trans-formation fromuncalibrated RAYS (i.e.
elements of P2)to calibrated RAYS (elements of
P2)
calibrated rays
112
uncalibrated points
Instead, we claim that there is a linear
trans-formation fromuncalibrated liftedimage
points (i.e. elements of P3)to calibrated
RAYS (elements of P2)
calibrated rays
113
Calibration transformation
114
Calibration transformation on the absolute conic
115
Calibration transformation on the absolute conic
The point ? is sent to the origin (0,0,0,1) in
P3 in The origin is in the null-space of the
projection P Hence PK-1 ? 0
116
(No Transcript)
117
End of Part IIQuestions?
118
Outline of Part III

1. The parabolic catadioptric fundamental matrix
2. Self-calibration
3. Kruppa equations trivially satisfied
4. Planar homographies self-calibration
5. Multiple view geometry
6. Infinitessimal motions
7. Conformal rectification

119
Deriving the parabolic epipolar constraint
Suppose two views are separated by a rotation R
and translation t. Given a point X in space,
what constraint must the image points p1 and p2
satisfy?
120
Deriving the parabolic epipolar constraint
If we know the calibrated rays, then they are
known to satisfy the epipolar constraint for
perspective cameras (C. Longuet-Higgins)
121
Deriving the parabolic epipolar constraint
If the image points are uncalibrated, then we
know that the calibrated rays are
linearlyrelated to the uncalibrated liftings
122
Deriving the parabolic epipolar constraint
F
(4?4 parabolic fundamental matrix)
123
Deriving the parabolic epipolar constraint
Consequently lifted image points satisfy a
bilinear epipolar constraint
124
Self-calibration
To each view there is associated an IAC which are
represented by ?1 and ?2
They are in the nullspaces of PKi-1 and so in
the nullspacesof F and FT
125
Self-calibration
If ? ?1 ?2 i.e., the intrinsic parameters
are the same, then ? can be uniquely recovered
from the intersection of the nullspaces
Unless in whichcase
126
A characterization of parabolicfundamental
matrices

• Recall that a 3?3 matrix E is an essential
  matrix if and only if
• for some U, V in SO(3)
• Claim A 4?4 matrix F is a parabolic fundamental
  matrix if and only if
• for some U, V in SO(3,1)

127
Simple proof

• (?)

128
Simple proof

• (?)
• (?)

129
Estimation

• Because we have a bilinear constraint (and in
  general multilinear constraints) many methods
  that apply to the estimation of structure and
  motion from multiple perspective images apply,
  with some exceptions, to parabolic cameras.
• Normalized epipolar constraint can be minimized
• Unfortunately no equivalent to the 8/7-point
  algorithm(averaging Lorentzian singular values
  does not minimize Frobenius norm)
• RANSAC and other robust methods apply
• Structure estimation identical to perspective
  case once calibrated
• Robust to modest deviations from ideal
  assumptions (e.g., non-aligned mirror,
  non-parabolic mirrors, etc.)

130
Two view example

• Given these two views with corresponding points
  estimate the parabolic fundamental matrix

131
(No Transcript)
132
(No Transcript)
133
epipolar circle
two epipoles
134
??1
??2
?(?1 , ?2)
135
(No Transcript)
136
(No Transcript)
137
??1
??2

• A consequence of this is that the epipolar
  geometry is completely determined by the two
  epipoles in each imageand the angle ?
• Therefore the epipolar geometry has 9 parameters
  whereas the motion (5) and intrinsics for each
  view (6) total 11. 2-parameter ambiguity.

138
(No Transcript)
139
(No Transcript)
140
(No Transcript)
141
(No Transcript)
142
(No Transcript)
143
What is the ambiguity?

• We showed that the the epipolar geometry is
  determined by nine parameters, and the motion and
  camera parameters by eleven, demonstrating that
  there is a two-parameter ambiguity. Meaning for
  any two images there is a two-parameter family of
  possible reconstructions giving rise to the
  images. What is this family? Is it closed under
  some subset of projective transformations?

144
This is your house
145
This is your house on a parabolic mirror
146
This is your house on drugs
i.e. this is the ambiguity in the reconstruction
of a house the ambiguity is not projective
147
End of Part IIIQuestions?
148
Outline of Part IV

1. Group-theoretic analysis of the parabolic
   fundamental matrix
2. Quotient spaces of bilinear forms (parabolic
   fundamental matrices and essential matrices)
3. Essential harmonic transform

149
The space of parabolicfundamental matrices
What is its structure? Is it a manifold? How many
degrees-of-freedom does it have? What
ambiguities are there in motion estimation?
parabolic fundamental matrices
150
Group theoretic analysis of bilinear constraints

• Lets examine the LSVD characterization of
  parabolic fundamental matrices
• implies fundamental matrices are closed under
  left or right multiplication by Lorentz
  transformations, i.e.
• is also a parabolic fundamental matrix.Note
  the same reasoning applies to essential matrices.

151

• Thus SO(3,1) ? SO(3,1) acts upon the set of
  fundamental matrices

parabolic fundamental matrices
SO(3,1) ? SO(3,1)
F
152
The identity of the group induces the identity map
F
e (I, I)
153

• The action is associative

F
g (U1,V1)
h (U2,V2)
g h (U1U2,V1V2)
154

• The action is (left) associative

F
g (U1,V1)
h (U2,V2)
g h (U1U2,V1V2)
155
The action is transitive for every F1 F2
there exists some g taking F1 to F2
F1
F2
g
156
With the action ? , SO(3,1) ? SO(3,1)parameteriz
es?????
F
157
Because of transitivity,the parameterization
issurjective (onto) there is a g mapping F to
F
F
F
158
Since SO(3) is itselfparameterized by
???????????? is parameterized by
exp
159
In fact since ???????????????is surjective in
SO(3,1), so then is the parameterization of ????
160
Parameterization not one-to-one
The paramaterizationmay be redundante.g., more
than one groupelement may map F to F
F
161
The set HF
So what elements leave F invariant? Call it HF
F
g
162
The set HF
At the very least it contains the identity
F
163
The set HF
Also HF is closed under (i) composition
F
164
The set HF
Also HF is closed underand (ii) inversion
F
165
The isotropy subgroup
Hence HF is a subgroupIt is called the isotropy
subgroup
F
166
Cosets of the isotropy subgroup
Multiply every elementof HF by an element g
F
167
Cosets of the isotropy subgroup
What we obtain is atranslation of HF by ga
coset of HF
F
168
Cosets of the isotropy subgroup
Claim any two elementsof the coset g ? HF map
Fto the same fundamentalmatrix
F1
F
F2
Claim F1 F2
169
Cosets of the isotropy subgroup
Since h1 is in HF and bythe associativity of
theaction, g and g h1 bothsend F to the same
point
F1
F
170
Cosets of the isotropy subgroup
The same reasoning applies to h2 and so F1 F2
F1
F
F2
171
Cosets of the isotropy subgroup
The same reasoning applies to h2 and so F1 F2
F1
F
F2
172
Cosets of the isotropy subgroup
Consequently every coset is in one-to-onecorrespo
ndence witha fundamental matrix
F
gF
hF
173
Cosets of HF partition SO(3,1) ? SO(3,1)
The cosets are pairwise disjoint and their union
is all of SO(3,1) ? SO(3,1) they form a
partition
F
174
Quotient spaces
The partition of a group into its cosets is
calledthe quotient space
F
175
The set of fundamental matrices form a quotient
space
Because of its one-to-one correspondence, the set
of fundamental matrices inherits the structure of
a quotient space
176
Quotient of Lie algebras are automatically
manifolds
The dimension of the quotient space is the
difference in the dimensions of the Lie groups
177
Quotient of Lie algebras are automatically
manifolds
The dimension of the quotient space is the
difference in the dimensions of the Lie groups
178
Quotient of Lie algebras are automatically
manifolds
The dimension of the quotient space is the
difference in the dimensions of the Lie groups


9
12
3
179
All of these results also apply to essential
matrices
Instead, SO(3) ? SO(3) acts on the set of
essentialmatrices
SO(3) ? SO(3)
HE
180
Harmonic analysis of bilinear forms

• Is it just a novelty that essential matrices and
  parabolic fundamental matrices are quotient
  spaces?
• In other words, who cares?
• We believe the description as a quotient space
  is important for the following reasons
• Simple unifying geometric description of bilinear
  
• Global (surjective) nowhere-singular
  parameterization
• These spaces are now endowed with Fourier
  transforms

181
The rotational harmonic transform

• Recall that the Fourier transform is a
  projection of functions on L2(0,?)
• Similarly the rotational harmonic transform
  (RHT) is a projection of square integrable
  functions on SO(3) denoted L2(SO(3)) onto an
  orthonormal basis

182
The rotational harmonic transform

• Recall that the Fourier transform is a
  projection of functions on L2(0,?)
• Similarly the rotational harmonic transform
  (RHT) is a projection of square integrable
  functions on SO(3) denoted L2(SO(3)) onto an
  orthonormal basis

Wigner d-coefficients
Rotation invariantmeasure on SO(3)
183
The rotational harmonic transform

• The rotational harmonic transform obeys a number
  of properties some of which are
• Limit of partial sums converge to function
• Parseval equality
• Shift theorem
• Convolution theorem

184
Functions on the quotient space

• To define a function on the space of essential
  matrices we take some function on SO(3)?? SO(3)
  and require that it be constant on cosets of HE.
  Alternatively f equals its average over all
  cosets.
• Recall that the subgroup
• and the cosets are

g HE
185
The essential harmonic transform

• The essential harmonic transform is a projection
  of such a function onto the bi-rotational
  harmonics of SO(3)?? SO(3)
• and because it is constant over the cosets it
  satisfies

186
Applications

• Q What can we do with an essential harmonic
  transform?
• A Fast convolutions.
• Might it be possible to estimate an essential
  matrix via a convolution of two signals to obtain
  a kind of correlation value for all possible
  essential matrices?
• Is it possible to unite signal processing and
  geometry?
• To be continued

187
Given two parabolic catad-ioptric cameras,
rectify the stereo pair, i.e., transform both
images so that corresp-onding points lie on the
same scanline.
188
(No Transcript)
189
(No Transcript)
190
2d
191
2d
192

• Both Möbius transformations and the equivalent to
  homographies must preserve line images (circles)
  and are therefore insufficient
• What transformations can perform the
  rectification?

193
Bipolar coordinate system
194
?
195
r1 / r2 constant
r1
r2
196
-1
1
?
This is analytic (i.e., differentiable) and
therefore conformal
197
-1
1
-1
1
198
(No Transcript)
199
(No Transcript)
200
End of Part IVQuestions?
201

• Two-view geometry of catadioptric cameras
• Geyer Daniilidis, Mirrors in Motion ICCV 2003
  
• Single-view geometry of catadioptric cameras
• Geyer Daniilidis, Catadioptric Projective
  Geometry IJCV Dec. 2001
• Epipolar geometry of central catadioptric cameras
• Pajdla Svoboda, IJCV 2002
• Theory of Catadioptric Image Formation
• Baker Nayar, IJCV
• Complex analysis inversive geometry
• Geometry of Complex Numbers by Hans
  Schwerdtfeger, Dover
• Visual Complex Analysis by Tristan
  Needham, Oxford University Press
• Inversion Theory and Conformal Mapping
  by David Blair, AMS

202
(No Transcript)
203
(No Transcript)
204
Deriving the fixed viewpoint constraint
( x, f (x) )
205
Deriving the fixed viewpoint constraint
( x, f (x) )
206
Deriving the fixed viewpoint constraint
207
Deriving the fixed viewpoint constraint
208
Deriving the fixed viewpoint constraint
209
(No Transcript)
210
(No Transcript)
211
(No Transcript)
212
(No Transcript)
213
(No Transcript)
214
Any questions?
215

• Two-view geometry of catadioptric cameras
• Geyer Daniilidis, Mirrors in Motion ICCV 2003
  
• Single-view geometry of catadioptric cameras
• Geyer Daniilidis, Catadioptric Projective
  Geometry IJCV Dec. 2001
• Epipolar geometry of central catadioptric cameras
• Pajdla Svoboda, IJCV 2002
• Theory of Catadioptric Image Formation
• Baker Nayar, IJCV
• Omnidirectional vision in general
• Baker Nayar, Panoramic Vision
• Relating to complex geometry
• Hans Schwerdtfeger Geometry of Complex Numbers,
  Dover
• Tristan Needham, Visual Complex Analysis, Oxford
  University Press
• David Blair, Inversion Theory and Conformal
  Mapping, AMS

216
5 minute break