Shape Analysis and Retrieval

1
Shape Analysis and Retrieval

• Statistical Shape Descriptors

Notes courtesy of Funk et al., SIGGRAPH 2004
2

• Outline
• Shape Descriptors
• Statistical Shape Descriptors
• Singular Value Decomposition (SVD)

3
Shape Matching

• General approachDefine a function that takes in
  two models and returns a measure of their
  proximity.

D
,
D
,
M1
M1
M3
M2
M1 is closer to M2 than it is to M3
4
Shape Descriptors

• Shape DescriptorA structured abstraction of a
  3D model that is well suited to the challenges of
  shape matching

3D Models
Descriptors
5
Matching with Descriptors

• Preprocessing
• Compute database descriptors
• Run-Time

3D Query
ShapeDescriptor
BestMatches
3D Database
6
Matching with Descriptors

• Preprocessing
• Compute database descriptors
• Run-Time
• Compute query descriptor

3D Query
ShapeDescriptor
BestMatches
3D Database
7
Matching with Descriptors

• Preprocessing
• Compute database descriptors
• Run-Time
• Compute query descriptor
• Compare query descriptor to database descriptors

3D Query
ShapeDescriptor
BestMatches
3D Database
8
Matching with Descriptors

• Preprocessing
• Compute database descriptors
• Run-Time
• Compute query descriptor
• Compare query descriptor to database descriptors
• Return best Match(es)

3D Query
ShapeDescriptor
BestMatches
3D Database
9
Shape Matching Challenge

• Need shape descriptor that is
• Concise to store
• Quick to compute
• Efficient to match
• Discriminating

3D Query
ShapeDescriptor
BestMatches
3D Database
10
Shape Matching Challenge

• Need shape descriptor that is
• Concise to store
• Quick to compute
• Efficient to match
• Discriminating

3D Query
ShapeDescriptor
BestMatches
3D Database
11
Shape Matching Challenge

• Need shape descriptor that is
• Concise to store
• Quick to compute
• Efficient to match
• Discriminating

3D Query
ShapeDescriptor
BestMatches
3D Database
12
Shape Matching Challenge

• Need shape descriptor that is
• Concise to store
• Quick to compute
• Efficient to match
• Discriminating

3D Query
ShapeDescriptor
BestMatches
3D Database
13
Shape Matching Challenge

• Need shape descriptor that is
• Concise to store
• Quick to compute
• Efficient to match
• Discriminating
• Invariant to transformations
• Invariant to deformations
• Insensitive to noise
• Insensitive to topology
• Robust to degeneracies

Different Transformations(translation, scale,
rotation, mirror)
14
Shape Matching Challenge

• Need shape descriptor that is
• Concise to store
• Quick to compute
• Efficient to match
• Discriminating
• Invariant to transformations
• Invariant to deformations
• Insensitive to noise
• Insensitive to topology
• Robust to degeneracies

Different Articulated Poses
15
Shape Matching Challenge

• Need shape descriptor that is
• Concise to store
• Quick to compute
• Efficient to match
• Discriminating
• Invariant to transformations
• Invariant to deformations
• Insensitive to noise
• Insensitive to topology
• Robust to degeneracies

Scanned Surface
Image courtesy ofRamamoorthi et al.
16
Shape Matching Challenge

• Need shape descriptor that is
• Concise to store
• Quick to compute
• Efficient to match
• Discriminating
• Invariant to transformations
• Invariant to deformations
• Insensitive to noise
• Insensitive to topology
• Robust to degeneracies

Different Genus
Different Tessellations
Images courtesy of Viewpoint Stanford
17
Shape Matching Challenge

• Need shape descriptor that is
• Concise to store
• Quick to compute
• Efficient to match
• Discriminating
• Invariant to transformations
• Invariant to deformations
• Insensitive to noise
• Insensitive to topology
• Robust to degeneracies

No Bottom!
Q?_at_A!
Images courtesy of Utah De Espona
18

• Outline
• Shape Descriptors
• Statistical Shape Descriptors
• Singular Value Decomposition (SVD)

19
Statistical Shape Descriptors

• ChallengeWant a simple shape descriptor that is
  easy to compare and gives a continuous measure of
  the similarity between two models.
• SolutionRepresent each model by a vector and
  define the distance between models as the
  distance between corresponding vectors.

20
Statistical Shape Descriptors

• Properties
• Structured representation
• Easy to compare
• Generalizes the matching problem

Models represented as points in a fixed
dimensional vector space
21
Statistical Shape Descriptors

• General Approaches
• Retrieval
• Clustering
• Compression
• Hierarchicalrepresentation

Models represented as points in a fixed
dimensional vector space
22

• Outline
• Shape Descriptors
• Statistical Shape Descriptors
• Singular Value Decomposition (SVD)

23
Complexity of Retrieval

• Given a query
• Compute the distance to each database model
• Sort the database models by proximity
• Return the closest matches

M1
M1


M2
M2
M1
D(Q,Mi)
Q

3D Query
M2

Mk
Mk
Best Match(es)
Database Models
Sorted Models
24
Complexity of Retrieval

• If there are k models in the database and each
  model is represented by an n-dimensional vector
• Computing the distance to each database model
• O(k n) time
• Sort the database models by proximity
• O(k logk) time
• If n is large, retrieval will be prohibitively
  slow.

25
Algebra

• Definition Given a vector space V and a subspace
  W?V, the projection onto W, written ?W, is the
  map that sends v?V to the nearest vector in W.
• If w1,,wm is an orthonormal basis for W, then

26
Tensor Algebra

• Definition The inner product of two
  n-dimensional vectors vv1,,vn and
  ww1,,wn, written ?v,w?, is the scalar value
  defined by

27
Tensor Algebra

• Definition The outer product of two
  n-dimensional vectors vv1,,vn and
  ww1,,wn, written v?w, is the matrix defined
  by

28
Tensor Algebra

• Definition The transpose of an mxn matrix M,
  written Mt, is the nxm matrix with
• Property For any two vectors v and w, the
  transpose has the property

29
SVD Compression

• Key IdeaGiven a collection of vectors in
  n-dimensional space, find a good m-dimensional
  subspace (mltltn) in which to represent the vectors.

30
SVD Compression

• SpecificallyIf Pp1,,pk is the initial
  n-dimensional point set, and w1,,wm is an
  orthonormal basis for the m-dimensional subspace,
  we will compress the point set by sending

31
SVD Compression

• ChallengeTo find the m-dimensional subspace
  that will best capture the initial point
  information.

32
Variance of the Point Set

• Given a collection of points Pp1,,pk, in an
  n-dimensional vector space, determine how the
  vectors are distributedacross different
  directions.

p1
pi
p2
pk
33
Variance of the Point Set

• Define the VarP as the functiongiving the
  variance of thepoint set P in direction
  v(assume v1).

p1
pi
p2
pk
34
Variance of the Point Set

• More generally, for a subspace W?V, define the
  variance of P in the subspace W asIf
  w1,,wm is an orthonormal basis for W, then

35
Variance of the Point Set

• Example
• The variance in the direction v1is large, while
  the variance inthe direction v2 is small.
• ? If we want to compressdown to one dimension,
  weshould project the points onto v1

p1
pi
v1
p2
v2
pk
36
Covariance Matrix

• Definition The covariance matrix MP, of a point
  set Pp1,,pk is the symmetric matrix which is
  the sum of the outer products of the pi

37
Covariance Matrix

• Theorem The variance of the point set P in a
  direction v is equal to

38
Covariance Matrix

• Theorem The variance of the point set P in a
  direction v is equal to
• Proof

39
Singular Value Decomposition

• Theorem Every symmetric matrix M can be written
  out as the productwhere O is a
  rotation/reflection matrix (OOtId) and D is a
  diagonal matrix with the property

40
Singular Value Decomposition

• Implication
• Given a point set P, we can compute the
  covariance matrix of the point set, MP, and
  express the matrix in terms of its SVD
  factorizationwhere v1,,vn is an
  orthonormal basis and ?i is the variance of the
  point set in direction vi.

41
Singular Value Decomposition

• Compression
• The vector subspace spanned by v1,,vm is the
  vector sub-space that maximizes the variance in
  the initial point set P.
• If m is too small, then too much information is
  discarded and there will be a loss in retrieval
  accuracy.

42
Singular Value Decomposition

• Hierarchical Matching
• First coarsely compare the query to database
  vectors.
• If query is coarsely similar to target
• Refine the comparison
• Else
• Do not refine
• O(k n) matching becomes O(k m) with mltltn and no
  loss of retrieval accuracy.

43
Singular Value Decomposition

• Hierarchical Matching
• SVD expresses the initial vectors in terms of the
  eigenbasisBecause there is more variance in v1
  than in v2, more variance in v2 than in v3, etc.
  this gives a hierarchical representation of the
  data so that coarse comparisons can be performed
  by comparing only the first m coefficients.

44
Efficient to match?

• Preprocessing
• Compute SVD factorization
• Transform database descriptors
• Run-Time
• Transform Query

Query
SVD
SVD
45
Efficient to match?

1. Low resolution sort

Database
Query
?0.040
?0.052
?0.103
?0.661
?0.430
Distance to Query
46
Efficient to match?

1. Update closest matches
2. Resort

Database
Query
?0.229
?0.052
?0.103
?0.661
?0.430
Distance to Query
47
Efficient to match?

1. Update closest matches
2. Resort

Database
Query
?0.229
?0.141
?0.103
?0.661
?0.430
Distance to Query
48
Efficient to match?

1. Update closest matches
2. Resort

Database
Query
?0.229
?0.141
?0.189
?0.661
?0.430
Distance to Query
49
Efficient to match?

1. Update closest matches
2. Resort

Database
Query
?0.229
?0.189
?0.230
?0.661
?0.430
Distance to Query
50
Efficient to match?

1. Update closest matches
2. Resort

Database
Query
?0.229
?0.200
?0.661
?0.430
?0.230
Distance to Query
51
Efficient to match?

1. Update closest matches
2. Resort

Database
Query
?0.229
0.289
?0.661
?0.430
?0.230
Distance to Query
52
Efficient to match?

1. Update closest matches
2. Resort

Database
Query
?0.334
0.289
?0.661
?0.430
?0.230
Distance to Query
53
Efficient to match?

1. Update closest matches
2. Resort

Database
Query
?0.334
0.289
?0.661
?0.430
0.301
Distance to Query
54
Efficient to match?

1. Update closest matches
2. Resort

Database
Query
?0.334
0.289
0.301
?0.661
?0.430
Distance to Query
55
Singular Value Decomposition

• Theorem Every symmetric matrix M can be written
  out as the productwhere O is a
  rotation/reflection matrix (OOtId) and D is a
  diagonal matrix with the property

56
Singular Value Decomposition

• Proof
• Every symmetric matrix has at least one real
  eigenvector v.
• If v is an eigenvector and w is perpendicular to
  v then Mw is also perpendicular to v.

v
w?v?
Since M maps the subspace of vectors
perpendicular to v back into itself, we can look
at the restriction of M to the subspace and
iterate to get the next eigenvector.
57
Singular Value Decomposition

• Proof (Step 1)
• Let F(v) be the function on the unit sphere
  (v1) defined by

F(v)
v
58
Singular Value Decomposition

• Proof (Step 1)
• Let F(v) be the function on the unit sphere
  (v1) defined byThen F must have a
  maximumat some point v0.

F(v0)
v0
59
Singular Value Decomposition

• Proof (Step 1)
• Let F(v) be the function on the unit sphere
  (v1) defined byThen F must have a
  maximumat some point v0.
• Then ?F(v0)?v0.

F(v0)
v0
60
Singular Value Decomposition

• If F has a maximum at some point v0 then
  ?F(v0)?v0.If w0 is on the sphere, next to v0,
  then w0-v0 is nearly perpendicular to v0. And for
  any small vector w1 perpendicular to v0, v0 w1
  is nearly on the sphere

61
Singular Value Decomposition

• If F has a maximum at some point v0 then
  ?F(v0)?v0.For small values of w0 close to v0,
  we have
• For v0 to be a maximum, we must havefor all
  w0 near v0.
• Thus, ?F(v0) must be perpendicular to all vectors
  that are perpendicular to v0, and hence must
  itself be a multiple of v0.

62
Singular Value Decomposition

• Proof (Step 1)
• Let F(v) be the function on the unit sphere
  (v1) defined byThen F must have a
  maximumat some point v0.
• Then ?F(v0)?v0.

F(v0)
v0
63
Singular Value Decomposition

• Proof (Step 1)
• Let F(v) be the function on the unit sphere
  (v1) defined byThen F must have a
  maximumat some point v0.
• Then ?F(v0)?v0.
• But ?F(v)2Mv

F(v0)
v0
? v0 is an eigenvector of M.
64
Singular Value Decomposition

• Proof
• Every symmetric matrix has at least one
  eigenvector v.
• If v is an eigenvector and w is perpendicular to
  v then Mw is also perpendicular to v.

65
Singular Value Decomposition

• Proof (Step 2)
• If w is perpendicular to v, then ?v,w?0.
• Since M is symmetricso that Mw is also
  perpendicular to v.