Signal%20Processing%20and%20Representation%20Theory

1
Signal Processingand Representation Theory

• Lecture 1

2

• Outline
• Algebra Review
• Numbers
• Groups
• Vector Spaces
• Inner Product Spaces
• Orthogonal / Unitary Operators
• Representation Theory

3
Algebra Review

• Numbers (Reals)
• Real numbers, R, are the set of numbers that we
  express in decimal notation, possibly with
  infinite, non-repeating, precision.

4
Algebra Review

• Numbers (Reals)
• Example ?3.1415926535897932384626433832795028841
  97
• Completeness If a sequence of real numbers gets
  progressively tighter then it must converge to
  a real number.
• Size The size of a real number a?R is the square
  root of its square norm

5
Algebra Review

• Numbers (Complexes)
• Complex numbers, C, are the set of numbers that
  we express as aib, where a,b?R and i .
• Example ei?cos?isin?

6
Algebra Review

• Numbers (Complexes)
• Let p(x)xnan-1xn-1a1x1a0 be a polynomial
  with ai?C.
• Algebraic Closure
• p(x) must have a root, x0 in C
• p(x0)0.

7
Algebra Review

• Numbers (Complexes)
• Conjugate The conjugate of a complex number aib
  is
• Size The size of a real number aib?C is the
  square root of its square norm

8
Algebra Review

• Groups
• A group G is a set with a composition rule that
  takes two elements of the set and returns another
  element, satisfying
• Asscociativity (ab)ca(bc) for all a,b,c?G.
• Identity There exists an identity element 0?G
  such that 0aa0a for all a?G.
• Inverse For every a?G there exists an element
  -a?G such that a(-a)0.
• If the group satisfies abba for all a,b?G,
  then the group is called commutative or abelian.

9
Algebra Review

• Groups
• Examples
• The integers, under addition, are a commutative
  group.
• The positive real numbers, under multiplication,
  are a commutative group.
• The set of complex numbers without 0, under
  multiplication, are a commutative group.
• Real/complex invertible matrices, under
  multiplication are a non-commutative group.
• The rotation matrices, under multiplication, are
  a non-commutative group. (Except in 2D when they
  are commutative)

10
Algebra Review

• (Real) Vector Spaces
• A real vector space is a set of objects that can
  be added together and scaled by real numbers.
• Formally
• A real vector space V is a commutative group with
  a scaling operator
• (a,v)?av,
• a?R, v?V, such that
• 1vv for all v?V.
• a(vw)avaw for all a?R, v,w?V.
• (ab)vavbv for all a,b?R, v?V.
• (ab)va(bv) for all a,b?R, v?V.

11
Algebra Review

• (Real) Vector Spaces
• Examples
• The set of n-dimensional arrays with real
  coefficients is a vector space.
• The set of mxn matrices with real entries is a
  vector space.
• The sets of real-valued functions defined in 1D,
  2D, 3D, are all vector spaces.
• The sets of real-valued functions defined on the
  circle, disk, sphere, ball, are all vector
  spaces.
• Etc.

12
Algebra Review

• (Complex) Vector Spaces
• A complex vector space is a set of objects that
  can be added together and scaled by complex
  numbers.
• Formally
• A complex vector space V is a commutative group
  with a scaling operator
• (a,v)?av,
• a?C, v?V, such that
• 1vv for all v?V.
• a(vw)avaw for all a?C, v,w?V.
• (ab)vavbv for all a,b?C, v?V.
• (ab)va(bv) for all a,b?C, v?V.

13
Algebra Review

• (Complex) Vector Spaces
• Examples
• The set of n-dimensional arrays with complex
  coefficients is a vector space.
• The set of mxn matrices with complex entries is a
  vector space.
• The sets of complex-valued functions defined in
  1D, 2D, 3D, are all vector spaces.
• The sets of complex-valued functions defined on
  the circle, disk, sphere, ball, are all vector
  spaces.
• Etc.

14
Algebra Review

• (Real) Inner Product Spaces
• A real inner product space is a real vector space
  V with a mapping ?V,V??R that takes a pair of
  vectors and returns a real number, satisfying
• ?u,vw? ?u,v? ?u,w? for all u,v,w?V.
• ?au,v?a?u,v? for all u,v?V and all a?R.
• ?u,v? ?v,u? for all u,v?V.
• ?v,v??0 for all v?V, and ?v,v?0 if and only if
  v0.

15
Algebra Review

• (Real) Inner Product Spaces
• Examples
• The space of n-dimensional arrays with real
  coefficients is an inner product space.If
  v(v1,,vn) and w(w1,,wn) then
• ?v,w?v1w1vnwn
• If M is a symmetric matrix (MMt) whose
  eigen-values are all positive, then the space of
  n-dimensional arrays with real coefficients is an
  inner product space.If v(v1,,vn) and
  w(w1,,wn) then
• ?v,w?MvMwt

16
Algebra Review

• (Real) Inner Product Spaces
• Examples
• The space of mxn matrices with real coefficients
  is an inner product space.If M and N are two mxn
  matrices then
• ?M,N?Trace(MtN)

17
Algebra Review

• (Real) Inner Product Spaces
• Examples
• The spaces of real-valued functions defined in
  1D, 2D, 3D, are real inner product space.If f
  and g are two functions in 1D, then
• The spaces of real-valued functions defined on
  the circle, disk, sphere, ball, are real inner
  product spaces.If f and g are two functions
  defined on the circle, then

18
Algebra Review

• (Complex) Inner Product Spaces
• A complex inner product space is a complex vector
  space V with a mapping ?V,V??C that takes a pair
  of vectors and returns a complex number,
  satisfying
• ?u,vw? ?u,v? ?u,w? for all u,v,w?V.
• ?au,v?a?u,v? for all u,v?V and all a?R.
• for all u,v?V.
• ?v,v??0 for all v?V, and ?v,v?0 if and only if
  v0.

19
Algebra Review

• (Complex) Inner Product Spaces
• Examples
• The space of n-dimensional arrays with complex
  coefficients is an inner product space.If
  v(v1,,vn) and w(w1,,wn) then
• If M is a conjugate symmetric matrix (
  ) whose eigen-values are all positive, then the
  space of n-dimensional arrays with complex
  coefficients is an inner product space.If
  v(v1,,vn) and w(w1,,wn) then
• ?v,w?MvMwt

20
Algebra Review

• (Complex) Inner Product Spaces
• Examples
• The space of mxn matrices with real coefficients
  is an inner product space.If M and N are two mxn
  matrices then

21
Algebra Review

• (Complex) Inner Product Spaces
• Examples
• The spaces of complex-valued functions defined in
  1D, 2D, 3D, are real inner product space.If f
  and g are two functions in 1D, then
• The spaces of real-valued functions defined on
  the circle, disk, sphere, ball, are real inner
  product spaces.If f and g are two functions
  defined on the circle, then

22
Algebra Review

• Inner Product Spaces
• If V1,V2?V, then V is the direct sum of subspaces
  V1, V2, written VV1?V2, if
• Every vector v?V can be written uniquely asfor
  some vectors v1?V1 and v2?V2.

23
Algebra Review

• Inner Product Spaces
• Example
• If V is the vector space of 4-dimensional arrays,
  then V is the direct sum of the vector spaces
  V1,V2?V where
• V1(x1,x2,0,0)
• V2(0,0,x3,x4)

24
Algebra Review

• Orthogonal / Unitary Operators
• If V is a real / complex inner product space,
  then a linear map AV?V is orthogonal / unitary
  if it preserves the inner product
• ?v,w? ?Av,Aw?
• for all v,w?V.

25
Algebra Review

• Orthogonal / Unitary Operators
• Examples
• If V is the space of real, two-dimensional,
  vectors and A is any rotation or reflection, then
  A is orthogonal.

A(v1)
v1
v2
A(v2)
A
26
Algebra Review

• Orthogonal / Unitary Operators
• Examples
• If V is the space of real, three-dimensional,
  vectors and A is any rotation or reflection, then
  A is orthogonal.

A
27
Algebra Review

• Orthogonal / Unitary Operators
• Examples
• If V is the space of functions defined in 1D and
  A is any translation, then A is orthogonal.

A
28
Algebra Review

• Orthogonal / Unitary Operators
• Examples
• If V is the space of functions defined on a
  circle and A is any rotation or reflection, then
  A is orthogonal.

A
29
Algebra Review

• Orthogonal / Unitary Operators
• Examples
• If V is the space of functions defined on a
  sphere and A is any rotation or reflection, then
  A is orthogonal.

A
30

• Outline
• Algebra Review
• Representation Theory
• Orthogonal / Unitary Representations
• Irreducible Representations
• Why Do We Care?

31
Representation Theory

• Orthogonal / Unitary Representation
• An orthogonal / unitary representation of a group
  G onto an inner product space V is a map ? that
  sends every element of G to an orthogonal /
  unitary transformation, subject to the
  conditions
• ?(0)vv, for all v?V, where 0 is the identity
  element.
• ?(gh)v?(g) ?(h)v

32
Representation Theory

• Orthogonal / Unitary Representation
• Examples
• If G is any group and V is any vector space,
  thenis an orthogonal / unitary
  representation.
• If G is the group of rotations and reflections
  and V is any vector space, thenis an
  orthogonal / unitary representation.

33
Representation Theory

• Orthogonal / Unitary Representation
• Examples
• If G is the group of nxn orthogonal / unitary
  matrices, and V is the space of n-dimensional
  arrays, thenis an orthogonal / unitary
  representation.

34
Representation Theory

• Orthogonal / Unitary Representation
• Examples
• If G is the group of 2x2 rotation matrices, and V
  is the vector space of 4-dimensional real /
  complex arrays, thenis an orthogonal /
  unitary representation.

35
Representation Theory

• Irreducible Representations
• A representation ?, of a group G onto a vector
  space V is irreducible if cannot be broken up
  into smaller representation spaces.
• That is, if there exist W?V such that
• ?(G)W?W
• Then either WV or W?.

36
Representation Theory

• Irreducible Representations
• If W?V is a sub-representation of G, and W? is
  the space of vectors perpendicular to W
• ?v,w?0
• for all v?W? and w?W, then VW?W? and W? is also
  a sub-representation of V.
• For any g?G, v?W?, and w?W, we have
• So if a representation is reducible, it can be
  broken up into the direct sum of two
  sub-representations.

37
Representation Theory

• Irreducible Representations
• Examples
• If G is any group and V is any vector space with
  dimension larger than one, thenis not an
  irreducible representation.

38
Representation Theory

• Irreducible Representations
• Examples
• If G is the group of 2x2 rotation matrices, and V
  is the vector space of 4-dimensional real /
  complex arrays, thenis not an irreducible
  representation since it maps the space
  W(x1,x2,0,0) back into itself.

39
Representation Theory

• Why do we care?

40
Representation Theory

• Why we care
• In shape matching we have to deal with the fact
  that rotations do not change the shape of a
  model.

41
Representation Theory

• Exhaustive Search
• If vM is a spherical function representing model
  M and vn is a spherical function representing
  model N, we want to find the minimum over all
  rotations T of the equation

42
Representation Theory

• Exhaustive Search
• If V is the space of spherical functions then we
  can consider the representation of the group of
  rotations on this space.
• By decomposing V into a direct sum of its
  irreducible representations, we get a better
  framework for finding the best rotation.

43
Representation Theory

• Exhaustive Search (Brute Force)
• Suppose that v1,,vn is some orthogonal basis
  for V, then we can express the shape descriptors
  in terms of this basis
• vMa1v1anvn
• vNb1v1bnvn

44
Representation Theory

• Exhaustive Search (Brute Force)
• Then the dot-product of M and N at a rotation T
  is equal to

45
Representation Theory

• Exhaustive Search (Brute Force)
• So that the nxn cross-multiplications are needed

v1
T(v1)


v2
T(v2)




vM
T(vN)




T(vn)
vn
46
Representation Theory

• Exhaustive Search (w/ Rep. Theory)
• Now suppose that we can decompose V into a
  collection of one-dimensional representations.
• That is, there exists an orthogonal basis
  w1,,wn of functions such that T(wi)?wiC for
  all rotations T and hence
• ?wi,T(wj)?0 for all i?j.

47
Representation Theory

• Exhaustive Search (w/ Rep. Theory)
• Then we can express the shape descriptors in
  terms of this basis
• vMa1w1anwn
• vNß1w1ßnwn

48
Representation Theory

• Exhaustive Search (w/ Rep. Theory)
• And the dot-product of M and N at a rotation T is
  equal to

49
Representation Theory

• Exhaustive Search (w/ Rep. Theory)
• So that only n multiplications are needed

w1
T(w1)


w2
T(w2)




vM
T(vN)




T(wn)
wn