The Story of Wavelets Theory and Engineering Applications

1
The Story of WaveletsTheory and Engineering
Applications
Jamboree 2 January 24, 2001

• In todays show
• Fundamentals of linear algebra
• Vector spaces
• Basis vectors, vector norms, inner product
• Orthogonality, orthogonal and orthonormal bases,
  biorthogonal bases, frames
• Frequency representations of periodic signals
  Fourier series
• Frequency representations of aperiodic signals
  Fourier transform

2
Vector Spaces

• Definition and properties A complex (real)
  vector space V is a set of elements u, v, etc.
  called vectors, with two arithmetic operations
  defined on these elements
• Addition
• Scalar multiplication
• Vector spaces have the following properties
• Commutativity
• Associativity
• Distributivity
• Zero vector
• Negative vector

3
Vector Subspace

• A subset W of a vector space V is called a
  subspace of V, as long as W itself is a vector
  space under the addition and scalar
  multiplication operations defined on V.
• Note that the definitions of addition and scalar
  multiplication do not specify the nature of
  vectors and / or the operations. As long as the
  properties are satisfied, anything can be a
  vector and any operation can be substituted for
  addition and scalar multiplication.
• For example Let humans be vectors, reproduction
  be the addition operation and putting on a
  clothing be the scalar multiplication. Is this a
  valid vector space? Why / why not?

4
Linear Combination

• A linear combination of vectors is another vector
  defined as
• where are scalar coefficients.
• A collection of vectors are linearly dependent,
  if any one of them can be written as a linear
  combination of others with at least one non-zero
  scalar. Otherwise, they are linearly independent.

5
Span

• If Sv1,v2,,vn is a set of vectors in vector
  space V, then the subspace W which consists of
  all linear combinations of these vectors in S is
  called the space spanned by Sv1,v2,,vn. Or,
  we say, v1,v2,,vn span W.

6
Basis Vectors

• A basis of V is a collection of linearly
  independent vectors such that any vector v in V
  can be written as a linear combination of these
  basis vectors. That if Bu1,u2,,un is a basis
  for V, than any v in V can be written as
• The alpha coefficients are unique for each v.
  Note that basis vectors span V.
• The number of vectors in a basis defines the
  dimension of the vector space V

7
Vector Norms

• A norm on a vector space V is a mapping which
  assigns a length (norm) , to each v
  with the following properties
• is real and positive.
• 0 if and only if v0
• For discrete sequences the lp norm is defined as
• For continuous functions, Lp norm is

8
Inner Product

• An inner product on a vector space is a mapping
  which assigns a complex (real) number ltv,wgt to
  each pair v,w in V, such that

9
Inner Products for Standard Norms

• For discrete sequences on lp norm, Euclidean
  inner product
• For continuous functions on Lp norm

10
Orthogonality

• Two vectors v,w are orthogonal (perpendicular) if
  ltv,wgt0
• If, furthermore, they both have the length 1,
  i.e.,
• Then these vectors are orthonormal.
• A collection of vectors v1,v2,,vn are
  orthonormal, if they are pair wise orthogonal and
  all have length 1, i.e.,

11
Orthonormal Bases

• Recall if v1,v2,,vn constitutes a basis for
  V, than any vector (function) in V can be written
  as
• However, ?j may be difficult to compute. If vj
  form an orthonormal basis, this difficulty is
  eliminated, since then
• Thus if vj is a set of orthonormal basis for V,
  than any w in V can be written as

12
Orthogonal Projections

• If W is a finite dimensional subspace of V, then
  every vector u in V can be expressed in exactly
  one way as sum of two perpendicular vectors
  uw1w2, where w1is in W, and w2 is in the
  orthogonal complement of W. Then, w1 is called
  the orthogonal projection, Pwu of u on W.
• Let W be a finite dimensional subspace of a
  vector space V with the Euclidean inner product.
  If v1,,vn is an orthonormal basis for W, and u
  is in V, then

13
Biorthogonal bases

• Sometimes, orthonormal bases are not available. A
  generalization is the concept of biorthogonal
  bases , which are actually a pair of bases.
• If vj and wj are both basis vectors
  themselves for V, satisfy
• then any u in V can be written as
• Biorthogonal bases are not numerically as
  stable as orthonormal bases, are they are just as
  easy to work with.

14
Frames

• Sometimes, biorthogonal bases are not available
  either. We still would like to be able to
  represent any vector in V as a linear combination
  of some other (simpler) vectors / functions,
  while giving up orthogonality and even linear
  independence
• A sequence of vectors vj is called a frame with
  frame bounds A and B, if for any vector u in V
• If AB, the frame is said to be tight. If
  removing an element from the frame makes it no
  longer a frame, thenthe original frame is said to
  be exact.
• Any vector u can then be written as a linear
  combination of frame vectors. However, the
  coefficients are no longer unique, vectors are no
  longer independent, and hence the frame does not
  constitute a basis. Only exact frames are bases.