ESE250: Digital Audio Basics

1
ESE250Digital Audio Basics

• Week 4
• October 6, 2009
• Time-Frequency

2
Course Map
Today
Numbers correspond to course weeks
3
Teaser Musical Representation
4
Teaser Musical Representation

• With this compact notation
• Could communicate a sound to pianist
• Much more compact than 44KHz time-sample
  amplitudes (fewer bits to represent)
• Represent frequencies

5
Week 4 Time-Frequency

• There are other ways to represent
• Frequency representation particularly efficient

In this lecture we will learn that the frequency
domain entails representing time-sampled
signals using a conveniently rotated coordinate
system
http//en.wikipedia.org/wiki/FileLead_Sheet.png
6
Prelude Harmonic Analysis

• Fourier Transform ( FT )
• Fourier ( other 19th Century Mathematicians)
• discovered that (real) signals
• can always (if they are smooth enough)
• be expressed as the sum of harmonics
• Defn Harmonics (Fourier Series)
• collections of periodic signals (e.g., cos, sin)
• whose frequencies are related by integer
  multiples
• arranged in order of increasing frequency
• summed in a linear combination
• whose coefficients provide
• an alternative representation

the job of this lecture is to replace this
signals-analysis perspective with a
symbols-synthesis perspective
7
A Sampled (Real) Signal
Measured Data
Sampled Signal
8
Reconstructing the Sampled Signal

• Exact Reconstruction
• May be possible
• Under the right assumptions
• Given the right model
• This example
• A harmonic signal
• Sampled in time
• Can be reconstructed
• exactly
• from the time-sampled values
• given knowledge of the harmonics

p(5/2) Cost

p(5/2) Sin2t

Cos0t, Sin1t, Cos1t, Sin2t, Cos2t,
Sin3t, Cos3t
9
Reconstructing the Sampled Signal

• Exact Reconstruction
• May be possible
• Under the right assumptions
• Given the right model
• This example
• A harmonic signal
• Sampled in time
• Can be reconstructed
• exactly
• from the time-sampled values
• given knowledge of the harmonics

p(5/2) Cos1t

p(5/2) Sin2t

Cos0t, Sin1t, Cos1t, Sin2t, Cos2t,
Sin3t, Cos3t
10
Fourier Analysis
Time-Values
Frequency-Amplitudes
FT
Sampled
DFT
Quantized
11
Another Sampled (Real) Signal
Measured Data
Sampled Signal
v
t
12
ApproximateReconstruction
Sum up the (black) harmonics using the (green)
coefficients
(Successively Thinner Green Dashed Curves Denote
Successively Fewer Harmonic Components)
13
Approximating the Sampled Signal

• Approximate Reconstruction
• is always achievable
• and more relevant
• to our problem
• Example
• A roughly harmonic signal
• Sampled in time
• Can be approximated
• arbitrarily closely
• from the time-sampled values
• using any good set of harmonics

Cos0t, Sint, Cost, Sin2t, Cos2t,
Sin3t, Cos3t
14
More Harmonics are Better
7 Samples 7 Harmonics
11 Samples 11 Harmonics
15 Samples 15 Harmonics
15
Yet Another Sampled (Real) Signal
Measured Data
Sampled Signal
v
t
16
Some Signals Dislike Some Harmonics

• Approximate Reconstruction
• although always achievable
• may require a lot of samples
• to get good performance
• from poorly chosen harmonics
• Different bases
• match different data
• better or worse
• (sometimes time is better than frequency)

15 Samples Harmonics
21 Samples Harmonics
31 Samples Harmonics
17
Intuitive Concept Inventory
11 Samples
11 Harmonics
Time Domain
Frequency Domain
r (received signal)
Q FT(q)
q
Q
(sampling)
18
Intuitive Concept Inventory
11 Samples
11 Harmonics
Time Domain
Frequency Domain
r (received signal)
Q DFT(q)
Sampling Quantization
this weeks idea
q
Q
Floating Point
Floating Point
Perceptual coding
19
Where Are We Heading After Today?

• q PCM r quantL SampleTsr
• Week 2
• Received signal is
• discrete-time-stamped
• quantized
• c code q
• Week 3
• Quantized Signal is Coded
• Q DFT q
• Week 4
• Sampled signal
• not coded directly
• but rather, Float -ed
• then linearly transformed
• into frequency domain

Generic Digital Signal Processor
q
c
Psychoacoustic Audio Coder
Q
q
c
Painter Spanias. Proc.IEEE, 88(4)451512,
2000
20
Interlude Audio Communications
21
Technical Concept Inventory

• Floating Point Quantization
• a symbolic representation
• admitting a mimic of continuous arithmetic
• Vectors
• sampled signals are points
• in a (high dimensional) vector space
• Linear Algebra
• the Swiss Army Knife of high dimensions
• provides a logical, geometric, and computational
• toolset for manipulating vectors
• Change of Basis
• DFT is a high dimensional rotation
• in the vector space of time-sampled signals

22
Technical Concept Inventory

• Floating Point Quantization
• a symbolic representation
• admitting a mimic of continuous arithmetic
• Vectors
• sampled signals are points
• in a (high dimensional) vector space
• Linear Algebra
• the Swiss Army Knife of high dimensions
• provides a logical, geometric, and computational
• toolset for manipulating vectors
• Change of Basis
• DFT is a high dimensional rotation
• in the vector space of time-sampled signals

23
Float-Quantized Symbols Act Real
r(t)

• q PCM r(t) Float(b,p,E) SampleTsr(t)
• eliminates continuous time dependence
• discretizes continuous values
• cannot represent an uncountable collection of
  functions
• with a countable (of course, in fact, finite!)
  set of symbols
• Floating Point Representation and Computer
  Arithmetic
• Choose Base (b), Precision (p), Magnitude (E)
• q be d0 d1 b-1 dp-1 b-(p-1)
• - E e E
• 0 lt di lt b
• Non-uniform quantization
• bp different mantissas
• 2E different exponents
• Log22E Log2bp bits
• Associated Flop Arithmetic
• op 2 , -, , / Sqrt, Mod, Flint
• ) Flop(x,y) Float op(x,y)
• Archetypal Computation Inner product
• x (x1, .., xn), y (y1, , yn)

q2
q3
q1
q4
q5
Widrow, et al., IEEE TIM96
Crucially important operation for signal
processing applications !
24
Technical Concept Inventory

• Floating Point Quantization
• a symbolic representation
• admitting a mimic of continuous arithmetic
• Vectors
• sampled signals are points
• in a (high dimensional) vector space
• Linear Algebra
• the Swiss Army Knife of high dimensions
• provides a logical, geometric, and computational
• toolset for manipulating vectors
• Change of Basis
• DFT is a high dimensional rotation
• in the vector space of time-sampled signals

25
Time Functions are Vectors
r(t)
q2
q1
q4
q3
q5

• Sampled received signal
• Is a discrete sequence of time-stamped floats
• q (q1, q2, qns)
• Float( r(T0Ts), r(T0 2Ts), . , r(T0
  nsTs))
• of real (i.e. Floated) values
• at each of the ns time-stamps
• Think of each of the time-stamps
• as an axis
• of real (float) values

26
Time Functions are Vectors

• Think of each of the time-stamps as an axis of
  real (float) values
• E.g., for three time stamps, ns 3,
• we can record the values
• arrange each axis located perpendicular
• to the other two in space
• mark their values
• and interpret them as a vector

27
Time Functions are Vectors

• Think of each of the time-stamps as an axis of
  real (float) values
• E.g., for two time stamps, ns 2,
• we can draw both axes
• on graph paper
• for a greater number of time stamps
• we can imagine arranging each axis
• in a mutually perpendicular direction
• in space of appropriately high dimension

b2
b1
t - 6.28
q1
q
q2
t 2.5
28
Technical Concept Inventory

• Floating Point Quantization
• a symbolic representation
• admitting a mimic of continuous arithmetic
• Vectors
• sampled signals are points
• in a (high dimensional) vector space
• Linear Algebra
• the Swiss Army Knife of high dimensions
• provides a logical, geometric, and computational
• toolset for manipulating vectors
• Change of Basis
• DFT is a high dimensional rotation
• in the vector space of time-sampled signals

29
Linear Algebra Swiss Army Knife

• We cannot see in high dimensions
• Linear Algebra enables us in high dimensions to
• reason precisely
• think geometrically
• compute
• Essential Ideas
• Basis expansion
• Change of basis
• Ingredients
• Orthonormality
• Inner Product h , i

r(t)
b2
q1
b1
t - 6.28
q1
q
q2
q2
t 2.5
BT b1 , b2 (1,0), (1,0)
q (q1, q2) (0.8, - 0.9) 0.8 (1,0)
0.9 (1,0) 0.8 b1 ( 0.9) b2
hq,b1i b1 hq,b2i b2 q1 b1
q2 b2
where hx,yi x1y1 x2y2 hq,b1i 0.8 1
(-0.9) 0 0.8 hq,b2i 0.8 0 (-0.9) 1
- 0.9
(computational definition)
30
Linear Algebra Swiss Army Knife
q1
q2

• Orthonormal Basis
• set of unit length vectors
• each perpendicular to all the others
• total number given by dimension of the space
• Inner Product
• (scaled) cosine of relative angle
• scales unit length

• geometric re-interpretation of computational
  definition
• hx,yi x1y1 x2y2

Generally hr, si Length(r)
Length(s) Cos Å(r,s) ) hr, ri
Length(r)2
q1 hq,b1i Length(q ) Cos Å(q,b1)
b2
Å(q,b2)
b1
t - 6.28
Å(q,b1)
q
q2 hq,b2i Length(q ) Cos Å(q,b2)
t 2.5
31
Technical Concept Inventory

• Floating Point Quantization
• a symbolic representation
• admitting a mimic of continuous arithmetic
• Vectors
• sampled signals are points
• in a (high dimensional) vector space
• Linear Algebra
• the Swiss Army Knife of high dimensions
• provides a logical, geometric, and computational
• toolset for manipulating vectors
• Change of Basis
• DFT is a high dimensional rotation
• in the vector space of time-sampled signals

32
Change of Coordinates
Vs. Independence Hall 500 Chestnut St.
Google Maps
33
Why Change Basis ?

• Efficiency
• data sets often lie along
• lower dimensional subspaces
• Of high dimensional data space
• Decoupling
• receiver model may prefer
• a specific basis

34
Linear Algebra Change of Basis

• Goal
• Re-express q
• In terms of BH
• Notation
• use new symbol, Q
• denoting different computational representation
• even though vector is geometrically unchanged
• Check good basis?
• mutually perpendicular vectors?
• both unit length?
• Further geometric Interpretation
• if old basis is orthonormal
• then new basis is also
• if and only if it is
• A rotation
• Away from the old

b2
H1
H2
b1
t - 6.28
BH H1 , H2 (1/p2 , 1/p2), (- 1/p2
, 1/p2)
- Q2
Q
t 2.5
-Q1
H1 h b1, H1 i 1/p
2 2 1/p 2 2 ½
½
1
Length(H1)2 h H1, H1 i
1/p 2 2 1/p 2 2
½ ½
1
Length(H2)2 h H2, H2 i
1/p 2 2 1/p 2 2
½ ½
1
hH1, H2i h11 h21 h12 h22
- 1/p 2 2 1/p 2 2
0
35
Linear Algebra Change of Basis

• Goal
• Re-express q (q1, q2)
• specified by coordinate representation
• in terms of the old basis, BT
• As Q Q1, Q2
• Specified by coordinate representation
• In terms of rotate basis, BH
• Idea
• recall geometric meaning
• of q (q1, q2)
• scale b1 by q1 h b1, q i
• scale b2 by q2 h b2, q i
• form the resultant vector
• Compute Q Q1, Q2
• using same geometric idea
• reveals how to obtain Q1, Q2
• scale H1 by Q1 hq,H1i
• scale H2 by Q2 hq,H2i
• form the resultant vector

) Q2 hq , H2i h (0.8, - 0.9),
(-1/p2, 1/p2)i - (0.8/1.1
0.9/1.1) ¼ - 1.6
b2
H1
H2
b1
t - 6.28
- Q2
Q
t 2.5
) Q1 hq , H1i h (0.8, - 0.9),
(1/p2, 1/p2)i (0.8/1.1 - 0.9/1.1)
¼ - 0.11
-Q1
q (q1, q2) q1 b1 q2
b2 hq, b1i b1 hq, b2i b2
Q Q1, Q2 hQ,H1i H1 hQ,H2i
H2 hq,H1i H1 hq,H2i H2
36
Generalize to ns 3 Samples
H0 Float h0(-2?/3), h0(0?/3), h0(2?/3)
h0(t) Cos0t/p3

• The 3-sample DFT
• take inner products
• of sampled signal
• with each harmonic

h1(t) 2 Sint/p3
H2 Float h2(-2?/3), h2(-0?/3), h2(2?/3)
H1 Float h1(-2?/3), h1(0?/3), h1(2?/3)
h2(t) 2 Cost/p3
37
Generalize to ns 3 Samples
h0(t) Cos0t/p3
h1(t) 2 Sint/p3
h2(t) 2 Cost/p3
38
Generalize to Arbitrary Samples
11 Samples
11 Harmonics
Time Domain
Frequency Domain
r (received signal)
Q DFT(q)
Sampling Quantization
this weeks idea
q
Q
Floating Point
Floating Point
Perceptual coding
39
for more understanding.

• Courses
• ESE 325 !
• (Math 240) ) Math 312 !!!
• Reading
• Quantization
• B. Widrow, I. Kollar, and M. C. Liu.
  Statistical theory of quantization. IEEE
  Transactions on Instrumentation and Measurement,
  45(2)353361, 1996.
• Floating Point
• D. Goldberg. What every computer scientist
  should know about floating-point arithmetic. ACM
  Computing Surveys, 23(1), 1991.
• Linear Algebra for Frequency Transformations
• G. Strang. The discrete cosine transform. SIAM
  Review, 41(1)135147, 1999

40
ESE250Digital Audio Basics

• End Week 4 Lecture
• Time-Frequency