Digital media I: Audio

1
Digital media I Audio

• Glenn Bresnahan Robert Putnam
• glenn_at_bu.edu putnam_at_bu.edu

2
Outline

• Part I (Glenn)
• What is sound?
• How do we hear?
• Part II (Robert)
• Qualities of sound
• Sound reproduction
• analog v. digital
• Sound in VR

3
Waves revisited
4
Waves (non)artistic rendering
5
Wave properties

• How might we describe waves?

6
Wave properties

• How might we describe waves?
• Height
• Time between waves
• Speed of the wave

7
Shoals and tides
8
Tide tables
9
Cause of tides

• Gravity from moon and sun

1 day
New moon
27.3 days (29.5 days)
Full moon
365 days
10
Phases of the moon
Moon phases New moon Waxing crescent First
quarter Waxing gibbous Full moon Waning
gibbous Last quarter New moon
11
Moon phases
12
Sunrise and sunset
13
Sunrise and sunset
solstice
solstice
14
Waves sine waves

• Sine wave is the fundamental wave

15
Waves properties
Wavelength (distance)
Amplitude
16
Waves in motion properties
Period (time for one cycle)
2
Frequency cycles per time interval
1
Time
17
What is sound?

• Examples

18
What is sound vibration

• Striking an object will cause it to vibrate
• The vibration is a sine wave
• Objects have a natural vibration frequency
• Resonance frequency
• Frequency depends on type of material, thickness,
  length/size, tension
• May have multiple vibrating frequencies
• The pitch depends on the frequency
• Loudness (amplitude) depends on size of the
  object

19
What is sound vibrations moves air
Wave
Energy (pluck)
Air pressure level
vibration
string
20
Properties of sound

• Pitch is perception of frequency
• Frequency is measured in cycles per second (cps)
• Hertz (Hz) cycles per second
• The A above middle C is 440 Hz.
• Humans hear appox. 20-20,000 Hz
• Sound travels at approx. 1100 feet/second
• Speed depends on pressure and temperature
• Approx. 750 miles/hour
• Approx. 1 mile every 4.8 seconds
• Perceived loudness depends on pressure level
• Sound pressure is measured in (micro)pascals
  (20uPa)
• Loudness is usually expressed in decibels (dB)

21
Real Waves
22
Properties of sound

• Real sounds are far more complex than simple sine
  waves
• Objects produce vibrations at multiple
  frequencies
• Sound waves interact with other objects
• Waves bounce (reflect) off surface
• Reverberation/echo
• Wave are absorbed by materials
• Sound waves interact with each other

23
Combinations of waves
24
Properties of sound real sounds
25
Electrification of sound

• Microphones
• Convert pressure levels into electrical signals
  (voltages)
• Guitar pickups
• Converts string vibration to voltages
• The pickup contains a magnet and a coil
• The vibrating metal strings alter the magnetic
  field and induce a voltage in the coil
• Loud speakers convert an electrical signal back
  into air pressure

26
How do we hear?

• Sound waves move through the air from the sound
  source to the ear

27
Anatomy of the ear
28
Anatomy of the ear - outer

• Divided into three principal sections
• Outer ear
• Middle ear
• Inner ear
• Outer ear
• External ear, aka pinna
• Ear canal
• Outer ear funnels the ear have to the eardrum

29
Anatomy of the ear - middle

• Middle ear
• Eardrum
• Set of 3 ear bones
• the 3 bones are rigid
• Act as a mechanical amplifier
• The 3rd bone, stapes, induces a vibration into
  the inner ear, i.e. the cochlea

30
Anatomy of the ear - inner

• Inner ear / cochlea
• Where the real work is done
• Cochlea is a spiral tube and filled with fluid
• Stapes causes a wave to pass through the fluid

31
Anatomy of the ear - inner

• Cochlea is a spiral tube lined with hair cells on
  a membrane (15K HCs)
• Hairs vary in length and thickness along the tube
• Hairs resonate at different frequencies
• High freq on near end, low at far end

32
Anatomy of the ear - inner
33
Anatomy of the ear - inner

• Hair cells are connected to the auditory nerve
  cells
• The vibrations excite the nerve cells and cause
  them to fire (electrical signal)
• A series of nerve cells pass the signal to brain

34
Binaural hearing - why two ears?

• Two ears, so we can identify locations of sounds
• Time difference
• Intensity difference
• Sound color difference (caused by movement of
  sound around head and shoulders)

35
Sound localization pinna
back

• Sound waves interact with the asymmetric Pinna
• The effect on the sound varies with the direction
• Up/down, back/front waves result in different
  sounds entering ear canal

front
36
Digital media I part II

• Other qualities of sound pitch, timbre, noise,
  envelope
• Sound reproduction analog v. digital
• Sound in VR

37
What is pitch?

• Our perception of the highness or lowness of a
  tone.
• Closely related to frequency
• When frequency doubles, pitch rises by an
  octave
• Examples
• But, what happens when theres more than one
  frequency in a sound?

38
Review modes of vibration of a string

• Fundamental
• e.g., 110 Hz
• 2nd harmonic
• e.g., 220 Hz
• 3rd harmonic
• e.g., 330 Hz

Examples
39
Timbre

• Sound color, or timbre is a quality of sound
  that derives from the particular combination of
  frequencies (a.k.a., harmonics or partials)
  in a tone.
• Two sounds can contain the same harmonics but
  sound very different because their individual
  harmonics are of different amplitudes.
• Examples

40
Timbre, continued

• Easy to demonstrate timbre with human voice
• Hum.
• Slowly open mouth.
• Hear how the sound color changes from dark to
  bright
• Example

41
Timbre, continued.

• Timbre changes as a wind instrument is played
  louder or softer.
• Example

42
Unpitched sounds

• Can use human voice to demonstrate another
  distinction pitched versus unpitched sounds
• Make s sound
• No identifiable pitch
• Related to concept of noise
• Examples

43
Examples

• Pitched sounds
• Birdsong
• Flutes
• Stringed instruments
• Etc.
• Examples of unpitched sounds
• Certain percussion instruments (cymbals,
  ratchets, etc.)
• Wind, rain, footsteps in snow
• Listen now. What do you hear? Frequencies,
  amplitudes. Pitched, unpitched. External versus
  internal sources.

44
Time variation of sounds

• Most naturally occurring sounds are not static
  i.e., they vary over time
• Amplitude
• Pitch
• Timbre
• Examples

45
Sound recording technologies

• Analog
• Digital

46
Analog recording

• Analog device or system that represents
  changing values as continuously variable physical
  quantities.
• Example clock with hour, minute and second hands
• Question what values are changing when we hear
  sound?

47
Analog recording technologies

• Phonautograph

48
Analog recording technologies

• Mechanical Gramophone, LP record, etc.

49
Analog recording technologies

• Magnetic Wire, tape recorder.

50
Analog recording technologies

• Optical movie soundtrack.

51
Analog sound reproduction

• Amplification
• Loudspeaker demo

52
Digital recording

• Digital device or system that represents
  changing values as discontinuous, or discrete,
  values.
• Example clock with number readout.

53
Digital recording

• With digital recording, we do not store a
  continuous record of the rise and fall of air
  pressure. We make measurements of the air
  pressure (or the voltage produced by a
  microphone) thousands of times per second and
  store these measurements as numbers.

54
Digital recording

• Analog continuous waveform
• Digital discrete values

55
Some buzzwords

• ADC analog-to-digital converter
• DAC digital-to-analog converter
• Sampling rate samples/second
• Word size how much storage for each sample
• Quantization see next slide

56
Quantization

• Selecting sample value from finite set of
  numbers.
• 16 bits 65,536 choices
• 20 bits 1,048,576 choices

Source http//advisor.matrasi-tls.fr/digital_samp
ling_index.html
57
CD audio

• 44,100 samples per second.
• 16-bit samples (65536 different possible values)
• Frequency range 0-22KHz.

58
Some benefits of digital audio

• Easy to edit (visual interface)
• No noise with additional generations
• Flexible signal processing (no special hardware)
• Examples (reverb, pitch shift, noise reduction,
  etc.)

59
An aside MP3 audio

• CD takes up a lot of space 3 minute song
  4410022360 31752000 B.
• MP3 compression results in a factor of 5-10
  savings in storage, but lower fidelity (e.g.,
  noisier).

60
Another aside MIDI

• Musical Instrument Digital Interface
• A communications scheme for computers,
  synthesizers, sequencers, etc.
• Suited for popular music
• Stores Note-On, Note-Off, Velocity, etc. (i.e.,
  not waveforms)
• Example

61
DAFFIE audio

• Soundserver
• Plays sound files associated with objects
• Mixes many simultaneous sounds
• Internet telephony support

62
DAFFIE localization

• All sounds are assigned a location in virtual
  space.
• Typically, sounds are associated with visible
  objects, but ambient sound (e.g., wind, nature)
  is supported too.
• Direction and distance are indicated by
  variations in loudness among the loudspeakers
  done automatically by soundserver.

63
Uses of sound in VR

• Communication (via telephony)
• Sound effects
• Music selections
• Ambient audio
• Live audio (via telephony)
• Previous projects have involved controlling
  synthesizers or musical instruments remotely.

64
Recording sounds for DAFFIE

• Field recording versus studio recording
• Record, record, record
• Remember that sounds are combined in real time
  by the soundserver (so no need to put everything
  in a single file).
• Stereo is good.

65
Recording sounds for DAFFIE, part II.

• Remember that you can change sounds in various
  ways
• Change pitch/tempo
• Use filters to change spectrum
• Cut and paste

66
DAFFIE demo

• Demonstrate
• Proximity triggering
• (Variable) Distance attenuation
• Sound localization