Why do we hear what we hear?

1
Why do we hear what we hear?

• James D. Johnston
• Chief Scientist, DTS, Inc.

2
First, some notes

• The talk Im about to give presents ideas
  gathered from a variety of papers and
  experiments, done by many people, over a long
  period of time.
• It is not inviolate.
• It is a discussion of phenomena
• The mechanism is, in most cases, unknown, once
  one gets beyond the basilar membrane
• There will be revisions as time goes on.
• There will ALWAYS be revisions

3
The auditory system
Periphery
CNS
4
What am I calling peripheral

• HRTFs, including ear canal and middle ear
  functions
• Cochlear analysis
• Reduction of sound into partial loudness as a
  function of time

5
Partial Loudness?

• First, two terms
• Intensity
• Sound Pressure Level
• MEASURED
• Loudness
• Sensation Level
• Perceived
• The inner ear reduces the sounds that reach your
  eardrum to partial loudness. That is the
  information, in a time/frequency analysis that
  results in loudness vs. frequency vs. time, that
  goes down the auditory nerve.

6
And Part of the CNS?

• Everything else
• Reduction from partial loudness to auditory
  features
• Reduction of auditory features to auditory
  objects
• Storage in short-term and long-term memory

7
Anything more about the CNS?

• Its extremely flexible
• It can consciously change what it does (leaving
  aside for now the definition of consciousness)
• Its output is what finally matters to us
• It evolved to do an extremely, distinctly
  excellent job of associating information from all
  senses and knowledge into the final result.
• All the time
• Everywhere

8
What actually gets to the CNS?

• Whatever is detected by the auditory periphery
• We will leave out extremely intense LF and VHF
  signals, which can be detected by other means,
  these are extreme conditions and should not
  generally be experienced by a listener.
• How does the auditory periphery deal with the
  sound waves in the atmosphere?

9
What does the periphery do?

• First the periphery adds directional information
  via HRTF and ITD
• Then, the cochlea does a time/frequency analysis
• The time/frequency analysis is converted into
  loudness via compression in each band,
  introducing
• Differences between loudness and intensity
• The Haas (precedence) effect
• The partial loudness across frequency is encoded
  into a kind of biological PPM and transmitted
  across the auditory nerve. (No, its really not
  that simple, but it will do for now.)

10
A Key Point or Two

• The auditory periphery analyzes all signals in a
  time/frequency tiling called ERBs or Barks.
• Due to the mechanics of the cochlea, first
  arrivals have very strong, seemingly
  disproportionate influence on what you actually
  hear
• But this is actually useful in the real world
• Signals inside an ERB mutually compress
• Signals outside an ERB do not mutually compress.

11
Then what?

• The short-term loudness, called partial loudness,
  is, roughly speaking, integrated across a short
  amount of time (200 milliseconds or less)
• Level Roving Experiments show that when delays of
  over 200 milliseconds exist between two sources,
  the ability to discern fine differences in
  loudness or timbre is reduced.

12
What happens after this Loudness Memory?

• Deep inside the CNS, in a fashion that I would
  not even care to speculate on, it seems clear
  that these partial loudness sensations are
  analyzed into both monaural and binaural auditory
  features
• There is a great deal of data loss at this
  juncture
• This memory can last seconds or so
• The analysis from partial loudness to features
  can be very strongly guided by learning,
  experience, and cognition

13
And then?

• These features are turned into what I refer to as
  auditory objects
• These can be committed to long-term memory
• There is another substantial reduction in data
  rate
• This process can be entirely steered by
  attention, cognition, other stimulii, etc.

14
A schematic of sorts
Mbits/second
Kb/sec
Mbit/sec
bit/sec
Loudness integration
Feature Analysis
Auditory Object Analysis
Cognitive and other Feedback
15
Something to notice

• Look at the amount of information lost at each
  step.
• You can guide the loss of information.
• Consider the implications.
• You control what gets lost and what stays.
• This is true both consciously and unconsciously.
• You WILL integrate the input from all of your
  senses.
• Its how people work.
• Even when they try not to.

16
What does this imply?

• If you listen to something differently (for
  different features or objects)
• You will REMEMBER different things
• This is not an illusion
• If you have reason to assume things may be
  different
• You will most likely listen differently
• Therefore, you will remember different things

17
So what?

• What this all means, in effect, is that any test
  of auditory stimulii that wants to distinguish
  only in terms of the auditory stimulii must
• Have a falsifiable nature (i.e. be able to
  distinguish between perception and an actual
  effect)
• Must isolate the subject from changes in other
  stimulii than audio
• Must be time-proximate
• Must have Controls
• Must have trained, comfortable listeners

18
Controls? What? NOW
what are you on about?

• A control is a test condition that tests the
  test. There can be many kinds of controls
• A positive control
• This is a condition that a subject should be able
  to detect.
• If they dont, you have a problem.
• A negative control
• A vs. A is the classical negative control
• If your subject hears a difference, you have a
  problem
• Anchoring elements
• Conditions that relate scoring of this test to
  results in other tests
• These can vary depending on need, and may not be
  obligatory

19
Do I have to have controls?
YES Well, unless you dont want to know how good
your test is, of course. ?
20
How does all this apply to High Fidelity and such?

• When somebody guides your listening, you will
  change what you listen to.
• If you know something is changed in the system,
  you will expect changes in the output, and
  probably refocus.
• This is normal human behavior.
• It is something everyone does
• It goes along with cognition, and is very nearly
  a property of cognition.

21
A word on cables.

• When you are doing something like auditioning
  cables, if you do that kind of thing, remember
• First, remove and replace the existing cables.
• RCA connectors need to be moved around once in a
  while to wipe the corrosion
• Have a third party swap cables without your
  knowledge
• See if you can tell which is which
• If you can, then its up to your preference
• But that means either one of the cables is
  broken, or
• One of the cables does deliberate frequency
  shaping or other modification.
• Remember to remove and replace the connecting
  cables.
• DO THAT FIRST.

22
Is that all?

• Not even close, but were talking about basics
  today.
• Before the break, I will attempt a demo that
  shows the effects of expectation.
• After the break, several of us will discuss some
  of the more interesting audio products
  available.