Behavioral and Psychophysiological Indices of Infant Attention to Speech Across the First Postnatal

1
Behavioral and Psychophysiological Indices of
Infant Attention to Speech Across the First
Postnatal Year
Robin Panneton Department of Psychology Virginia
Tech
2
Collaborators

• Drs. Denis Burnham, Christine Kitamura, and Karen
  Mattock (MARCS Auditory Laboratories)
• Dr. Dick Aslin (University of Rochester)
• Dr. John Richards (University of South Carolina)
• Megan McIlreavy
• Naureen Bhullar
• Wendy Ostroff

Graduate students at VT
3

General Methodology
One way to study infants preference for and
discrimination of speech is to make hearing
contingent on some reliable aspect of their
behavior
We make hearing something contingent on their
visual behavior, and use looking time as a
behavioral index of attention one visual event
is presented serially but different auditory
events are paired with it, depending on the
trial infants control their exposure to this A/V
display works well with infants across a wide
range of ages
4
General Methodology
ORDER 1
ORDER 2
5
We use this method to manipulate both visual and
auditory events across a wide range of infants
ages
6
For example
7
More recently, we also measure changes in
infants heart rate (HR) patterns as a second
measure of attention
8
So, we often study Infant Directed Speech (IDS)
9
Experiment 1 AV influences on non-native phoneme
discrimination

• To examine whether the augmentation of visual and
  auditory information impacts infants
  discrimination of non-native speech contrasts

• We manipulated
• type of visual display (dynamic face or static
  form
• type of speaking style (IDS or ADS)
• gender of speaker (female or male)
• We tested both 11-month-olds and undergraduates

10
Various Discrimination Conditions
ID
AD
ID/AD
Hindi retroflex d Hindi retroflex t
Hindi retroflex d Hindi retroflex t
English bilabial b English alveolar d
11
Experiment 1a Female ID/AD and Face/Target (n
40)Significant Trial main effect and Trial x
Condition interaction
Native English Contrast
Mean looking time on posttest gt pretest except
for AD Face




p lt .05
12
Experiment 1a Female ID/AD and Face/Target (n
40)Significant Trial main effect and Trial x
Condition interaction
Non-native Hindi Contrast
Mean looking time on posttest gt pretest for ID
Face and ID Form
15



Pretest
10
Posttest

Average Looking Time

5
0
ID/Face
ID/Form
AD/Face
AD/Form
plt.05 p lt.08
Condition
13
College Students Discrimination of Non-native
Female Hindi Contrasts

• 17 undergraduate students (all monolingual
  American English speakers) were presented with
  Hindi Female contrasts
• Saw and heard 2 tokens (x 4 repetitions), and
  were asked to respond if they were the same or
  different

14
Experiment 1a Male AD Voice Target (n
20)Significant Trial x Condition interaction
Mean looking time on posttest pretest


Average Looking Time
Condition
plt.05 p lt.08
15
Male ID Face ID Voice
16
Experiment 1b Male ID Voice Face (n
20)Significant main effect of trial no sig
interactions
Non-native Hindi Contrast
Mean looking time on posttest pretest

p lt .05
17
College Students Discrimination of Non-native
Male Hindi Contrasts

• 22 undergraduate students (all monolingual
  American English speakers) were presented with
  Hindi Male contrasts
• Saw and heard 2 tokens (x 4 repetitions), and
  were asked to respond if they were the same or
  different

92
31
8
69
18
Experiment 1c Female ID Voice Face (n 7
preliminary data)
Non-native Hindi Contrast
dental
retroflex
19
Waveforms of Hindi Contrasts
Female Retroflex /d/
Female Dental /d/
Male Dental /d/
Female Retroflex /t/
Female Retroflex /d/
Male Retroflex /d/
20
Interpretation

• Providing additional visual (dynamic speaking
  face) and auditory (infant-directed speech)
  information augmented the ability of 11-month-old
  infants to discriminate a non-native phoneme
  contrast
• But this perceptual augmentation needs to be
  qualified by other contextual factors (e.g.,
  gender of the speaker)
• And may also be qualified by the degree of
  difficulty in the non-native contrast per se
  (either in psychoacoustic properties and/or
  visual-speech properties)
• Why did the additional information augment
  discrimination?

21
Attention-general vs. Attention-specific Effects?

• Attention-General dynamic faces and voices
  increase infants focused attention to the task
  overall, facilitating discrimination
• Synchrony that is apparent in AV speech
  facilitates perception
• Infant preference for female over male
  faces/voices
• Infant preference for IDS over ADS
• Attention-Specific dynamic faces and voices
  provide speech-relevant information that
  facilitates discrimination directly
• Information about phonemes in facial movement
  (e.g., place of articulation)
• Hyperarticulated (female?) speech (IDS) is more
  clear

22
Experiment 3 Infants attention to unimodal and
bimodal ID displays
VISUAL
Geometric Form
AD Face
ID Face
AUDITORY
Instrumental Music
AD Voice
ID Voice
VISUAL AUDITORY
AD Face/Voice
Geometric Form/ Instrumental Music
ID Face/Voice
23
Generally, we found longer periods of sustained
lowered HR in infants as they experienced bimodal
events, but particularly to ID speech ID face
15
Mean Duration of First Deceleration (D1)
Mean Duration of Looking During D1
10
Mean Duration (secs)
5
0
Forms
AD Face
ID Face
Music
AD Voice
ID Voice
MusFor
AD FV
ID FV
24
Experiment 4 Developmental changes in infants
attention to static and dynamic visual events
Static form
Static face
Dynamic form
Dynamic face
25
Thank You!