When the Shoe Fits: Cross-Situational Learning in Realistic Learning Environments

1
When the Shoe FitsCross-Situational Learning
inRealistic Learning Environments

• Tamara N. Medina1
• John Trueswell1
• Jesse Snedeker2
• Lila Gleitman1
• 1Institute for Research in Cognitive Science,
  University of Pennsylvania
• 2Department of Psychology, Harvard University

2
(No Transcript)
3
A baby hears a word like shoes, for example,
over and over again in daily life as the one
constant sound in a large variety of statements.
In one day you may say to him
That word shoes is the one sound which occurs
in all those sentences and it is always
associated with those things that go on his feet.
Eventually he will associate the spoken sound
with the objects and when he has made that
association, he will have learned what the word
shoes means.

• Where are your shoes?
• Oh, what dirty shoes!
• Lets take your shoes off.
• Ill put your shoes on.
• Look what nice new shoes.

• ?1998, Third Edition, Completely Revised

4
Cross-Situational Learning

• Find a set of possible meanings in each
  situation and intersect those sets across all
  situations in which a word occurs to determine
  the meaning for that word.
• Siskind, J.M. (1996, Cognition)
• Its not so easy!
• Augustine, Locke, Quine, Gleitman, Fodor,
  Siskind, etc.
• Frame / Level of Description
• Animal? Dog? Terrier? Fido? Friendly?
• Referential Uncertainty
• Which referent?

5
Frame Problem Solved?

• Xu Tenenbaum (2007) learn appropriate
  extensions of a word via Bayesian inference (note
  suspicious coincidences)

VASH
VASH
VASH
6
Reference Problem Solved?

• Yu Smith (2007) learn word-object
  associations in spite of referential uncertainty

DOON VASH MIPEN ZANT
7
VASH ??
8
VASH !
Goal Explore cross-situational word learning
using naturalistic settings both the cluttered
and potentially uninformative or misleading
environments and these somewhat more transparent
ones.
9
Overview

• Adaptation of the Human Simulation Paradigm
  (Gillette et al., 1999)
• Norming Study
• Current Study
• 2 measures to evaluate word learning

10
Adaptation of Human Simulation Paradigm (Gillette
et al., 1999)

• Selected stimuli based on results of earlier
  norming study
• Gertner, Y., Fisher, C., Gleitman, G., Joshi, A.,
  Snedeker, J. (In progress). Machine
  implementation of a verb learning algorithm.
• Large video corpus of parent-child interactions
  in natural settings (home, outdoors, etc.)
• Snedeker, J. (2001). Interactions between
  infants (12-15 months) and their parents in four
  settings. Unpublished corpus.

11
Norming Study

• Identified 48 most frequently occurring content
  words.
• Randomly selected six instances of each word.
• Each instance was edited into a 40-second
  vignette.
• Sound turned off.
• Visual context only cue to word meaning, placing
  viewers in the situation of the early word
  learner.
• Utterance of target word indicated by a BEEP.

Gertner, Y., Fisher, C., Gleitman, G., Joshi, A.,
Snedeker, J. (In progress). Machine
implementation of a verb learning algorithm.
12
(silence)
(silence)
ltBEEPgt
30 sec
(silence)
10 sec
Drawings courtesy of Emily Trueswell
13
No opportunity for cross-situational learning in
norming study
8 correct
83 correct
0 correct
High Informative (gt50 correct)
Low Informative (lt33 correct)

Subject Guesses (Target Word Shoe)
Subject Guesses (Target Word Horse)
Subject Guesses (Target Word Shoe)
90 of Vignettes Low Informative (typical)
7 of Vignettes High Informative (atypical)
Gertner, Y., Fisher, C., Gleitman, G., Joshi, A.,
Snedeker, J. (In progress). Machine
implementation of a verb learning algorithm.
14
Questions

• Does the observation of multiple naturalistic
  learning instances generate a gradually
  increasing learning curve?
• With regard to informativeness, given only the
  Low Informative vignettes, is cross-situational
  learning successful? Or is a High Informative
  instance necessary?
• If learners are building an interpretation across
  instances, does it matter when the High
  Informative instance occurs?

15
Current Study

• Similar to norming study,
• BUT allows for cross-situational learning

16
(silence)
Current Study Allows for Cross-Situational
Learning
(silence)
VASH
30 sec
(silence)
10 sec
17
Opportunity for cross-situational learning

VASH (Target Word Shoe) Subject makes
guess Subject rates Confidence (1 to 5)
MIPEN (Target Word Horse) Subject makes
guess Subject rates Confidence (1 to 5)
VASH (Target Word Shoe) Subject makes
guess Subject rates Confidence (1 to 5)
Final Conjectures and Confidence Ratings for each
word
18
Manipulated the distribution of informative events

• For each of 8 Target nouns, there were
• 1 High Informative vignette (gt50 of
  participants correct in norming study)
• 4 Low Informative vignettes (lt33 of
  participants correct in norming study)
• 4 Filler nouns
• 5 Low Informative vignettes
• Participants assigned to one of 4 orders
• High Informative First H-L-L-L-L
• High Informative Middle L-L-H-L-L
• High Informative Last L-L-L-L-H
• High Informative Absent L-L-L-L-L
• (fifth vignette is a repeat of the first)

19
Accuracy Across Vignettes
20
Accuracy Across Vignettes
21
Accuracy Across Vignettes
22
Accuracy Across Vignettes
23
Accuracy Across Vignettes
24
Interim Summary What have we learned about
learning?

1. Gradual learning from partially informative
   instances is small to nonexistent.
2. Successful learning depends on the presence of a
   High Informative instance. (Epiphany!)
3. Low Informative instances have a corrupting
   influence on later-occurring High Informative
   instances. (Cross-situational learning of the
   bad sort.)

25
Epiphany!

• Successful learning depends on the presence of a
  High Informative instance.
• Explicit and immediate insight?
• After using evidence from later instances?
• High Informative instance provides key for
  interpreting later instances.

26
Confidence on Correct Guesses across Vignettes
27
Confidence on Correct Guesses across Vignettes
28
Confidence on Correct Guesses across Vignettes
29
Confidence on Correct Guesses across Vignettes
30
Implications

• Shape of the word learning curve may be very
  different than what cross-situational learning
  models (e.g., Yu Smith, 2007) have suggested
• Rapid
• Incremental

31
Implications

• Successful word learning from cross-situational
  observation requires the occurrence of a highly
  informative instance.
• But must it occur first?
• Greater delay between instances of a novel word
  Every day is a new day.
• Multiple High Informative learning instances.
• Previous studies which show striking rapid word
  learning are such cases.
• Less weight on interpretations of Low Informative
  instances.

Logically, no!
32
Implications

• A High Informative instance is the first step in
  successful cross-situational word learning.
• Prior Low Informative instances might not be
  remembered over time.
• Later Low Informative instances become useful
  (confirmatory evidence?)
• Supported by rising confidence levels after a
  High Informative vignette.

33
Cross-situational learning does work,but only
when the shoe fits.
34
Accuracy Confidence
35
Perseverance of First Guess