Bootstrapping

1
Bootstrapping

• Tom Griffiths

2
Bootstrapping

• How to learn words without knowing words
• Various proposals
• semantic bootstrapping (Pinker, 1984)
• syntactic bootstrapping (Gleitman,
  1990)
• Characterized by accelerated learning
• (e.g. Regier, 2004)
• Question
• when is bootstrapping possible?

3
Word learning
blicket
blicket
blicket
4
Bayes theorem
h hypothesis d data
5
Bayesian word learning
(Tenenbaum, 1999 Tenenbaum Xu, 2002)

• Data
• scene-word pairs
• Hypotheses
• functions labeling scenes
• Likelihood
• weak sampling
• strong sampling

x
h
w
6
blicket
p(dh) 0
7
blicket
p(dh) 1/3
8
blicket
blicket
blicket
p(dh) (1/3)3
9
blicket
p(dh) 1/12
10
blicket
blicket
blicket
p(dh) (1/12)3
11
Bootstrapping

• Bayesian word learning is a form of semantic
  bootstrapping (Niyogi, 2002)
• What about accelerated learning?
• non-linear increase in probability of correct
  answer for a random scene and word
• When can it occur?
• not when hypotheses independent and all equally
  likely, when using weak sampling
• speculation hypotheses are dependent

12
Forms of dependency

• Hierarchical priors
• unknowns across learning events
• Compositional priors
• unknowns within learning events

13
Hierarchical priors
x
x
x
x
h
h
h
h
w
w
w
w
blicket
toma
dax
wug
14
dax
blicket
toma
wug?
15
Hierarchical priors

• What is contained in a hierarchical prior?
• Any learned information that constrains
  scene-word mappings
• typical referents (whole object)
• dimensions of stimuli (shape/substance)
• pragmatic dependencies (mutual exclusivity)
• sound and meaning (morphology)

16
Compositional hypotheses
blicket toma
17
Compositional hypotheses

• Good news
• express syntactic bootstrapping
• model referential uncertainty
• Bad news
• requires complete linguistic theory

18
Bootstrapping

• When do we see accelerated learning?
• speculation dependent hypotheses
• Sources of dependency in language
• hierarchical priors
• compositional hypotheses
• Bootstrapping goes beyond language
• learning causal theories aids learn causal
  relationships, learning concepts

19
(No Transcript)