NTL Converging Constraints - PowerPoint PPT Presentation

1 / 33
About This Presentation
Title:

NTL Converging Constraints

Description:

Structured Connectionist Models can capture both of these processes nicely. ... Farsi. With identical settings, learned senses not in English. errors, NTL. language ... – PowerPoint PPT presentation

Number of Views:50
Avg rating:3.0/5.0
Slides: 34
Provided by: vittorio6
Category:

less

Transcript and Presenter's Notes

Title: NTL Converging Constraints


1
NTL Converging Constraints
  • Basic concepts and words derive their meaning
    from embodied experience.
  • Abstract and theoretical concepts derive their
    meaning from metaphorical maps to more basic
    embodied concepts.
  • Structured Connectionist Models can capture both
    of these processes nicely.
  • Grammar extends this by Constructions pairings
    of form with embodied meaning.

2
Simulation-based language understanding
Utterance
Harry walked to the cafe.
Constructions
Analysis Process
General Knowledge
Simulation Specification
Schema Trajector Goal walk Harry cafe
Belief State
Simulation
3
(No Transcript)
4
Background Primate Motor Control
  • Relevant requirements (Stromberg, Latash, Kandel,
    Arbib, Jeannerod, Rizzolatti)
  • Should model coordinated, distributed,
    parameterized control programs required for motor
    action and perception.
  • Should be an active structure.
  • Should be able to model concurrent actions and
    interrupts.
  • Model
  • The NTL project has developed a computational
    model based on that satisfies these requirements
    (x- schemas).
  • Details, papers, etc. can be obtained on the web
    at http//www.icsi.berkeley.edu/NTL

5
Active representations
  • Representation based on stochastic Petri nets
    captures dynamic, parameterized nature of actions
  • Many inferences about actions derive from what we
    know about executing them
  • Generative model action, planning, recognition,
    language.

Walking bound to a specific walker with a
direction or goal consumes resources (e.g.,
energy) may have termination condition(e.g.,
walker at goal) ongoing, iterative action
6
Somatotopy of Action Observation
Foot Action
Hand Action
Mouth Action
Buccino et al. Eur J Neurosci 2001
7
Active Motion Model
Evolving Responses of Competing Models over Time.
Nigel Goddard 1989
8
Language Development in Children
  • 0-3 mo prefers sounds in native language
  • 3-6 mo imitation of vowel sounds only
  • 6-8 mo babbling in consonant-vowel segments
  • 8-10 mo word comprehension, starts to lose
    sensitivity to consonants outside native language
  • 12-13 mo word production (naming)
  • 16-20 mo word combinations, relational words
    (verbs, adj.)
  • 24-36 mo grammaticization, inflectional
    morphology
  • 3 years adulthood vocab. growth,
    sentence-level grammar for discourse purposes

9
food toys misc. people
sound emotion action prep.
demon. social
Words learned by most 2-year olds in a play
school (Bloom 1993)
10
Learning Spatial Relation Words Terry Regier
  • A model of children learning spatial relations.
  • Assumes child hears one word label of scene.
  • Program learns well enough to label novel scenes
    correctly.
  • Extended to simple motion scenarios, like INTO.
  • System works across languages.
  • Mechanisms are neurally plausible.

11
Learning System
dynamic relations (e.g. into)
structured connectionistnetwork (based on
visual system)
  • Well look at the details next lecture

12
Limitations
  • Scale
  • Uniqueness/Plausibility
  • Grammar
  • Abstract Concepts
  • Inference
  • Representation
  • Biological Realism

13
(No Transcript)
14
Constrained Best Fit in Nature
inanimate animate
15
Learning Verb MeaningsDavid Bailey
  • A model of children learning their first verbs.
  • Assumes parent labels childs actions.
  • Child knows parameters of action, associates with
    word
  • Program learns well enough to
  • 1) Label novel actions correctly
  • 2) Obey commands using new words (simulation)
  • System works across languages
  • Mechanisms are neurally plausible.

16
(No Transcript)
17
(No Transcript)
18
Motor Control (X-schema) for SLIDE
19
(No Transcript)
20
Parameters for the SLIDE X-schema
21
Feature Structures for PUSH
22
(No Transcript)
23
System Overview
24
Learning Two Senses of PUSH
Model merging based on Bayesian MDL
25
Training ResultsDavid Bailey
  • English
  • 165 Training Examples, 18 verbs
  • Learns optimal number of word senses (21)
  • 32 Test examples 78 recognition, 81 action
  • All mistakes were close lift yank, etc.
  • Learned some particle CXN,e.g., pull up
  • Farsi
  • With identical settings, learned senses not in
    English

26
(No Transcript)
27
(No Transcript)
28
Constrained Best Fit in Nature
inanimate animate
29
(No Transcript)
30
Compositionality
  • Traditional
  • Context-free composition of logical forms
  • Contemporary
  • Constructional composition of conceptual
  • frames
  • Formal Cognitive Linguistics

31
Embodied Construction Grammar(Bergen, Chang
Paskin 2000)
  • Assumptions from Construction Grammar
  • Constructions are form-meaning pairs(Lakoff
    1987, Goldberg 1995)
  • Constructions vary in degree of specificity and
    level of description (morphological, lexical,
    phrasal, clausal)
  • Constructions evoke and bind semantic schemas
  • Additional influences
  • Cognitive Grammar (Langacker 1987)
  • Frame Semantics (Fillmore 1982)
  • Structured Connectionism (Feldman 1987)

32
Traditional Levels of Analysis
Pragmatics
Semantics
Syntax
Morphology
Phonetics
33
Harry walked into the cafe.
Pragmatics
Semantics
Utterance
Syntax
Morphology
Phonetics
34
Language understanding analysis simulation
construction WALKED in context c constituents
form f of type wakt meaning walking construed
as Walk-Action semantic constraints walking.ti
me before c.speech-time walking.aspect
encapsulated designates walking
Harry walked into the cafe.
Utterance
Analysis Process
Constructions
General Knowledge
Simulation Specification
Belief State
Simulation
35
Simulation specification
  • A simulation specification consists of
  • semantic schemas evoked by constructions
  • bindings between schemas (labeled by the
    constructions that enforce them)

36
Conclusion
  • Language acquisition and use is a hallmark of
    being human
  • Language seems to rely on fine-grained aspects of
    embodied (sensory-motor and social cognition)
    primitives and brain-like computation (massively
    parallel, distributed, spreading activation,
    temporal binding).
  • Understanding requires imaginative simulation!
  • We have built a pilot system that demonstrates
    the use of motor control representations in
    grounding the language of abstract actions and
    policies.
  • Sensory-Motor imagination and simulation is
    crucial in interpretation!
  • Ongoing Work.
  • Formalize and use a compositional set of embodied
    conceptual primitives and grammatical
    constructions.
  • Perform both behavioral and fMRI imaging
    experiments to test the predictions of the
    simulation hypothesis.
  • Further refine and ground the model in details of
    neural anatomy and functional architecture
    (basal-thalamic-cortical loops).
Write a Comment
User Comments (0)
About PowerShow.com