Paul Smolensky - PowerPoint PPT Presentation

1 / 61
About This Presentation
Title:

Paul Smolensky

Description:

Paul Smolensky. Cognitive Science Department. Johns Hopkins University. G raldine Legendre ... Cognitive psychology ICS. a2 must be active (strength: 0.5) ... – PowerPoint PPT presentation

Number of Views:95
Avg rating:3.0/5.0
Slides: 62
Provided by: paulsmo
Category:

less

Transcript and Presenter's Notes

Title: Paul Smolensky


1
The Harmonic Mind
  • Paul Smolensky
  • Cognitive Science Department
  • Johns Hopkins University

with
Géraldine Legendre Donald Mathis Melanie
Soderstrom
Alan Prince Suzanne Stevenson Peter Jusczyk
2
Advertisement
The Harmonic Mind From neural computation to
optimality-theoretic grammar   Paul
Smolensky   Géraldine Legendre
  • Blackwell 2003 (??)
  • Develop the Integrated Connectionist/Symbolic
    (ICS) Cognitive Architecture
  • Case study in formalist multidisciplinary
    cognitive science (point out imports/exports of
    ICS)

3
Cognitive Science 101
  • Computation is cognition
  • But what type?
  • Fundamental question of research on the human
    cognitive architecture

4
Table of Contents
5
Table of Contents
  • Implications of architecture for nativism
  • Learnability
  • Initial state
  • Experimental test Infants
  • (Genomic encoding of UG)

6
Processing Algorithm Activation
  • Computational neuroscience ? ICS
  • Key sources
  • Hopfield 1982, 1984
  • Cohen and Grossberg 1983
  • Hinton and Sejnowski 1983, 1986
  • Smolensky 1983, 1986
  • Geman and Geman 1984
  • Golden 1986, 1988

Processing spreading activation is
optimization Harmony maximization
7
Function Optimization
  • Cognitive psychology ? ICS
  • Key sources
  • Hinton Anderson 1981
  • Rumelhart, McClelland, the PDP Group 1986

CONFLICT!!
8
Representation
  • Symbolic theory ? ICS
  • Complex symbol structures
  • Generative linguistics ? ICS
  • Particular linguistic representations
  • PDP connectionism ? ICS
  • Distributed activation patterns
  • ICS
  • realization of (higher-level) complex symbolic
    structures in distributed patterns of activation
    over (lower-level) units (tensor product
    representations etc.)

9
Representation
10
Knowledge Constraints
NOCODA A syllable has no coda
H(as k æ t) sNOCODA lt 0
11
Constraint Interaction I
  • ICS ? Grammatical theory
  • Harmonic Grammar
  • Legendre, Miyata, Smolensky 1990 et seq.

12
Constraint Interaction I
The grammar generates the representation that
maximizes H this best-satisfies the constraints,
given their differential strengths
Any formal language can be so generated.
13
Harmonic Grammar Parser
  • Simple, comprehensible network
  • Simple grammar G
  • X ? A B Y ? B A
  • Language

Completion
14
Harmonic Grammar Parser
  • Representations

15
Harmonic Grammar Parser
?
16
Harmonic Grammar Parser
17
Harmonic Grammar Parser
  • Representations

18
Harmonic Grammar Parser
H(Y, B) gt 0H(Y, A) gt 0
  • Weight matrix for Y ? B A

19
Harmonic Grammar Parser
  • Weight matrix for entire grammar G

20
Bottom-up Processing
21
Top-down Processing
22
Scaling up
  • Not yet
  • Still conceptual obstacles to surmount

23
Constraint Interaction II OT
  • ICS ? Grammatical theory
  • Optimality Theory
  • Prince Smolensky 1993

24
Constraint Interaction II OT
  • Differential strength encoded in strict
    domination hierarchies
  • Every constraint has complete priority over all
    lower-ranked constraints (combined)
  • Approximate numerical encoding employs special
    (exponentially growing) weights

25
Constraint Interaction II OT
  • Grammars cant count
  • Stress is on the initial heavy syllable iff the
    number of light syllables n obeys

No way, man
26
Constraint Interaction II OT
  • Constraints are universal
  • Human grammars differ only in how these
    constraints are ranked
  • factorial typology
  • First true contender for a formal theory of
    cross-linguistic typology

27
The Faithfulness / Markedness Dialectic
  • cat /kat/ ? kæt NOCODA why?
  • FAITHFULNESS requires identity
  • MARKEDNESS often opposes it
  • Markedness-Faithfulness dialectic ? diversity
  • English NOCODA FAITH
  • Polynesian FAITH NOCODA (French)
  • Another markedness constraint M
  • Nasal Place Agreement Assimilation (NPA)
  • mb ? nb, ?b nd ? md, ?d ?g ? ?b,
    ?d
  • labial coronal
    velar

28
Nativism I Learnability
  • Learning algorithm
  • Provably correct and efficient (under strong
    assumptions)
  • Sources
  • Tesar 1995 et seq.
  • Tesar Smolensky 1993, , 2000
  • If you hear A when you expected to hear E,
    minimally demote each constraint violated by A
    below a constraint violated by E

29
Constraint Demotion Learning
  • If you hear A when you expected to hear E,
    minimally demote each constraint violated by A
    below a constraint violated by E

Correctly handles difficult case multiple
violations in E
30
Nativism I Learnability
  • M F is learnable with /inpossible/?impossible
  • not in- except when followed by
  • exception that proves the rule M NPA
  • M F is not learnable from data if there are no
    exceptions (alternations) of this sort, e.g.,
    if no affixes and all underlying morphemes have
    mp vM and vF, no M vs. F conflict, no evidence
    for their ranking
  • Thus must have M F in the initial state, H0

31
Nativism II Experimental Test
  • Linking hypothesis
  • More harmonic phonological stimuli ? Longer
    listening time
  • More harmonic
  • ?M ? M, when equal on F
  • ?F ? F, when equal on M
  • When must choose one or the other, more harmonic
    to satisfy M M F
  • M Nasal Place Assimilation (NPA)
  • Collaborators
  • Peter Jusczyk
  • Theresa Allocco
  • (Elliott Moreton, Karen Arnold)

32
Experimental Paradigm
  • Headturn Preference Procedure (Kemler Nelson et
    al. 95 Jusczyk 97)
  • X/Y/XY paradigm (P. Jusczyk)
  • un...b?...umb?
  • un...b?...umb?

FNP
R
p .006
?FAITH
  • Highly general paradigm Main result

33
4.5 Months (NPA)
34
4.5 Months (NPA)
35
4.5 Months (NPA)
36
4.5 Months (NPA)
37
Nativism III UGenome
  • Can we combine
  • Connectionist realization of harmonic grammar
  • OTs characterization of UG
  • to examine the biological plausibility of UG as
    innate knowledge?
  • Collaborators
  • Melanie Soderstrom
  • Donald Mathis
  • Oren Schwartz

38
Nativism III UGenome
  • The game take a first shot at a concrete example
    of a genetic encoding of UG in a Language
    Acquisition Device
  • Introduce an abstract genome notion parallel to
    (and encoding) abstract neural network
  • Is connectionist empiricism clearly more
    biologically plausible than symbolic nativism?
  • No!

39
Summary
  • Described an attempt to integrate
  • Connectionist theory of mental processes
  • (computational neuroscience, cognitive
    psychology)
  • Symbolic theory of
  • Mental functions (philosophy, linguistics)
  • Representations
  • General structure (philosophy, AI)
  • Specific structure (linguistics)
  • Informs theory of UG
  • Form, content
  • Genetic encoding

40
The Problem
  • No concrete examples of such a LAD exist
  • Even highly simplified cases pose a hard problem
  • How can genes which regulate production of
    proteins encode symbolic principles of
    grammar?
  • Test preparation Syllable Theory

41
Approach Multiple Levels of Encoding
Biological Genome
42
Basic syllabification Function
  • /underlying form/ ? surface form
  • Plural form of dish
  • /d?s/ ? .d?. ? z.
  • /CVCC/ ? .CV.C V C.

43
Basic syllabification Function
  • /underlying form/ ? surface form
  • Plural form of dish
  • /d?s/ ? .d?. ? z.
  • /CVCC/ ? .CV.C V C.
  • Basic CV Syllable Structure Theory
  • Prince Smolensky 1993 Chapter 6
  • Basic No more than one segment per syllable
    position .(C)V(C).

44
Syllabification Constraints (Con)
  • PARSE Every element in the input corresponds to
    an element in the output
  • FILLV/C Every output V/C segment corresponds to
    an input V/C segment
  • ONSET No V without a preceding C
  • NOCODA No C without a following V

45
SAnet architecture
  • /C1 C2/ ? C1 V C2

/C1 C2 /
C1 V C2
46
Connection substructure
47
PARSE
  • All connection coefficients are 2

48
ONSET
  • All connection coefficients are ? 1

49
Crucial Open Question(Truth in Advertising)
  • Relation between strict domination and neural
    networks?
  • Apparently not a problem in the case of the CV
    Theory

50
To be encoded
  • How many different kinds of units are there?
  • What information is necessary (from the source
    units point of view) to identify the location of
    a target unit, and the strength of the connection
    with it?
  • How are constraints initially specified?
  • How are they maintained through the learning
    process?

51
Unit types
  • Input units C V
  • Output units C V x
  • Correspondence units C V
  • 7 distinct unit types
  • Each represented in a distinct sub-region of the
    abstract genome
  • Help ourselves to implicit machinery to spell
    out these sub-regions as distinct cell types,
    located in grid as illustrated

52
Connectivity geometry
  • Assume 3-d grid geometry

53
Constraint PARSE
  • Input units grow south and connect
  • Output units grow east and connect
  • Correspondence units grow north west and
    connect with input output units.

54
Constraint ONSET
  • Short connections grow north-south between
    adjacent V output units,
  • and between the first V node and the first x
    node.

55
Direction of projection growth
  • Topographic organizations widely attested
    throughout neural structures
  • Activity-dependent growth a possible alternative
  • Orientation information (axes)
  • Chemical gradients during development
  • Cell age a possible alternative

56
Projection parameters
  • Direction
  • Extent
  • Local
  • Non-local
  • Target unit type
  • Strength of connections encoded separately

57
Connectivity Genome
  • Contributions from ONSET and PARSE
  • Key

58
ONSET
x0 segment S S VO
N S x0
  • VO segment NS S VO

59
Learning Behavior
  • Simplified system can be solved analytically
  • Learning algorithm turns out to
  • Dsi(?) e violations of constrainti P?

60
Possible Conclusions
  • Empiricist connectionism is not more
    biologically plausible than nativist
    connectionism
  • (except possibly local vs. distributed
    representations)
  • It might be possible to do evolutionary
    simulation (not mathematical analysis) in a space
    including bona fide LADs
  • I must have too much time on my hands

61
Thanks for your attention
Write a Comment
User Comments (0)
About PowerShow.com