The Nature of Systematicity in Natural Language

1
The Nature of Systematicity in Natural Language

2
Systematicity

• Introduced in Fodor Pylyshyn (1988)
• A feature of cognition, inferential coherence
• The ability to produce/understand some
  sentences is intrinsically connected to the
  ability to produce/understand certain others.
  (p. 37)
• Fodor Pylyshyn give no clear definition (see
  van Gelder) and only two examples

3
The operationalization of systematicity

• When an agent understands the sentence Peter
  likes Maria, she understands the sentence Maria
  likes Peter as well.
• When an agent understands the expressions brown
  triangle and black square, she understands the
  expressions brown square and black triangle as
  well.

4
Outline

• What is systematicity?
• and to what degree is our understanding
  systematic
• Does compositionality entail systematicity?
• No, because
• context-dependence of meaning
• intrusion of encyclopedic knowledge on language
  understanding
• What are appropriate models of natural language
  understanding if systematicity is taken into
  account ?

5
Is our understanding systematic ?

1. Do all who understand good writer and bad teacher
   also understand bad writer and good teacher?
2. Do all who understand within an hour and without
   a watch also understand within a watch and
   without an hour? (Szabó 2004).
3. Does a child who understands Mom drives me to
   kindergarten also understand I drive mom to
   kindergarten ?

6
Understanding not Systematic

• Language understanding is not completely
  systematic.
• This is because the world is not entirely
  systematic
• A. Clark (1996) Instead of treating
  systematicity as a property to be directly
  induced by a canny choice of basic architecture,
  it may be fruitful to try to treat it as
  intrinsic to the knowledge we want a system to
  acquire.
• A good model of language
• explains the systematic patterns of our language
  understanding
• gives an account of when and why we find patterns
  that are not really predictable.

7
Does compositionality entail systematicity?

• Compositionality The meaning of a compound
  expression is determined by the meanings of its
  constituents and the syntactic way these
  constituents are combined.
• Systematicity there are predictable patterns
  among the sentences we understand.

8
The present position

• Compositionality does not entail systematicity
• Systematicity is possible without compositionality

9
How to derive systematicity?

• Fodor Pylyshyn
• in the presented case the systematicity of
  linguistic competence derives from the fact that
  the syntactic operation of modification relates
  to the semantic operation of conjunction (or
  intersection)

10
Understanding NL Expressions

• According to the classical view, the phrase
  'understanding natural language expressions'
  means more than just fixing a denotation.
• We 'understand' a phrase as brown triangle if
• (i) we are able to grasp the corresponding
  conceptual representation (brown?triangle)
• (ii) the corresponding conceptual components
  have an known truth functional content
• (iii) With the help of the involved logical
  operators it is possible to determine the truth
  conditions of the whole phrase under discussion

11
Sytematicity Clauses A sample derivation

• understanding the expressions brown triangle and
  black square
• deriving brown?triangle and black?square
• extracting the lexicon entries brown ? brown,
  black ? black, triangle ? triangle, square ?
  square.
• calculating the corresponding conceptual
  representations for brown square and black
  triangle brown?square and black?triangle
• can get the truth-conditional impact of these
  forms
• we understand the expressions brown square and
  black triangle

12
But does this actually work?

• This proposal doesnt work for even the simplest
  of adjective-noun combinations
• Intersection doesnt give you the systematic
  inferences that you would expect or would desire

13
Why compositionality fails to explain
systematicity
14
Context dependence

• In his Grundlagen der Mathematik Frege (1884)
  noticed the context-dependence of words (and took
  this as a argument against compositionality in
  NL)
• One should ask for the meaning of a word only
  in the context of a sentence, and not in
  isolation
• There are compositional models of semantics that
  take context dependence into account
• We can use the phenomenon of context-dependence
  for construing an argument against deriving
  systematicity from compositionality.

15
What is a white triangle?

• A

The white triangle
A
The black triangle
From P. Bosch (2002) Explaining semantic
productivity. Paper presented at the Symposium on
Logic and Creativity Integrating Categorial
Rules and Experience, Osnabrück.
16
Context Dependence?

• Does the proposition expressed by black
  triangle depend on the contrast set?
• Answer 1 No.
• The contrast set is relevant only for choosing a
  good/appropriate/optimal referring expression
• PROBLEM ? Triangle 1 is black and white (in one
  and the same respect)
• Answer 2 Yes.
• Context-dependency of utterance meaning is
  widely accepted and also applies here
• (Kaplans characters Haas-Spohns hidden
  indexicality e.g. in connection with natural
  kind terms like water theories of
  underspecification, e.g. Carstons 2002
  explicatures in relevance theory)

17
What is a red apple?

• (a) a red apple red peel
• (b) a sweet apple sweet pulp
• (c) a reddish grapefruit reddish pulp
• (d) a white room/ a white house inside/outside

A red apple? No, its a green apple but its red
on the inside
18
More examples

• Quine (1960) was the first who noted the contrast
  between red apple (red on the outside) and pink
  grapefruit (pink on the inside).
• In a similar vein, Lahav (1993) argues that an
  adjective such as brown doesnt make a simple and
  fixed contribution to any composite expression in
  which it appears
• In order for a cow to be brown most of its
  bodys surface should be brown, though not its
  udders, eyes, or internal organs. A brown
  crystal, on the other hand, needs to be brown
  both inside and outside. A brown book is brown if
  its cover, but not necessarily its inner pages,
  are mostly brown, while a newspaper is brown only
  if all its pages are brown. For a potato to be
  brown it needs to be brown only outside, ...
  (Lahav 1993 76).

19
Three consequences

• Intersectivity, A(B) A(?)?B,
  doesnt hold for most absolute adjectives and
  Fodor Pylyshyn's compositional analysis breaks
  down
• Systematicity statements cannot be derived from
  compositionality if intersectivity fails
• Encyclopedic knowledge is required to determine
  the truth conditional content of an utterance
  (explicature in Relevance Theory)
• When an agent understands the expressions red
  apple (RED PEEL) and sweet grapefruit (SWEET
  PULP), then it's likely that she understands red
  grapefruit (RED PULP) and sweet apple (SWEET
  PULP) as well.

20
The Theoretical Part
21
Approach 1 The underspecification view

• Radical underspecification augmented with
  contextual enrichment
• small ? ?x small(x,N) small terrier ? ?x
  small(x,N) terrier(x)
• Analogously for red apple with place-holders for
  the relevant partsred ? ?x part(Y,x)
  red(Y)red apple ??x part(Y,x) red(Y)
  apple(x)
• How to determine the proper values for N and Y,
  respectively?
• with small(x,N) ? size(x) lt N

22
A mechanism of contextual enrichment

• The variables are specified in a way that
  maximizes the relevance of the corresponding
  question
• Small Terrier Is a (randomly selected) terrier
  smaller than N?
• Red Apple What color is part Y (of a randomly
  selected apple)?
• Probabilistic Theory of Relevance, see
• Robert van Rooy (2000) Comparing Questions and
  Answers A bit of Logic, a bit of Language, and
  some bits of Information

23
Entropy of a question

• The semantic value of a question Q is a partition
  q1, ..., qn of the domain O.
• ? inf(q) -log2 prob(q) information
  measure of surprise
• ?
  Entropy of a question Q
• The entropy of a question expresses our
  uncertainty about the answer. Good questions have
  high entropies

24
What is a black triangle?
What color is the inner part? E1 E0
What color is the outer part? E0 E1

25
What is a red apple?
A red apple? What color is an apple? Q1 What
color is its peel? Q2 What color is its
pulp? E(Q1) gtgt E(Q2) Color differences between
apples are expected for the peel and not for the
pulp.
Therefore, the presented apple is considered as a
green apple (inside red) and not as a red apple
(outside green).This can change if we update
our probability distribution.
26
Problems with the underspecification view

• Requires rather clumsy lexical entries
• How much of the peel of an apple has to be red in
  order to call it a red peel?
• This theory does not really clarify how the
  border line between the underspecified
  representation and the contextual enrichment is
  ever to be determined

27
Approach 2 Adnominal functors

• Take Montague (1970) as its starting-point and
  take adjectives as adnominal functors.
• red(X) means roughly the property
• (a) of having a red inner volume if X denotes
  fruits only the inside of which is edible
• (b) of having a red surface if X denotes fruits
  with edible outside
• (c) of having a functional part that is red if X
  denotes tools,

28
Montague and systematicity clauses

• We cannot derive the intended systematicity
  clauses if we realize compositionality via
  adnominal functors!
• f(a) p, g(b) q
• f(b) ?, g(a) ?
• Compositionality is often very simple to realize
  if (higher order) functions are introduced
• In many cases, this idea realizes generalizations
  to the worst case
• Unfortunately, we cannot derive interesting
  systematicity clauses from this style of
  compositionality!

29
Constraining the models

• Additional constraints are required that restrict
  the set of possible models
• I will assume that these constraints can be
  extracted from a Bayesian picture of the mental
  encyclopedia
• This is not so different from assuming a system
  of violable constraints (ranked defaults instead
  of inviolable meaning postulates)
• Systematicity mainly results from these
  additional constraints!

30
Approach 3 Connectionst model of adjectival
modification

• Overcoming the gap between compositionality and
  systematicity
• Modeling both the truth-functional aspects of
  adjectival modification and the typicality
  effects (Kamp Partee 1995)
• Connectionist variant of the selective
  modification model of Smith et al. (1988)
• It shares with this model the (localist)
  attribute-value representation for the prototypes
  (apple, grapefruit, ) and for the relevant
  instances.
• From Blutner, Hendriks, de Hoop, Schwartz (to
  appear) When compositionality fails to predict
  systematicity

31
The simplified model
32
Conclusions

1. Systematicity of natural language understanding
   is intimately related to knowledge about the
   world
2. Systematicity clauses have to reflect this
   reality
3. A compositional representational system doesnt
   give you systematicity of understanding because
   context-dependency of lexical meaning and
   world-knowledge both affect understanding

33
Conclusions cont.

• (4) If lexical principles like intersectivity
  fail (because of context dependency)
  systematicity clauses can no longer be derived
  from compositionality
• (5) A Bayesian and/or connectionist picture of
  the mental encyclopedia is the key for deriving
  systematicity clauses!
• (6) The principle of compositionality of meaning
  may be interesting ? not as part of cognitive
  architecture, but as a consequence of
  evolutionary learning (e.g. Kirby).

34
Thank you for your attention

35
... other problems with adjectival modification

• Typicality effects
• The problems of typicality don't relate to the
  truth-conditional impact of adjectival
  modification. However, a good model should
  account for both effects the truth-conditional
  peculiarities of adjectival modification and the
  typicality effects.

36
The red hair problem

• Quine (1960) the different colors typically
  denoted (strong preference) by red in red apple
  and red hair
• In Japanese, aka-zatoo 'brown sugar' (lit. 'red
  sugar') comes in the same range of colors as
  shira-miso, lit. 'white bean paste
• the actual color value deviates in a systematic
  way from the prototypical color value that can be
  assigned to the color adjective in isolation in
  dependency on the conceptual properties of the
  modified noun

37
The pet fish problem

• Goldfish is a poorish example of a fish, and a
  poorish example of a pet, but it's quite a good
  example of a pet fish (the conjunction effect of
  typicality cx(AB) gt cx(B))
• In case of "incompatible conjunctions" such as
  pet fish, striped apple or brown apple the
  conjunction effect is greater than in "compatible
  conjunctions (red apple).

38
Prototype effects

• Using Tversky's (1977) contrast rule (formulated
  for activation vectors)
• sim(s,t) ??i min(si,ti) ? ??i si ?ti
• sim(sred apple, t1) gt sim(sapple, t1)
• sim(sbrown apple,t2)?sim(sapple,t2) gt sim(sred
  apple,t1)?sim(sapple,t1)

39
Next steps

• Extend the model by distinguish different parts
  of the fruits (inside and outside)
• Since without further learning the model starts
  with a uniform weight matrix, we expect that the
  'neutral' force of modification affects all parts
  uniformly
• After learning, that the color of the outside of
  fruits is more discriminating than the color of
  its inside, we expect that the learning mechanism
  correctly reproduces the expected sort of
  modification
• Similarities to Zwarts' (2003) "strongest
  meaning" model which starts with an initial
  default hypothesis which is subsequentially
  modified if more encyclopedic knowledge comes in.