When Compositionality Fails to Predict Systematicity

1
When Compositionality Fails to Predict
Systematicity

• Reinhard Blutner, Petra Hendriks,
• Helen de Hoop, Oren Schwartz

2
The immediate appeal of compositionality Frege
(1923)

• It is astonishing what language can do. With a
  few syllables it can express an incalculable
  number of thoughts, so that even a thought
  grasped by a terrestrial being for the first time
  can be put into a form of words which will be
  understood by

• someone to whom the thought is entirely new. This
  would be impossible, were we not able to
  distinguish parts in the thoughts corresponding
  to the parts of a sentence, so that the structure
  of the sentence serves as the image of the
  structure of the thoughts.

3
Compositionality

• The meaning of a complex expression is
  determined by its structure and the meanings of
  its constitutents.
• equivalently
• There is a homomorphism between the syntactic
  term algebra to the algebra of meanings.

4
Compositionality

• Compositionality is necessary to explain the
  productivity of language.
• Compositionality entails productivity.

5
Systematicity

• The term and empirical hypothesis was introduced
  by Fodor Pylyshyn (1988), who didnt attempt a
  precise definition of the concept or a complete
  description of the empirical phenomenon.
• Van Gelder Niklasson (1994) summarize FPs
  notion of systematicity as the ability to do
  some things of a given cognitive type (including
  at least thinking a thought and making an
  inference) is intrinsically connected with their
  ability to do other, structurally related things
  of that type.

6
Systematicity

• When an agent understands the expressions brown
  triangle and black square, she understands the
  expressions brown square and black triangle as
  well.

Does compositionality entail systematicity?
7
A Classical Example

• Agent understands brown triangle and black
  square.
• She constructs the conceptual representations
  BROWN?TRIANGLE and BLACK?SQUARE (via
  compositionality).
• She knows the truth conditional impact of the
  corresponding constitutents, and she extracts the
  lexicon entries brown -gt BROWN, square-gtSQUARE,
  etc.
• Using these, she constructs BROWN?SQUARE and
  BLACK?TRIANGLE (via compositionality)
• Thus compositionality may derive systematicity

8
Systematicity

• If an agent understands
• within an hour and without a watch,
• does she also understand
• within a watch and without an hour?
• (Szabo 2004)

9
Our View

• Language is (mostly) systematic and
  compositional, but not always.
• Traditional semantic analyses have difficulty
  with cases where the systematicity of meaning is
  weak or missing.
• Connectionist models are well suited to handling
  these types of cases, and can account for the
  semantic contribution of a constitutent that is
  both systematic (in general) and idiosyncratic
  (in certain cases).

10
Why compositionality fails to explain
systsematicity

• In his Grundlagen der Mathematik Frege (1884)
  noticed the context-dependence of words (and took
  this as an argument against compositionality in
  NL)
• One should ask for the meaning of a word only
  in the context of a sentence, and not in
  isolation.

11
Empirical Phenomena

• Quine (1960) was the first who noted the contrast
  between red apple (red on the outside) and pink
  grapefruit (pink on the inside).
• Lahav (1993) argues that an adjective such as
  brown doesnt make a simple and fixed
  contribution to any composite expression in which
  it appears
• brown cow, brown crystal, brown book, brown
  newspaper

12
Empirical Phenomena

• Different colors typically denoted by red in red
  apple and red hair.
• the actual color value deviates in a systematic
  way from the prototypical color value that can be
  assigned to the color adjective in isolation in
  dependency on the conceptual properties of the
  modified noun
• Similarly, the same color is often described with
  different adjectives
• In Japanese, brown sugar, aka-zatoo (lit. red
  sugar) comes in the same range of colors as
  shira-miso, (lit. white bean paste)

13
Three consequences

• Intersectivity, A(B) A(x)?B,
  doesnt hold, even for most absolute adjectives
• Systematicity statements cannot be derived from
  compositionality in these cases
• Encyclopedic knowledge is required to determine
  the truth conditional content of an utterance
  (explicature in Relevance Theory).

14
A Related Phenomenon

• Typicality
• A specific instance of a red apple is more
  typical of a red apple than of an apple
• Incompatible Conjunctions
• The typicality effects are greater for
  incompatible conjunctions (striped apple) than
  for compatible conjunctions (green apple)

15
A Concrete Question

• How should we account for the semantic
  contribution of an (absolute) adjective to an
  (Adjective Noun)N complex expression in the face
  of these phenomena?

16
Radical underspecificationwith contextual
enrichment

• small ? ?x small(x,N)
• small terrier ? ?x small(x,N) terrier(x)
• Analogously for red apple with place-holders for
  the relevant parts
• red ? ?x part(Y,x) red(Y)
• red apple ??x part(Y,x) red(Y) apple(x)
• How to determine the proper values for N and Y,
  respectively?
• Contextual enrichment, as in Probabilistic Theory
  of Relevance (van Rooy 2000)
• with small(x,N) ? size(x) lt N

17
Problems with underspecification

• Does not really clarify how to determine the
  border line between the underspecified
  representation and the contextual enrichment
• It is difficult to see how the available
  mechanisms account for the prototype effects
  found in adjectival modification

18
A connectionist approach

• Inspired by the (symbolic)
• selective modification model
• (Smith, Osherson, Rips Keane 1988)
• Prototype representations consisting of
• Symbolic, structured concepts
• Attribute-value pairs, weighted by salience
• Modifiers act on prototypes by increasing
  salience of attribute and changing its value
• Tverskys (1977) contrast rule for similarity

19
A connectionist approach
Adjective
Noun
Conceptual Layer
20
A connectionist approach
Adjective
Noun
Conceptual Layer
Color
21
A connectionist approach
Adjective
Noun
Conceptual Layer
Taste
22
A connectionist approach
Adjective
Noun
Conceptual Layer
Peel
23
A connectionist approach
Adjective
Noun
Pulp
Conceptual Layer
24
A connectionist approach
Adjective
Noun
(Color x Peel)
Conceptual Layer
Peel
Color
25
A connectionist approach
Adjective
Noun
Conceptual Layer
red
26
A connectionist approach
Adjective
Noun
Conceptual Layer
white
27
A connectionist approach
Adjective
Noun
white pulp
red peel
Conceptual Layer
red
28
A connectionist approach
Adjective
Noun
Conceptual Layer
29
A connectionist approach
Adjective
Noun
Conceptual Layer
30
A connectinist approach

• Default uniform connections within conceptual
  layer
• In training, instances are presented at the
  conceptual layer and at the (localist) adjective
  and noun layers.
• The prototypes learned by the model are found by
  turning on units in the adjective and noun layers
  and letting the network settle into a
  representation at the conceptual layer.

31
Prototype effects

• Using Tversky's (1977) contrast rule (formulated
  for activation vectors) sim(s,t) ??i
  min(si,ti) ? ??i si ?ti
• sim(sred apple, t1) gt sim(sapple, t1)
• sim(spurple apple,t2)?sim(sapple,t2)
• gt sim(sred apple,t1)?sim(sapple,t1)
• t1 a red apple, t2 a purple apple

32
Conclusions

• Compositionality alone is not formally
  restrictive enough to explain the type of
  productivity in NL
• Semantic contributions of constituents to their
  complex expressions are more complex and less
  systematic than we might expect
• Encyclopedic knowledge is necessary to account
  for the variability in the semantic contribution
  of constituents to complex phrases
• Connectionist approaches easily handle the
  accumulation and application of this knowledge.