Antonymy and Conceptual Vectors

1
Antonymy and Conceptual Vectors
Didier Schwab, Mathieu Lafourcade, Violaine
Prince
presented by Ch. Boitet (works with
M. Lafourcade on conceptual vectors UNL)
Laboratoire dinformatique, de robotique Et de
microélectronique de Montpellier CNRS -
Université Montpellier II
2
Outline

• The main idea
• Background on conceptual vectors
• How we use CVs
• why we need to distinguish CVs of antonyms
• Brief study of antonymies
• Representation of antonymies
• Measure for  antonymousness 

3
The main idea

• Work on meaning representation in NLP, using
  conceptual vectors (CV)
• applications WSD thematic indexing
• but V(existence) V(non-existence) !
• basic  concepts  activated the same
• Idea
• use lexical functions to improve the adequacy
• For this,  transport  the lexical functions in
  the vector space

4
Background on conceptual vectors

• Lexical Item ideas combination of concepts
  Vector V
• Ideas space vector space (generator space)
• Concept idea vector Vc
• Vc taken from a thesaurus hierarchy (Larousse)
• translation of Rogets thesaurus, 873 leaf nodes
• the word peace has non zero values for concept
  PEACE and other concepts

5
Our conceptual vectors Thesaurus

• H thesaurus hierarchy K concepts
• Thesaurus Larousse 873 concepts
• V(Ci) lta1, , ai, , a873gt
• aj 1/ (2 Dum(H, i, j))

1/4
1
1/4
1/4
1/16
1/16
1/64
1/64
2
6
4
6
Conceptual vectors Concept c4 PEACE
peace
conflict relations
hierarchical relations
society
The world, manhood
7
Conceptual vectors Term peace
c4PEACE
8

exchange
profit
finance
9
Angular or  thematic  distance

• Da(x,y) angle(x,y) acos(sim(x,y))
• acos(x.y /x y )
• 0 D(x,y) ? (positive components)
• If 0 then x and y are colinear same idea.
• If ?/2 nothing in common.

x
y
10
The angular distance is a true distance

• DA(x, y) acos(sim(x,y))
• DA(x, y) acos(x.y/xy))
• DA(x, x) 0
• DA(x, y) DA(y, x)
• DA(x, y) DA(y, z) ? DA(x, z)
• DA(0, 0) 0 and DA(x, 0) ?/2 by definition
• DA(?x, ?y) DA(x, y) with ?? ? 0
• DA(?x, ?y) ? - DA(x, y) with ?? lt 0
• DA(xx, xy) DA(x, xy) ? DA(x, y)

11
Thematic Distance (examples)

• Da(anteater , anteater ) 0 (0)
• Da(anteater , animal ) 0,45 (26)
• Da(anteater , train ) 1,18 (68)
• Da(anteater , mammal ) 0,36 (21)
• Da(anteater , quadruped ) 0,42 (24)
• Da(anteater , ant ) 0,26 (15)
• thematic distance ? ontological distance

12
Vector Proximity

• Function V gives the vectors closest to a
  lexical item.
• V (life) life, alive, birth
• V (death) death, to die, to kill

13
How we build use conceptual vectors

• Conceptual vectors give thematic representations
• of word senses
• of words (averaging CVs of word senses)
• of the content ( ideas ) of any textual segment
• New CVs for word senses are permanently learned
  from NL definitions
• coming from electronic dictionaries
• CVs of word senses are permanently recomputed
• for French, 3 years, 100000 words, 300000 CVs

14
Continuous building of the conceptual vectors
database
15
We should distinguish CVs of different but
related words

• Non-existent who or which does not exist
• cold ant warm, hot
• Without a specific treatment, we get
• V(non-existence) V(existence)
• V(cold) V(hot)
• We want to obtain
• V(non-existence) ? V(existence)
• V(cold) ? V(hot)

16
in order to improve applications and resources

• Applications more precision
• Thematic analysis of texts
• Thematic analysis of definitions
• Resources coherence adequacy
• General coherence of the CV data base
• Conceptual Vector quality (adequacy)

17
Lexical functions may help!

• Lexical function (Meltchuk)
• WS ? WS1WSn
• synonymy (Syn), antonymy (Anti),
  intensification (Magn)
• Examples
• Syn (car) automobile
• Anti (respect) disrespect disdain
• Sing (fleet) boat, ship embarcation

18
Method transport the LFs as functions on the CV
space

• e.g. for antonymy,
• to get V(non-existence) ? V(existence)
• find vector function Anti such that
• V(non-existence)
• V(Anti(existence)) Anti (V(existence))
• similarly for other lexical functions
• we simply began by studying antinomy

19
Brief study of antonymy

• Definition
• Two lexical items are in antonymy relation if
  there is a symmetry between their semantic
  components relatively to an axis
• Antonymy relations depend on the type of medium
  that supports symmetry
• There are several types of antonymy
• On the axis, there are fixed points
• Anti (V(car)) V(car) because Anti (car) ?

20
1- Complementary antonymy

• Values are boolean symmetric (0?1)
• Examples
• event/non-event dead/alive
• existence/non-existence
• He is present ? He is not absent
• He is absent ? He is not present

21
2- Scalar antonymy

• Values are scalar
• Symmetry is relative to a reference value
• Examples cold/hot, small/tall
• This man is small ? This man is not tall
• This man is tall ? This man is not small
• This man is neither tall nor small
• reference value  of medium height 

22
3- Dual Antonymy (1)

• Conversive duals
• same semantics but inversion of roles
• Examples sell/buy, husband/wife, father/son
• Jack is Johns son ? John is Jacks father
• Jack sells a car to John ? John buys a car from
  Jack

23
3- Dual Antonymy (2)

• Contrastive duals
• contrastive expressions accepted by usage
• Cultural sun/moon, yin/yang
• Associative question/answer
• Spatio-temporal birth/death, start/finish

24
Coherence and adequacy of the base

• Learning bootstrap based on a kernel composed of
  pre-computed vectors considered as adequate
• Learning must be coherent preserve adequacy
• Adequacy judgement that activations of concepts
  (coordinates) make sense for the meaning
  corresponding to a definition
• For coherence improvement, we use semantic
  relations between terms

25
Antonymy function

• Based on the antonym vectors of concepts one
  list for each kind of antonymy
• Antic (EXISTENCE) V (NON-EXISTENCE)
• Antis (HOT) V (COLD)
• Antic (GAME) V (GAME)
• Anti (X,C) builds the vector  opposite  of
  vector X in context C

26
Construction of the antonym vector of X in
context C

• The method is to focus on the salient notions in
  V(X) and V(C)
• If the notions can be opposed, then the antonym
  should have the inverse ideas in the same
  proportions
• The following formula was obtained after several
  experiments

27
Construction of the antonym vector (2)
N

• AntiR (V(X), V(C)) ? Pi AntiC (Ci, V(C))
• Pi V max (V(X), V(Ci))
• Not symmetrical
• Stress more on vector X than on context C
• Consider an important idea of the vector to
  oppose even if it is not in the referent

i1
1CV(V(X))
Xi
28
Results

• V (Anti (death, life death)) (LIFE 0,3),
  (birth 0,48), (alive 0,54)
• V (Anti (life, life death)) (death
  0,336), (killer 0,45), (murdered 0,53)
• V (Anti (LIFE)) (DEATH 0,034), (death
  0,43), (killer 0,53)...

29
Antonymy evaluation measure

• Assess  how much  two lexical items are
  antonymous
• Manti(A,B) DA(A?B, Anti(A,C) ?Anti(B,C))

A
Anti(B)
Anti(A)
B
30
Examples

• Manti (EXISTENCE, NON-EXISTENCE) 0,03
• Manti (existence, non-existence) 0,44
• Manti (EXISTENCE, CAR) 1,45
• Manti (existence, car) 1,06
• Manti (CAR, CAR) 0,006
• Manti (car, car) 0,407

31
Conclusion and perspectives

• Progress so far
• Antonymy definition based on a notion of symmetry
• Implemented formula to compute an antonym vector
• Implemented measure to assess the level of
  antonymy between two items
• Perspectives
• Use of the symbolic opposition found in
  dictionaries
• Search the opposite meaning of a word
• Study of the other semantic relations
• (hyperonymy/hyponymy, meronymy/holonymy)