CPSC 503 Computational Linguistics

1
CPSC 503Computational Linguistics

• Lecture 3
• Giuseppe Carenini

2
Formalisms and associated Algorithms
Linguistic Knowledge

• State Machines (no prob.)
• Finite State Automata (and Regular Expressions)
• Finite State Transducers

(English) Morphology
Syntax
Rule systems (and prob. version) (e.g., (Prob.)
Context-Free Grammars)
Semantics
Logical formalisms (First-Order Logics)
Pragmatics Discourse and Dialogue
AI planners
3
Computational tasks in Morphology

• Recognition recognize whether a string is an
  English/ word (FSA)
• Parsing/Generation

stem, class, lexical features
.
word
.
buy V PAST-PART
bought
e.g.,
buy V PAST

• Stemming

stem
word
.
4
Today Sept 16

• Finite State Transducers (FSTs) and Morphological
  Parsing
• Stemming (Porter Stemmer)

5
FST definition (Recap.)

• Q a finite set of states
• I,O input and an output alphabets (which may
  include e)
• S a finite alphabet of complex symbols io, i?I
  and o?O
• Q0 the start state
• F a set of accept/final states (F?Q)
• A transition relation d that maps QxS to 2Q

E.g., Q 3 Ia,b,c, e Oa,b S? 0
lt d lt ?
6
FST can be used as

• Translators input one string from I, output
  another from O (or vice versa)
• Recognizers input a string from IxO
• Generator output a string from IxO

Terminology warning!
E.g., if Ia,b Oa,b,e
7
FST inflectional morphology of plural
Some regular-nouns
Notes
X -gt XX
lexicalsurface
Some irregular-nouns
oi
8
Examples
lexical
m
i
c
surface
e
lexical
N
PL
c
a
t
surface
9
Computational Morphology Problems/Challenges

• Ambiguity one word can correspond to multiple
  structures (more critical in morphologically
  richer languages)
• Spelling changes may occur when two morphemes
  are combined
• e.g. butterfly -s -gt butterflies

10
Deal with Morphological Ambiguity

• Find all the possible outputs (all paths) and
  return them all (without choosing)

Then Part-of-speech tagging to choose look at
the neighboring words
11
Ambiguity more complex example

• Whats the right parse for Unionizable?
• Union-ize-able
• Un-ion-ize-able
• Each would represent a valid path through an FST
  for derivational morphology.
• Both Adj

12
(2) Spelling Changes

• When morphemes are combined inflectionally the
  spelling at the boundaries may change

• Examples
• E-insertion when s is added to a word, -e is
  inserted if word ends in s, -z, -sh, -ch, -x
  (e.g., kiss, miss, waltz, bush, watch, rich, box)
• Y-replacement when s or -ed are added to a word
  ending with a y, -y changes to ie or i
  respectively (e.g., butterfly, try)

13
Solution Multi-Tape Machines

• Add intermediate tape
• Use the output of one tape machine as the input
  to the next
• Add intermediate symbols
• morpheme boundary
• word boundary

14
Multi-Level Tape Machines
FST-1
FST-2

• FST-1 translates between the lexical and the
  intermediate level
• FTS-2 handles the spelling changes (due to one
  rule) to the surface tape

15
FST-1 for inflectional morphology of plural
(Lexical lt-gt Intermediate )
Some regular-nouns
PLs



Some irregular-nouns
oi
es
e
PL
16
Example
lexical
f
o
x
PL
N
intemediate
s
e
m
o
u
lexical
N
PL
intemediate
17
FST-2 for E-insertion(Intermediate lt-gt Surface)

• E-insertion when s is added to a word, -e is
  inserted if word ends in s, -z, -sh, -ch, -x
• as in foxs lt-gt foxes

e
18
Examples
intermediate

s
f
o
x

surface
intermediate

i
b
o
x
n
g

surface
19
Where are we?

20
Final Scheme Part 1
21
Final Scheme Part 2
22
Intersection (FST1, FST2) FST3

• States of FST1 and FST2 Q1 and Q2
• States of intersection (Q1 x Q2)
• Transitions of FST1 and FST2 d1, d2
• Transitions of intersection d3

• For all i,j,n,m,a,b d3((q1i,q2j), ab)
  (q1n,q2m) iff
• d1(q1i, ab) q1n AND
• d2(q2j, ab) q2m

ab
q1i
q1n
ab
ab
ab
(q1i,q2j)
(q1n,q2m)
?
q2j
q2m
23
Composition(FST1, FST2) FST3

• States of FST1 and FST2 Q1 and Q2
• States of composition Q1 x Q2
• Transitions of FST1 and FST2 d1, d2
• Transitions of composition d3

• For all i,j,n,m,a,b d3((q1i,q2j), ab)
  (q1n,q2m) iff
• There exists c such that
• d1(q1i, ac) q1n AND
• d2(q2j, cb) q2m

ac
q1i
q1n
cb
ab
ab
?
q2j
q2m
(q1i,q2j)
(q1n,q2m)
24
FSTs in Practice

• Install an FST package (pointers)
• Describe your formal language (e.g, lexicon,
  morphotactic and rules) in a RegExp-like notation
  (pointer)
• Your specification is compiled in a single FST
• Ref Finite State Morphology (Beesley and
  Karttunen, 2003, CSLI Publications) (pointer)

• Complexity/Coverage
• FSTs for the morphology of a natural language may
  have 105 107 states and arcs
• Spanish (1996) 46x103 stems 3.4 x 106 word
  forms
• Arabic (2002?) 131x103 stems 7.7 x 106 word
  forms

25
Other important applications of FST in NLP

• From segmenting words into morphemes to
• Tokenization
• finding word boundaries in text (?!) maxmatch
• Finding sentence boundaries punctuation but .
  is ambiguous look at example in Fig. 3.22
• Shallow syntactic parsing e.g., find only noun
  phrases
• Phonological Rules (Chpt. 11)

26
Computational tasks in Morphology

• Recognition recognize whether a string is an
  English word (FSA)
• Parsing/Generation

stem, class, lexical features
.
word
.
buy V PAST-PART
bought
e.g.,
buy V PAST

• Stemming

stem
word
.
27
Stemmer

• E.g. the Porter algorithm, which is based on a
  series of sets of simple cascaded rewrite rules
• (condition) S1-gtS2
• ATIONAL ? ATE (relational ? relate)
• (v) ING ? ? if stem contains vowel (motoring ?
  motor)
• Cascade of rules applied to computerization
• ization -gt -ize computerize
• ize -gt e computer
• Errors occur
• organization ? organ, university ? universe

Code freely available in most languages Python,
Java,
28
Stemming mainly used in Information Retrieval

1. Run a stemmer on the documents to be indexed
2. Run a stemmer on users queries
3. Compute similarity between queries and documents
   (based on stems they contain)

Seems to work especially well with smaller
documents
29
Porter as an FST

• The original exposition of the Porter stemmer did
  not describe it as a transducer but
• Each stage is a separate transducer
• The stages can be composed to get one big
  transducer

30
NLP research at UBC

• TOPICS
• Generation and Summarization of Evaluative Text
  (e.g., customer reviews)
• Summarization of conversations (emails, blogs,
  meetings)
• Subjectivity Detection, Domain Adaptation,
  Rhetorical Parsing
• PEOPLE G. Carenini R. Ng (Profs), G. Murray
  (Postdoc) Students
• SUPPORT NSERC, Google, BObjects(now SAP),
• COLLABORATIONS MSResearch

http//people.cs.ubc.ca/rjoty/Webpage/
31
Next Time

• Read handout
• Probability
• Stats
• Information theory
• Next Lecture
• finish Chpt 3, 3.10-11
• Start Probabilistic Models for NLP (Chpt. 4, 4.1
  4.2 and 5.9!)