Context-Free Grammar Parsing by Message Passing

1
Context-Free Grammar Parsing by Message Passing

• Paper by Dekang Lin and Randy Goebel
• Presented by Matt Watkins

2
Context-Free Grammars
A context-free grammar is represented by a
4-tuple
Vt ? A set of terminals Vn ? A set of
non-terminals P ? A set of production rules S ?
A member of Vn, representing the starting
non-terminal
Context-free grammars are used to represent the
syntax of both programming languages and natural
languages, as well as other things
3
Context-Free Grammars
Example
ltRsSgt ? ltNPgtltVPgt ltRnp1NPgt ? ltngt ltRnp2NPgt ?
ltdgtltngt ltRnp3NPgt ? ltNPgtltPPgt ltRvp1VPgt ?
ltVPgtltPPgt ltRvp2VPgt ? ltvgtltNPgt ltRppPPgt ? ltpgtltNPgt
ltngt ? "I" ltngt ? "saw" ltngt ? "man" ltngt ?
"park" ltvgt ? "saw" ltdgt ? "a" ltdgt ? "the" ltpgt ?
"in"
4
Context-Free Grammars
ltSgt
ltNPgt
ltVPgt
ltPPgt
ltVPgt
ltNPgt
ltNPgt
ltngt
ltvgt
ltdgt
ltngt
ltpgt
ltdgt
ltngt
I
saw
a
man
in
the
park
5
Parsing Context-Free Grammars
Given only the definition of a context-free
grammar, determine if a particular expression is
a valid output of the grammar, and if so, how it
is generated. Earleys parser CYK
parser Message passing parser
6
Grammar Representation
Message passing algorithm represents a CFG as a
6-tuple ltN, O, T, s, P, Lgt N ? set of
non-terminals O ? set of pre-terminals T ? set of
terminals s ? start symbol P ? production rules L
? a lexicon consisting of pairs (w, p), w?T and
p?O
7
Grammar Representation
N S, NP, VP, PP, n, v, p, d O n, v, p,
d T I, saw, a, man, in, the, park s S P
ltRsSgt ? ltNPgtltVPgt ltRnp1NPgt ? ltngt ltRnp2NPgt ?
ltdgtltngt ltRnp3NPgt ? ltNPgtltPPgt
ltRvp1VPgt ? ltVPgtltPPgt ltRvp2VPgt ? ltvgtltNPgt ltRppPPgt
? ltpgtltNPgt
L (I, n), (saw, v, n), (a, d), (man,
n), (in, p), (the, d), (park, n)
8
Message Passing Network
S
VP
1
Rs
0
NP
0
1
Rvp2
0
Rnp3
0
Rnp1
v
1
Rnp2
Rvp1
0
1
Rpp
1
0
1
0
PP
n
d
p
9
Message Passing Rules
Non-terminal nodes are called NT nodes Phrase
structure rule nodes are called PSR
nodes Messages that are passed are integer pairs
representing an interval in the expression being
parsed I saw a man in
the park NT nodes and PSR nodes have
different rules for receiving and sending messages
0 1 2 3 4
5 6 7
10
Message Passing Rules

• NT nodes
• Never send the same message twice
• Always send all unique messages to parents
• PSR nodes
• Have a memory bank of pairs (I, n) where I is an
  interval and n is a link number
• Store pairs where n 0 in memory bank
• Combines pairs where applicable if n ? 0

• Pair (I, n) is combined with (I, n) iff
• ? i, j, k such that I i, j and I j, k
• n n 1
• If n is the last link, send a message to parents

11
Message Passing Rules
Use T to identify the locations of terminals in
the expression to be parsed Use the lexicon to
determine the terminals part of speech. Pass a
message to all parts of speech indicating the
starting and ending position of all the terminals
in the expression.
12
Message Passing Example
S
Parsing I
VP
1
Rs
0
NP
0
1
Rvp2
0
Rnp3
0
Rnp1
v
1
Rnp2
Rvp1
0
1
Rpp
1
0
1
0
PP
n
d
p
0,1
13
Message Passing Example
S
Parsing I
VP
1
Rs
0
NP
0
1
Rvp2
0
Rnp3
0
Rnp1
(0,1, 0)
v
1
Rnp2
Rvp1
0
1
Rpp
1
0
1
0
PP
n
d
p
0,1
14
Message Passing Example
S
Parsing I
VP
1
Rs
0
0,1
NP
0
1
Rvp2
0
Rnp3
0
Rnp1
(0,1, 0)
v
1
Rnp2
Rvp1
0
1
Rpp
1
0
1
0
PP
n
d
p
0,1
15
Message Passing Example
S
Parsing I
VP
(0,1, 0)
1
Rs
0
0,1
NP
0
1
Rvp2
0
Rnp3
0
(0,1, 0)
Rnp1
(0,1, 0)
v
1
Rnp2
Rvp1
0
1
Rpp
1
0
1
0
PP
n
d
p
0,1
16
Completion
Each node will contain a set of intervals that
represent where in the expression the
non-terminals can be found. After message
passing has completed, if the expression is
represented by the grammar, then the network will
contain a packed parse forest
17
Completion
18
Completion
Tested on SPARCstation SLC
19
(No Transcript)
20
Analysis

• Strengths
• O(Gn3) time complexity
• Easily parallelizable
• Can handle empty rules

• Weaknesses
• Must convert some grammars in Backus Naur form

21
Questions?