Some Computational Complexity Results for Synchronous ContextFree Grammars

1
Some Computational Complexity Results for
Synchronous Context-Free Grammars

• Enoch Peserico
• University of Padua

Giorgio Satta University of Padua
presentation David Chiang, University of Maryland
2

• Summary of content
• Synchronous context-free grammars as translation
  models
• Analysis of membership problem
• Time lower bound for membership problem under
  hypothesis of chart parsing techniques
• Analysis of string-to-tree translation problem

3
State of the art machine translation systems
based on statistical models rooted in the theory
of formal grammars/automata Translation models
based on finite state devices cannot easily model
translations between languages with strong
differences in word ordering Recently, several
models based on context-free grammars have been
investigated, borrowing from the theory of
compilers the idea of synchronous rewriting
4

• Translation models based on synchronous
  rewriting
• Inversion Transduction Grammars (Wu, 1997)
• Head Transducer Grammars (Alshawi et al., 2000)
• Tree-to-string models (Yamada Knight, 2001
  Galley et al, 2004)
• Loosely tree-based model (Gildea, 2003)
• Multi-Text Grammars (Melamed, 2003)
• Hierarchical phrase-based model (Chiang, 2005)
• We use synchronous CFGs to study formal
  properties of all these

5

• A synchronous context-free grammar (SCFG) is
  based on three components
• Context free grammar (CFG) for source language
• CFG for target language
• Pairing relation on the productions of the two
  grammars and on the nonterminals in their
  right-hand sides

6
Example (Yamada Knight, 2001)
VB ? PRP(1) VB1(2) VB2(3) VB2 ? VB(1) TO(2)
TO ? TO(1) NN(2) PRP ? he VB1 ?
adores VB ? listening TO ? to NN ? music
VB ? PRP(1) VB2(3) VB1(2) VB2 ? TO(2)
VB(1) ga TO ? NN(2) TO(1) PRP ? kare
ha VB1 ? daisuki desu VB ? kiku no TO ?
wo NN ? ongaku
7
Example (contd)
8

• A pair of CFG productions in a SCFG is called a
  synchronous production
• A SCFG generates pairs of trees/strings, where
  each component is a translation of the other
• A SCFG can be extended with probabilities
• Each pair of productions is assigned a
  probability
• Probability of a pair of trees is the product of
  probabilities of synchronous productions involved

9

• The membership problem (Wu, 1997) for SCFGs is
  defined as follows
• Input SCFG and pair of strings w1, w2
• Output Yes/No depending on whether w1 translates
  into w2 under the SCFG
• Applications in segmentation, word alignment and
  bracketing of parallel corpora
• Assumption that SCFG is part of the input is made
  here to investigate the dependency of problem
  complexity on grammar size

10

• Result Membership problem for SCFGs is
  NP-complete
• Proof uses SCFG derivations to explore space of
  consistent truth assignments that satisfy source
  3SAT instance
• Remarks
• Result transfers to (Yamada Knight, 2001),
  (Gildea, 2003), (Melamed, 2003), which are at
  least as powerful as SCFG

11

• Remarks (contd)
• Problem can be solved in polynomial time if
• input grammar is fixed or production length is
  bounded (Melamed, 2004)
• Inversion Transduction Grammars (Wu, 1997)
• Head Transducer Grammars (Alshawi et al., 2000)
• For NLP applications, it is more realistic to
  assume a fixed grammar and varying input string

12
Providing an exponential time lower bound for the
membership problem would amount to showing P ?
NP But we can show such a lower bound if we make
some assumptions on the class of algorithms and
data structures that we use to solve the
problem Result If chart parsing techniques are
used to solve the membership problem for SCFG, a
number of partial analyses is obtained that grows
exponentially with the production length of the
input grammar
13
Chart parsing for CFGs works by combining
completed constituents with partial analyses
A ? B1 B2 B3 Bn
Three indices are used to process each
combination, for a total number of O(n3)
possible combinations that must be checked, n
the length of the input string
14
Consider the synchronous production A ? B
(1) B (2) B (3) B (4) , A ? B (3) B (1) B (4)
B (2) representing the permutation
15
When applying chart parsing, there is no way to
keep partial analyses contiguous
16
The proof of our result generalizes the previous
observations We show that, for some worst case
permutations of length q, any combination
strategy we choose leads to a number of indices
growing with order at least sqrt(q) Then for
SCFGs of size q, sqrt(q) is an asymptotic lower
bound for the membership problem when chart
parsing algorithms are used
17

• A probabilistic SCFG provides the probability
  that tree t1 translates into tree t2
• Pr( t1 , t2 )
• Accordingly, we can define the probability that
  string w1 translates into string w2
• Pr( w1 , w2 ) ?t1?w1,t2?w2 Pr( t1 , t2 )
• and the probability that string w translates into
  tree t
• Pr( w , t ) ?t1?w Pr( t1 , t )

18

• The string-to-tree translation problem for
  probabilistic SCFGs is defined as follows
• Input Probabilistic SCFG and string w
• Output tree t such that Pr(w, t ) is
  maximized
• Application in machine translation
• Again, assumption that SCFG is part of the input
  is made to investigate the dependency of problem
  complexity on grammar size

19

• Result string-to-tree translation problem for
  probabilistic SCFGs (summing over possible source
  trees) is NP-hard
• Proof reduces from consensus problem
• Strings generated by probabilistic finite
  automaton or hidden Markov model have
  probabilities defined as sum of probabilities of
  several paths
• Maximizing such summation is NP-hard (Casacuberta
  Higuera, 2000) (Lyngso Pedersen, 2002)

20

• Remarks
• Source of complexity of the problem comes from
  the fact that several source trees can be
  translated into the same target tree
• Result persists if there is a constant bound on
  length of synchronous productions
• Open can the problem be solved in polynomial
  time if probabilistic SCFG is fixed?

21

• Constant bound on production length is the most
  realistic hypothesis for NLP applications
• Under such hypothesis, the following problems can
  be solved in polynomial time
• Membership problem
• Compute argmaxt1??t2 Pr( t1 , t2 )
• For input strings w1, w2
• For input string w1
• For input tree t1