Forest-to-String Statistical Translation Rules

1
Forest-to-String Statistical Translation Rules

• Yang Liu, Qun Liu, and Shouxun Lin
• Institute of Computing Technology
• Chinese Academy of Sciences

2
Outline

• Introduction
• Forest-to-String Translation Rules
• Training
• Decoding
• Experiments
• Conclusion

3
Syntactic and Non-syntactic Bilingual Phrases
NP
NP
VP
NR
NN
VV
NN
syntactic
BUSH
PRESIDENT
MADE
SPEECH
non-syntactic
President
Bush
made
a
speech
4
Importance of Non-syntactic Bilingual Phrases

• About 28 of bilingual phrases are non-syntactic
  on a English-Chinese corpus (Marcu et al., 2006).
• Requiring bilingual phrases to be syntactically
  motivated will lose a good amount of valuable
  knowledge (Koehn et al., 2003).
• Keeping the strengths of phrases while
  incorporating syntax into statistical translation
  results in significant improvements (Chiang,
  2005) .

5
Previous Work
Galley et al., 2004
NP
NP
VP
NR
NN
VV
NN
BUSH
PRESIDENT
MADE
SPEECH
President
Bush
made
a
speech
6
Previous Work
Marcu et al., 2006
NPB_NN
DT
JJ
NPB
the
mutual
DT
JJ
NN
THE
MUTUAL
the
mutual
understanding
NPB
THE
MUTUAL
UNDERSTANDING
NPB_NN
NN
7
Previous Work
Liu et al., 2006
NP
NP
VP
NP
NR
NN
VV
NN
NR
NN
BUSH
PRESIDENT
MADE
SPEECH
BUSH
PRESIDENT
President
Bush
President
Bush
8
Our Work

• We augment the tree-to-string translation model
  with
• forest-to-string rules that capture non-syntactic
  phrase pairs
• auxiliary rules that help integrate
  forest-to-string rules into the tree-to-string
  model

9
Outline

• Introduction
• Forest-to-String Translation Rules
• Training
• Decoding
• Experiments
• Conclusion

10
Tree-to-String Rules
VP
IP
SB
VP
NN
WAS
NP
VV
NP
VP
PU
GUNMAN
NN
KILLED
the
gunman
was
killed
by
11
Derivation

• A derivation is a left-most composition of
  translation rules that explains how a source
  parse tree, a target sentence, and the word
  alignment between them are synchronously
  generated.

12
A Derivation Composed of TRs
13
A Derivation Composed of TRs
IP
IP
NP
VP
PU
NP
VP
PU
14
A Derivation Composed of TRs
IP
NP
VP
PU
NP
NN
NN
GUNMAN
GUNMAN
the
gunman
the
gunman
15
A Derivation Composed of TRs
IP
VP
NP
VP
PU
SB
VP
NN
SB
VP
WAS
NP
VV
GUNMAN
WAS
NP
VV
NN
KILLED
NN
KILLED
was
killed
by
killed
the
gunman
was
by
16
A Derivation Composed of TRs
IP
NP
VP
PU
NN
NN
SB
VP
POLICE
GUNMAN
WAS
NP
VV
NN
KILLED
POLICE
police
killed
the
gunman
was
by
police
17
A Derivation Composed of TRs
PU
.
.
18
Forest-to-String and Auxiliary Rules
IP
NP
NP
VP
PU
NN
SB
SB
VP
GUNMAN
WAS
the
gunman
was
forest tree sequence !
care about only root sequence while
incorporating forest rules
19
A Derivation Composed of TRs, FRs, and ARs
20
A Derivation Composed of TRs, FRs, and ARs
IP
IP
NP
VP
PU
NP
VP
PU
SB
VP
SB
VP
21
A Derivation Composed of TRs, FRs, and ARs
IP
NP
NP
VP
PU
NN
SB
SB
VP
GUNMAN
WAS
NN
GUNMAN
WAS
the
gunman
was
the
gunman
was
22
A Derivation Composed of TRs, FRs, and ARs
IP
VP
NP
VP
PU
NP
PU
VV
NN
SB
VP
.
.
KILLED
GUNMAN
WAS
NP
VV
KILLED
.
killed
by
killed
the
gunman
was
by
.
23
A Derivation Composed of TRs, FRs, and ARs
IP
NP
NP
VP
PU
NN
SB
VP
.
NN
GUNMAN
WAS
NP
VV
POLICE
NN
KILLED
POLICE
police
killed
the
gunman
was
by
police
.
24
Outline

• Introduction
• Forest-to-String Translation Rules
• Training
• Decoding
• Experiments
• Conclusion

25
Training

• Extract both tree-to-string and forest-to-string
  rules from word-aligned, source-side parsed
  bilingual corpus
• Bottom-up strategy
• Auxiliary rules are NOT learnt from real-world
  data

26
An Example
NR
BUSH
Bush
27
An Example
NN
PRESIDENT
President
28
An Example
VV
MADE
made
29
An Example
NN
SPEECH
speech
30
An Example
NP
NP
NR
NN
NR
NN
PRESIDENT
President
NP
NP
NR
NN
NR
NN
BUSH
PRESIDENT
BUSH
President
Bush
Bush
31
An Example
32
An Example
VP
VP
VV
NN
VV
NN
MADE
a
made
a
VP
VP
VV
NN
VV
NN
SPEECH
MADE
SPEECH
a
speech
made
a
speech
33
An Example
NP
VV
NP
VV
NR
NN
10 FRs
NP
VV
NR
NN
MADE
BUSH
PRESIDENT
made
President
Bush
34
An Example
35
An Example
NP
NP
VP
max_height 2
36
Why We Dont Extract Auxiliary Rules ?
IP
NP
VP-B
NP-B
NP-B
VV
NR
NR
NN
CC
NN
NN
STEP
SHANGHAI
PUDONG
DEVE
WITH
LEGAL
ESTAB
The development of Shanghai s Pudong is in step
with the establishment of its legal system
37
Outline

• Introduction
• Forest-to-String Translation Rules
• Training
• Decoding
• Experiments
• Conclusion

38
Decoding

• Input a source parse tree
• Output a target sentence
• Bottom-up strategy
• Build auxiliary rules while decoding
• Compute subcell divisions for building auxiliary
  rules

39
An Example
Rule
NR
BUSH
Bush
Derivation
( NR BUSH ) Bush 11
Translation
Bush
40
An Example
Rule
NN
PRESIDENT
President
Derivation
( NN PRESIDENT ) President 11
Translation
President
41
An Example
Rule
VV
MADE
made
Derivation
( VV MADE ) made 11
Translation
made
42
An Example
Rule
NN
SPEECH
speech
Derivation
( NN SPEECH ) speech 11
Translation
speech
43
An Example
Rule
NP
NR
NN
Derivation
( NP ( NR ) ( NN ) ) X1 X2 12 21 ( NR
BUSH ) Bush 11 ( NN PRESIDENT )
President 11
Translation
President Bush
44
An Example
Rule
Derivation
Translation
45
An Example
VP
Rule
VV
NN
MADE
made
a
Derivation
( VP ( VV MADE ) ( NN ) ) made a X 11
23 ( NN SPEECH ) speech 11
Translation
made a speech
46
An Example
Rule
NP
NR
NN
VV
PRESIDENT
MADE
President
made
a
Derivation
( NP ( NN ) ( NN PRESIDENT ) ) ( VV MADE )
President X made a 12 21 33 ( NR BUSH )
Bush 11
Translation
President Bush made a
47
An Example
Rule
Derivation
Translation
48
An Example
Rule
NP
NP
VP
VV
NN
Derivation
( NP ( NP ) ( VP ( VV ) ( NN ) ) ) X1 X2
11 21 32 ( NP ( NN ) ( NN PRESIDENT ) ) ( VV
MADE ) President X made a 12 21 33 (
NR BUSH ) Bush 11 ( NN SPEECH )
speech 11
Translation
President Bush made a speech
49
Subcell Division
11 22 33 44
50
Subcell Division
13 44
51
Subcell Division
14 11 24 12 34 13 44 11 22 34 11 23
44 12 33 44 11 22 33 44
2(n-1)
52
Build Auxiliary Rule
NP
NP
VP
NR
NN
VV
NN
53
Penalize the Use of FRs and ARs

• Auxiliary rules, which are built rather than
  learnt, have no probabilities.
• We introduce a feature that sums up the node
  count of auxiliary rules to balance the
  preference between
• conventional tree-to-string rules
• new forest-to-string and auxiliary rules

54
Outline

• Introduction
• Forest-to-String Translation Rules
• Training
• Decoding
• Experiments
• Conclusion

55
Experiments

• Training corpus 31,149 sentence pairs with 843K
  Chinese words and 949K English words
• Development set 2002 NIST Chinese-to-English
  test set (571 of 878 sentences)
• Test set 2005 NIST Chinese-to-English test set
  (1,082 sentences)

56
Tools

• Evaluation mteval-v11b.pl
• Language model SRI Language Modeling Toolkits
  (Stolcke, 2002)
• Significant test Zhang et al., 2004
• Parser Xiong et al., 2005
• Minimum error rate training optimizeV5IBMBLEU.m
  (Venugopal and Vogel, 2005)

57
Rules Used in Experiments
Rule L P U Total
BP 251, 173 0 0 251,173
TR 56, 983 41, 027 3, 529 101, 539
FR 16, 609 254, 346 25, 051 296, 006
58
Comparison
System Rule Set BLEU4
Pharaoh BP 0.21820.0089
Lynx BP 0.20590.0083
Lynx TR 0.23020.0089
Lynx TR BP 0.23460.0088
Lynx TR FR AR 0.24020.0087
59
TRs Are Still Dominant

• To achieve the best result of 0.2402, Lynx made
  use of
• 26, 082 tree-to-string rules
• 9,219 default rules
• 5,432 forest-to-string rules
• 2,919 auxiliary rules

60
Effect of Lexicalization
Forest-to-String Rule Set BLEU4
None 0.22250.0085
L 0.2297 0.0081
P 0.22790.0083
U 0.22700.0087
L P U 0.23120.0082
61
Outline

• Introduction
• Forest-to-String Translation Rules
• Training
• Decoding
• Experiments
• Conclusion

62
Conclusion

• We augment the tree-to-string translation model
  with
• forest-to-string rules that capture non-syntactic
  phrase pairs
• auxiliary rules that help integrate
  forest-to-string rules into the tree-to-string
  model
• Forest and auxiliary rules enable tree-to-string
  models to derive in a more general way and bring
  significant improvement.

63
Future Work

• Scale up to large data
• Further investigation in auxiliary rules

64

• Thanks!