Dependency Model Using Posterior Context

1
Dependency Model Using Posterior Context

• Kiyotaka Uchimoto Masaki Murata
• Satoshi Sekine Hitoshi Isahara
• Kansai Advanced Research Center, Communications
  Research Laboratory, Japan
• New York University, USA

2
Background

• Japanese dependency structure analysis

?????????????? Taro bought a red rose.

• Preparing a dependency matrix
• Finding an optimal set of dependencies for the
  entire sentence

3
Conventional (old) model
bunsetsu
dependency
?

• Statistical approach
• Each element in the dependency matrix is
  estimated as a probability.
• Assigning one of two tags, a 1 or a 0, to
  each relationship between two bunsetsus
• Whether or not there is a dependency between two
  bunsetsus
• Considers only the relationship between two
  bunsetsus.

or
1
0
4
New model using posterior context
dependent
between
beyond
?
or
or
1
2
0
0

• A relationship between two bunsetsus
• The anterior bunsetsu can depend on
• 0 a bunsetsu between the two
• 1 the posterior bunsetsu
• 2 a bunsetsu beyond the posterior one
• The dependency probability of two bunsetsus
• Product of the probabilities of the relationship
  between the left bunsetsu and those to its right
  in a sentence
• Overall dependencies in a sentence
• Product of the probabilities of all the
  dependencies
• Identified by analyzing a sentence from right to
  left

5
Bunsetsu
Current bunsetsu
1
3
4
5
2
Modifiee candidates
Normalized dependency probability
0.4 0.1 1.0 1.0 0.6 0.155
18.0
0.6 0.3 1.0 1.0 0.6 0.329
38.1
Candidate Beyond (bynd) Dependent (dpnd) Between (btwn)
1 0.6 0.4 0
2 0.6 0.3 0.1
3 0.3 0.5 0.2
4 0.1 0.5 0.4
5 0 0.4 0.6
0.6 0.6 0 1.0 0.6 0
0.6 0.6 1.0 0 0.6 0
0.6 0.6 1.0 1.0 0.4 0.379
43.9
6
Experiments

• Implemented the models within a maximum entropy
  framework
• Features basically some attributes of a bunsetsu
  itself or those between bunsetsus
• Using the Kyoto University text corpus (Kurohashi
  and Nagao, 1997)
• a tagged corpus of the Mainichi newspaper
• Training 7,958 sentences (Jan. 1st to 8th)
• Testing 1,246 sentences (Jan. 9th)
• The input sentences were morphologically analyzed
  and their bunsetsus were identified correctly.

7
Results of dependency analysis

• The accuracy of the new model was about 1 better
  than that of the old model and there was a 3
  improvement in sentence accuracy even using
  exactly the same features.

8
Relationship between the number of bunsetsus and
accuracy

• The accuracy of the new model is almost always
  better than that of the old model.

9
Amount of training data and accuracy

• The accuracy of the new model is about 1 higher
  than that of the old model for any size of
  training data.

10
Conclusion

• A new model for dependency structure analysis
• Learns the relationship between two bunsetsus as
  three categories between, dependent, and
  beyond.
• Estimates the dependency likelihood by
  considering not only the relationship between two
  bunsetsus but also the relationship between the
  left bunsetsu and all of the bunsetsus to its
  right.
• The dependency accuracy of the new model was
• Almost always better than that of the old model
  for any sentence length.
• About 1 higher than that of the old model for
  any size of training data used.
• Future work
• Applying the similar model to English sentences