An Attempt at Unsupervised Learning of Hierarchical Dependency Parsing

1
An Attempt at Unsupervised Learning of
Hierarchical Dependency Parsing

• via the Dependency Model with Valence (DMV)

2
Motivation

• Dependency Parsing
• Search Query Refinement
• Statistical Machine Translation
• Unsupervised Learning
• Availability of Large Quantities of Data

3
DMV

• Pick a Direction (left or right)
• Generate the first child, or stop
• Generate more children, until stop.
• Repeat in the other direction.
• Recurse
• Porder
• Pstop
• Pattach

4
EM

• Inside-Outside Algorithm
• Inside Pi(i,X,j) P(X derives ij)
• Outside Po(i,X,j) P(S derives 0iXjl)
• Re-Estimation
• Frequency of sub-tree (i,X,j)Pi(i,X,j)Po(i,X,j
  )

5
Evaluation

• Head-percolation of Penn Treebank parses
• edges correct (directed or undirected) in the
  best (P)CFG parse
• Zero Knowledge 14.4 (29.9)
• Adjacent Word Heuristic 33.6
• Klein Manning 43.2 (63.7)
• Oracle 75.5 (77.5)
• - Pattach 60.0 (63.3) - Pstop 53.9 (57.7)
• - PstopA 50.0 (54.8) - PstopN 12.5 (30.8)

6
EM

• Didnt work out always made things worse, even
  when initialized with very good solutions.
• If started using Zero Knowledge, then after 1
  iteration already gets 18.4 (38.4), then worsens.
• If started using an Ad-Hoc Harmonic for Pattach,
  then 21.5 (47.1) after 1 iteration, then worse,
  and similarly even for the Oracle solution
• Summary
• - DMV useful, simple, extensible model
• - EM more thorough debugging needed.