Viterbi training

1
Viterbi training

• Initialize emission and transition probabilities
  to random numbers.
• while (true)
• Do Viterbi decoding using current parameters
• Save current parameters as previous parameters.
• Re-estimate emission and transition parameters
  from the state path decoded by Viterbi. (add
  pseduocounts, see next page).
• if sum of absolute difference between current
  and previous parameters is tiny (e.g., lt
  0.00001), break
• end
• print current parameter and P(sequence, viterbi
  path)
• Repeat the above procedure several times (with
  different random seed), and compare P(sequence,
  viterbi path). Report the parameters learned that
  give the largest P.

2
Re-estimate parameters with pseudocounts

• Count number of transitions, n_xy, where x, y
  a, b
• t_xy (n_xyc) / sum_x(n_xyc)
• e.g. t_ab (n_ab 1) / (n_ab n_aa 2)
• Count number of symbols in each state, N_aX and
  N_bX, where X A, C, G, T
• e_aX (N_aX 1) / (sum_X N_aX 4)
• e_bX (N_bX 1) / (sum_X N_bX 4)

Pseudocount
3
Backward-Forward algorithmCompute sum of
probabilities in log space

• Two probabilities x and y, x lt y
• lx log(x), ly log(y), (lx lt ly)
• z x y y (1 x/y)
• lz log(z) log(xy)
• log(y) log(1 x/y)
• ly log(1 exp(log(x)-log(y))
• ly log(1 exp(lx ly))
• Also see page 4 in this doc http//cs.utsa.edu/j
  ruan/teaching/cs5263_fall_2007/proj1.pdf
• and page 77 of the handouts.