Trellis Algorithms

1
Trellis Algorithms

• Array of states vs. time

2
Trellis Algorithms

• Overlap in paths implies repetition of the same
  calculations
• Harness the overlap to make calculations
  efficient
• A node at (si , t) stores info about state
  sequences that contain Xt si
• This is the basis of dynamic programming

3
Trellis Algorithms

• Consider 2 states and 3 time points

4
Forward Algorithm

• A node at (si , t) stores info about state
  sequences up to time t that arrive at si

s1
sj
s2
5
Forward Algorithm
6
Viterbi Algorithm

• Define

is the best score along a single path, at
time t, which accounts for the first t
observations and ends in state si
By induction we have
7
Viterbi Algorithm
8
Viterbi Algorithm
9
Viterbi vs. Forward Algorithm

• Similar in implementation
• Forward sums over all incoming paths
• Viterbi maximises
• Viterbi probability ? Forward probability
• Both efficiently implemented using a trellis
  structure

10
Implementations
function FORWARD(observations, a, b) returns
forward-probability num-states ?
NUM-OF-STATES(pi) num-obs ?
LENGTH(observations) Create and initialise
probability matrix forwardnum-states1,
num-obs1 forward0, 0 ? 1.0 for each
time t from 1 to num-obs for each state s
from 1 to num-states sum ? 0
for each previous state p from 0 to
num-states sum ? sum forwardp,
t-1 ap, s bs, observationst
end forwards, t ? sum end
end sum ? 0 for each previous state
p from 0 to num-states sum ? sum
forwardp, num-obs ap, FINAL-STATE end
return sum
11
Implementations
function VITERBI(observations, a, b) returns
viterbi-probability num-states ?
NUM-OF-STATES(pi) num-obs ?
LENGTH(observations) Create and initialise
probability matrix viterbinum-states1,
num-obs1 viterbi0, 0 ? 1.0 for each
time t from 1 to num-obs for each state s
from 1 to num-states max ? 0
for each previous state p from 0 to
num-states max ? MAX(max,
viterbip, t-1 ap, s bs,
observationst) end
viterbis, t ? max end end max
? 0 for each previous state p from 0 to
num-states max ? MAX(max, viterbip,
num-obs ap, FINAL-STATE) end return
max