Title: I Spent an Interesting Evening Recently with a Grain of Salt
1"I Spent an Interesting Evening Recently with a
Grain of Salt "
"As I've commented before, really relating to
someone involves standing next to impossible."
"One morning I shot an elephant in my arms and
kissed him. So it was too small for a pill? "
"Oh, sorry. Nevermind. I am afraid of it
becoming another island in a nice suit."
The power of the 2nd order Markov Chain. After
Claude Shannon. With thanks to Alyosha Efros.
2Markov chains and processes
1st order Markov chain
2nd order Markov chain
1st order with stochastic observations -- HMM
3HMM analysis of speech
4.. Markov processes
- Markov Random Field eg for texture modelling
(section 4) - intractable cf. Markov chain
Intractable because of
- bidirectionality
- twodimensionality
5Markov Chain joint distribution
1st order Markov chain
Joint distribution
Markov property, 1st order
-- process distribution, so joint
Matrix
-- reversible
6Markov Chain transition diagram
-- for
1st order Markov chain
Process model
7Hidden Markov Model (HMM)
emission/observation
Observer distribution
eg
Examples
speech
phoneme
spectral o/p
disparity
pixel pair
stereo
viseme
motion field
gesture
Hidden state
Observations
8HMMs three famous problems (Rabiner, 89,93)
- Model classification
- MAP state sequence estimation
- MLE model parameter learning
for various
emission
transition
9HMMs three famous solutions
- Model classification
- MAP state sequence estimation
- MLE model parameter learning
Forward algorithm
Viterbi algorithm (DP)
Baum-Welch algorithm (DP)
101. Model classification Forward algorithm
Forward probability
Iteration
Model likelihood
112. MAP state sequence Viterbi algorithm (DP)
Partial posterior probability
Iteration
MAP sequence
for efficiency (additivity), define instead
123. MLE for HMM parameter learning Baum-Welch
would like
so do EM maximising
where
emission
transition
substituting expected values from
and
using FB (see Belief Propagation), and iterate
13Markov Chain Bidirectional process
1st order Markov Random Field (MRF)
Joint distribution
equivalent to bidirectional, factorised, joint
distribution
? stereo scanlines
14Low-level view interpolation
StereoReconstruction
Left camera
Right camera
Cyclopean ground truth
15Epipolar matchingas optimal path finding
(Ohta Kanade, 1985 Cox, Hingorani Rao, 1996)
Min cost path
16Elaborated hidden variables ? constraints
(Criminisi, Shotton, Blake, Rother, Torr 2003)
3-move, 1-plane DP
Improvement 3-plane DP explicit occlusion
Much better quality streaks and halo removed a
little temporal flicker
Artefacts streaks, halo, temporal flicker
17Elaborated hidden variables ? constraints
3-plane DP explicit occlusion labelling
18Probabilistic stereo
(Belhumeur 1996)
- Entire disparity distribution ambiguous
matches - MAP invariant to
- State lifetimes (Markov chain) from
- Learned observation distribution
19Probabilistic model
Path Cost
Posterior distribution
Bayes
Markov chain
Observations
20Probabilistic model -- well formed?
- Observations
- Space of paths
- unconstrained
- cyclopean ordering on k
- DG limit
each observable pixel/patch
counted singly
In
not too large
DP complexity
Image width N
21Cyclopean Coordinates
(Belhumeur 1996)
- Cyclopean monotonicity
- HV moves
22FB disparity distributions
disparity
23Video Rewrite (C. Bregler, Siggraph 97)
- Phoneme-labelled speech data from TIMIT
- Learn 3-state HMM for each of 61 phonemes
- Use known groupings into 26 visemes
- Model match viseme triphones in training
- video (coarticulation)
- Synthesis extract video cliplets corresponding
to - desired sequence of viseme triphones
/SIL-T-IY/ /T-IY-P/ /IY-P-AA/ /P-AA-T/
/AA-T-SIL/
Teapot
24Video Rewrite