Realization, identification and filtering for hidden Markov models using matrix factorization techniques

1
Realization, identification and filtering for
hidden Markov models using matrix factorization
techniques

• Bart Vanluyten

2
Mathematical modeling
Bel-20
Process with finite valued output Ç, È,
1. INTRODUCTION
Modeling HMMs Finite valued process Open
problems Relation to LSM
3
Hidden Markov model

• Example Bel-20
• Output process up, down, unchanged
• State process bull market, bear market, stable
  market

Andrey Markov (1856 - 1922)

• State process has Markov property and is hidden

1. INTRODUCTION
Modeling HMMs Finite valued process Open
problems Relation to LSM
4
Finite-valued processes
Coin flipping - dice-tossing (with memory)

• Bio-informatics

TGGAGCCAACGTGGAATGTCACTAGCTAGCTTAGATGGCTAAACGTAGGA
ATAC ACGTGGAATATCGAATCGTTAGCTTAGCGCCTCGACCTAGATCGA
GCCGATCGGACTAGCTAGCTCGCTAGAAGCACCTAGAAGCTTAGACGTGG
AAATTGCTTAATC
head, tail
A, C, G, T
1, 2, ..., 6
FINITE-VALUEDPROCESSES
Speech recognition
Economics
BEL20
4.800 4.600 4.400 4.200 4.000 3.800 3.600
i, e, æ, a, ai, ..., z
Ç, È,
1. INTRODUCTION
Modeling HMMs Finite valued process Open
problems Relation to LSM
5
Open problems for HMMs
Identification problem Given output
sequence Find HMM that models the sequence
Realization problem Given string probs Find
HMM generating string probs
Obtain model from data
Estimation problem Given output sequence Find
state distribution at time
Use model for estimation
1. INTRODUCTION
Modeling HMMs Finite valued process Open
problems Relation to LSM
6
Relation to linear stochastic model (LSM)

• Mathematical model for stochastic processes
• Output process continuous range of values
• State process continuous range of values

NOISE
NOISE
STATE
OUTPUT


1. INTRODUCTION
Modeling HMMs Finite valued process Open
problems Relation to LSM
7
Relation to linear stochastic model
Hidden Markov model
Linear stochastic model
Realization
Identification
Realization
Identification
Estimation
Estimation
1. INTRODUCTION
Modeling HMMs Finite valued process Open
problems Relation to LSM
8
Relation to linear stochastic model
Hidden Markov model
Linear stochastic model
Singular value decomposition
Realization
Identification
Realization
Identification
Estimation
Estimation
1. INTRODUCTION
Modeling HMMs Finite valued process Open
problems Relation to LSM
9
Relation to linear stochastic model
Hidden Markov model
Linear stochastic model
Nonnegative matrix factorization
Singular value decomposition
Realization
Identification
Realization
Identification
Estimation
Estimation
1. INTRODUCTION
Modeling HMMs Finite valued process Open
problems Relation to LSM
10
Outline
Matrix factorizations Given matrix Find low
rank approximation of
2nd objective
Identification problem Given output
sequence Find HMM that models the sequence
Realization problem Given string probs Find
HMM generating string probs
1st objective
Estimation problem Given output sequence Find
state distribution at time
11
Outline
Matrix factorizations Given matrix Find low
rank approximation of
Identification problem Given output
sequence Find HMM that models the sequence
Realization problem Given string probs Find
HMM generating string probs
Estimation problem Given output sequence Find
state distribution at time
12
Matrix Decomposition Rank example

• Matrix

• Matrix decomposition

• Matrix rank
• minimal inner dimension of exact decomposition

2. MATRIX FACTORIZATIONS
Introduction Existing factorizations
Structured NMF NMF without nonneg. factors
13
Low rank matrix approximation

• Rank approximation of

James Sylvester (1814 - 1897)

• Singular value decomposition (SVD)

orthogonal

• SVD yields (global) optimal low rank
  approximation in Frobenius distance

2. MATRIX FACTORIZATIONS
Introduction Existing factorizations
Structured NMF NMF without nonneg. factors
14
Nonnegative matrix factorization

• In some applications is nonnegative and
  and need to be nonnegative too

• Nonnegative matrix factorization (NMF) of

NONNEGATIVE
NONNEGATIVE
NONNEGATIVE

• Algorithm (Kullback-Leibler divergence) Lee,
  Seung

• This thesis 2 modifications to NMF

2. MATRIX FACTORIZATIONS
Introduction Existing factorizations
Structured NMF NMF without nonneg. factors
15
Structured NMF

• Structured nonnegative matrix factorization of

NONNEGATIVE
NONNEGATIVE
NONNEGATIVE
NONNEGATIVE

• Algorithm (Kullback-Leibler divergence)

• Convergence to stationary point of divergence

2. MATRIX FACTORIZATIONS
Introduction Existing factorizations
Structured NMF NMF without nonneg. factors
16
Structured NMF application

• Applications apart from HMMs clustering data
  points

• petal width
• petal length
• sepal width
• sepal length

Given
of 150 iris flowers
SEPAL
Asked Divide 150 flowers into clusters
Setosa
Versicolor
Virginica
2. MATRIX FACTORIZATIONS
Introduction Existing factorizations
Structured NMF NMF without nonneg. factors
17
Structured NMF application

• Computing distance matrix between points
• Applying structured nonnegative matrix
  factorization on distance matrix

• Clustering obtained by

PETAL LENGTH
SEPAL WIDTH
PETAL LENGTH
PETAL WIDTH
PETAL WIDTH
SEPAL LENGTH
SEPAL LENGTH
SEPAL WIDTH
2. MATRIX FACTORIZATIONS
Introduction Existing factorizations
Structured NMF NMF without nonneg. factors
18
NMF without nonnegativity of the factors

• NMF without nonnegativity constraints on the
  factors of

NONNEGATIVE
NO NONNEGATIVITY CONSTRAINTS
NONNEGATIVE

• Example

3
3

• We provide algorithm (Kullback-Leibler divergence)

• Problem allows to deal with upper bounds in an
  easy way

2. MATRIX FACTORIZATIONS
Introduction Existing factorizations
Structured NMF NMF without nonneg. factors
19
NMF without nonnegativity of the factors

• Applications apart from HMMs database compression

Given Database containing 1000 facial images
of size 19 x 19 361 pixels
Asked Compression of database using matrix
factorization techniques
20
1000
361
. . .
NMF without nonneg. factors
Upperbounded NMF without nonneg. fact.
ORIGINAL
NMF
gt 1
Kullback-Leibler divergence
339
383
564
2. MATRIX FACTORIZATIONS
Introduction Existing factorizations
Structured NMF NMF without nonneg. factors
20
Outline
Matrix factorizations Given matrix Find low
rank approximation of
Identification problem Given output
sequence Find HMM that models the sequence
Realization problem Given string probs Find
HMM generating string probs
Estimation problem Given output sequence Find
state distribution at time
21
Hidden Markov models Moore - Mealy
ORDER

• Moore HMM

NONNEGATIVE

• Mealy HMM

NONNEGATIVE
3. REALIZATION
Introduction Realization Quasi realization
Approx. realization Modeling DNA
22
Realization problem

• String from
• String probabilities

• String probabilities generated by Mealy HMM

POSITIVE REALIZATION
such that
3. REALIZATION
Introduction Realization Quasi realization
Approx. realization Modeling DNA
23
Realization problem importance

• Theoretical importance transform external
  model into internal model
• Realization can be used to identify model from
  data

POSITIVE REALIZATION
3. REALIZATION
Introduction Realization Quasi realization
Approx. realization Modeling DNA
24
Realizability problem

• Generalized Hankel matrix

Hermann Hankel (1839 - 1873)

• Necessary condition for realizability Hankel
  matrix has finite rank
• No necessary and sufficient conditions for
  realizability are known
• No procedure to compute minimal HMM from string
  probabilities
• This thesis two relaxations to positive
  realization problem
• Quasi realization problem
• Approximate positive realization problem

3. REALIZATION
Introduction Realization Quasi realization
Approx. realization Modeling DNA
25
Quasi realization problem
QUASI REALIZATION

• NO NONNEGATIVITY
• CONSTRAINTS !

such that

• Finiteness of rank of Hankel matrix N S
  condition for quasi realizability
• Rank of hankel matrix minimal order of exact
  quasi realization
• Quasi realization is more easy to compute than
  positive realization
• Quasi realization typically has lower order than
  positive realization
• Negative probabilities
• No disadvantage in several estimation applications

3. REALIZATION
Introduction Realization Quasi realization
Approx. realization Modeling DNA
26
Partial quasi realization problem

• Given String probabilities of strings up to
  length t
• Asked Quasi HMM that generates the string
  probabilities

• This thesis
• Partial quasi realization problem has always a
  solution
• Minimal partial quasi realization obtained with
  quasi realization algorithm if a rank condition
  on the Hankel matrix holds
• Minimal partial quasi realization problem has
  unique solution (up to similarity transform) if
  this rank condition holds

3. REALIZATION
Introduction Realization Quasi realization
Approx. realization Modeling DNA
27
Approximate quasi realization problem

• Given String probabilities of strings up to
  length t
• Asked Quasi HMM that approximately generates the
  string probabilities

• This thesis algorithm
• Compute low rank approximation of largest Hankel
  block subject to consistency and stationarity
  constraints

Upperbounded NMF without nonnegativity of the
factors with additional constraints

• Reconstruct Hankel matrix from largest block

We prove that rank does not increase in this
step

• Apply partial quasi realization algorithm

3. REALIZATION
Introduction Realization Quasi realization
Approx. realization Modeling DNA
28
Approximate positive realization problem

• Given String probabilities of strings up to
  length t
• Asked Positive HMM that approximately generates
  the string probabilities

APPROXIMATE POSITIVE REALIZATION
such that
3. REALIZATION
Introduction Realization Quasi realization
Approx. realization Modeling DNA
29
Approximate positive realization problem

• Moore, t 2
• Define

• If string probabilities are generated by Moore HMM

where
Structured nonnegative matrix factorization

• Mealy, general t
• Generalize approach for Moore, t 2

3. REALIZATION
Introduction Realization Quasi realization
Approx. realization Modeling DNA
30
Modeling DNA sequences

• DNA

TGGAGCCAACGTGGAATGTCACTAGCTAGCTTAGATGGCTAAACGTAGGA
ATACCCT ACGTGGAATATCGAATCGTTAGCTTAGCGCCTCGACCTAGAT
CGAGCCGATCGGTCT ACTAGCTAGCTCGCTAGAAGCACCTAGAAGCTTA
GACGTGGAAATTGCTTAATCTAG

• 40 sequences of length 200
• String probabilities of strings up to length 4
  stacked in Hankel matrix

Ù
SINGULAR VALUE
Ù

• Kullback-Leibler divergence

Ù
Ù
Ù
Ù
Ù
Ù
Ù
Ù
Ù
Ù
Ù
Ù
Ù
Ù
Ù
Ù
Ù
Ù
Ù
ORDER 1 2 3 4 5 6 7
Quasi realization 0.1109 0.0653 0.0449 0.0263 0.0220 0.0211 0.0210
Positive realization 0.3065 0.1575 0.0690 0.0411 0.0374 0.0373 0.0371
3. REALIZATION
Introduction Realization Quasi realization
Approx. realization Modeling DNA
31
Outline
Matrix factorizations Given matrix Find low
rank approximation of
Identification problem Given output
sequence Find HMM that models the sequence
Realization problem Given string probs Find
HMM generating string probs
Estimation problem Given output sequence Find
state distribution at time
32
Identification problem

• Given Output sequence of length T
• Asked (Quasi) HMM that models the sequence

NONNEGATIVE

• NO NONNEGATIVITY
• CONSTRAINTS!

• Approach

Linear Stochastic Models
Subspace basedidentification
Prediction error identification
SVD
HiddenMarkovModels
Baum-Welch identification
Subspace inspiredidentification
NMF
4. IDENTIFICATION
Introduction Subspace inspired identification
HIV modeling
33
Identification problem
output sequence
system matrices
state sequence
system matrices
state sequence
Baum-Welch
Subspace inspired
4. IDENTIFICATION
Introduction Subspace inspired identification
HIV modeling
34
Subspace inspired identification

• Estimate the (quasi) state distribution

• quasi state predictor can be built from data
  using upperbounded NMF without nonnegativity of
  the factors
• state predictor can be built from data using NMF

We have shown that
. . .
. . .
. . .
. . .
. . .
. . .
. . .

• Compute the system matrices least squares problem

Quasi HMM
Positive HMM
4. IDENTIFICATION
Introduction Subspace inspired identification
HIV modeling
35
Modeling sequences from HIV genome

• Mutation

• HIV virus

ENVELOPE
CORE
MATRIX

• 25 mutated sequences of length
  222 from the part of the HIV1 genome that codes
  for the envelope protein NCBI database
• Training set
• Test set
• HMM model using Baum-Welch Subspace inspired
  identification

4. IDENTIFICATION
Introduction Subspace inspired identification
HIV modeling
36
Modeling sequences from HIV genome

• Kullback-Leibler divergence (string probabilities
  of length-4 strings)

ORDER 1 2 3 4 5
Baum-Welch 3.15 4.65 8.27 21.02 22.93
Subspace 3.15 2.14 1.13 1.08 1.10

• Mean likelihood of the given sequences

ORDER 1 2 3 4 5
Baum-Welch 8.13 10-5 9.03 10-5 1.40 10-4 1.45 10-4 1.50 10-4
Subspace 8.14 10-5 8.84 10-5 9.84 10-5 9.60 10-5 9.83 10-5

• Likelihood of using third
  order subspace inspired model

TEST-SEQUENCE
Likelihood 9.18 10-5 9.15 10-5 9.26 10-5 8.82 10-5 9.15 10-5

• Model can be used to predict new viral strains
  and to distinguish between different HIV
  subtypes

4. IDENTIFICATION
Introduction Subspace inspired identification
HIV modeling
37
Outline
Matrix factorizations Given matrix Find low
rank approximation of
Identification problem Given output
sequence Find HMM that models the sequence
Realization problem Given string probs Find
HMM generating string probs
Estimation problem Given output sequence Find
state distribution at time
38
Estimation for HMMs

• State estimation output estimation

HMM
HMM

• Filtering smoothing prediction

span of available measurements
FILTERING
t
TIME
SMOOTHING
t
TIME
PREDICTION
t
TIME

• We derive recursive formulas to solve state and
  output filtering, prediction and smoothing
  problems

5. ESTIMATION
Estimation for HMMs Application
39
Estimation for HMMs

• Example
• Recursive algorithm to compute

• Recursive output estimation algorithms effective
  with quasi HMM

• Finiteness of rank of Hankel matrix N S
  condition for quasi realizability
• Rank of hankel matrix minimal order of exact
  quasi realization
• Quasi realization is easier to compute than
  positive realization
• Quasi realization typically has lower order than
  positive realization
• Negative probabilities
• No disadvantage in output estimation problems

5. ESTIMATION
Estimation for HMMs Application
40
Finding motifs in DNA sequences

• Find motifs in muscle specific regulatory regions
  Zhou, Wong
• Make motif model
• Make quasi background model (see Section
  realization)
• Build joint HMM
• Perform output estimation
• Results (compared to results from Motifscanner
  Aerts et al.)

MOTIF PROBABILITY
MOTIF PROBABILITY
POSITION
POSITION
5. ESTIMATION
Estimation for HMMs Application
41
Conclusions

• Two modification to the nonnegative matrix
  factorization
• Structured nonnegative matrix factorization
• Nonnegative matrix factorization without
  nonnegativity of the factors
• Two relaxations to the positive realization
  problem for HMMs
• Quasi realization problem
• Approximate positive realization problem
• Both methods were applied to modeling DNA
  sequences
• We derive equivalence conditions for HMMs
• We propose a new identification method for HMMs
• Method was applied to modeling DNA sequences of
  HIV virus
• Quasi realizations suffice for several estimation
  problems
• Quasi estimation methods were applied to finding
  motifs in DNA sequences

6. CONCLUSIONS
Conclusions Further research List of
publications
42
Further research

• Matrix factorizations
• Develop nonnegative matrix factorization with
  nesting property (cfr. SVD)
• Hidden Markov models
• Investigate Markov models (special case of hidden
  Markov case)
• Develop realization and identification methods
  that allow to incorporate prior-knowledge in the
  Markov chain
• Method to estimate minimal order of positive HMM
  from string probabilities
• Canonical forms of hidden Markov models
• Model reduction for hidden Markov models
• System theory for hidden Markov models with
  external inputs

. . .
6. CONCLUSIONS
Conclusions Further research List of
publications
43
List of publications

• Journal papers
• B. Vanluyten, J.C. Willems and B. De Moor.
  Recursive Filtering using Quasi-Realizations.
  Lecture Notes in Control and Information
  Sciences, 341, 367374, 2006.
• B. Vanluyten, J.C. Willems and B. De Moor.
  Equivalence of State Representations for Hidden
  Markov Models. Systems and Control Letters,
  57(5), 410419, 2008.
• B. Vanluyten, J.C. Willems and B. De Moor.
  Structured Nonnegative Matrix Factorization with
  Applications to Hidden Markov Realization and
  Filtering. Accepted for publication in Linear
  Algebra and its Applications, 2008.
• B. Vanluyten, J.C. Willems and B. De Moor.
  Nonnegative Matrix Factorization without
  Nonnegativity Constraints on the Factors.
  Submitted for publication.
• B. Vanluyten, J.C. Willems and B. De Moor.
  Approximate Realization and Estimation for Quasi
  hidden Markov models. Submitted for publication.
• International conference papers
• I. Goethals, B. Vanluyten, B. De Moor. Reliable
  spurious mode rejection using self learning
  algorithms. In Proc. of the International
  Conference on Modal Analysis Noise and Vibration
  Engineering (ISMA 2004), Leuven, Belgium, pages
  9911003, 2004.
• B. Vanluyten, J. C.Willems and B. De Moor. Model
  Reduction of Systems with Symmetries. In Proc. of
  the 44th IEEE Conference on Decision and Control
  (CDC 2005), Seville, Spain, pages 826831, 2005.
• B. Vanluyten, J. C. Willems and B. De Moor.
  Matrix Factorization and Stochastic State
  Representations. In Proc. of the 45th IEEE
  Conference on Decision and Control (CDC 2006),
  San Diego, California, pages 4188-4193, 2006.
• I. Markovsky, J. Boets, B. Vanluyten, K. De Cock,
  B. De Moor. When is a pole spurious? In Proc. of
  the International Conference on Noise and
  Vibration Engineering (ISMA 2007), Leuven,
  Belgium, pp. 16151626, 2007.
• B. Vanluyten, J. C. Willems and B. De Moor.
  Equivalence of State Representations for Hidden
  Markov Models. In Proc. of the European Control
  Conference 2007 (ECC 2007), Kos, Greece, 2007.
• B. Vanluyten, J. C. Willems and B. De Moor. A new
  Approach for the Identification of Hidden Markov
  Models. In Proc. of the 46th IEEE Conference on
  Decision and Control (CDC 2006), New Orleans,
  Louisiana, 2007.

6. CONCLUSIONS
Conclusions Further research List of
publications
44
(No Transcript)