Nonnegative Matrix Factorization via Rank-one Downdate

1
Nonnegative Matrix Factorization via Rank-one
Downdate

• Ali Ghodsi
• Department of Statistics and Actuarial Science
• David R. Cheriton School of Computer Science
• University of Waterloo
• Joint work with
• Stephen Vavasis and Michael Biggs
• University of Waterloo

2
Nonnegative Matrix Factorization
3
(No Transcript)
4
2 by 1965
560 by 1965
560 by 2

20 by 28
2 by 1
2 by 1
20 by 28
5
Singular Value Decomposition (SVD)
6
History
7
History
8
History
9
History
10
History (Algorithms)
11
History (Algorithms)
12
First observation
13
Power method

• Computes the leading singular vectors/value (or
  eigenvector/value) of a matrix

1 2 while not converged
3
4
5 6 end
14
Naive approach to NMF using this observation
1 2 3 4 5 for all set 6 end for

• Without step 5, this will simply compute the SVD
  (Jordan's algorithm, Camille Jordan 1874. )

15
Rank-one Downdata (R1D)
16
Objective function
17
ApproxRankOneSubmatrix(A)
18
Modified power iteration Demo
Rank-1 submatrix
A
Rank-1 submatrix
19
Modified power iteration Demo
v
0.14 0.07 0.64 0.41 0.55
Rank-1 submatrix
Rank-1 submatrix
20
Modified power iteration Demo
v
0.0 0.0 0.64 0.41 0.55
Rank-1 submatrix
Rank-1 submatrix
21
Modified power iteration Demo
v
0.0 0.0 0.64 0.41 0.55
Rank-1 submatrix
Rank-1 submatrix
22
Modified power iteration Demo
v
0.0 0.0 0.64 0.41 0.55
Rank-1 submatrix
0.16 0.21 0.22 0.44 0.74 0.20

• u

Rank-1 submatrix
23
Modified power iteration Demo
v
0.0 0.0 0.64 0.41 0.55
Rank-1 submatrix
0.0 0.0 0.0 0.44 0.74 0.20

• u

Rank-1 submatrix
24
Modified power iteration Demo
v
0.0 0.0 0.64 0.41 0.55
Rank-1 submatrix
0.0 0.0 0.0 0.44 0.74 0.20

• u

Rank-1 submatrix
25
Modified power iteration Demo
v
0.0 0.0 0.60 0.28 0.59
Rank-1 submatrix
0.0 0.0 0.0 0.44 0.74 0.20

• u

Rank-1 submatrix
26
Modified power iteration Demo
v
0.0 0.0 0.60 0.28 0.59
Rank-1 submatrix
0.0 0.0 0.0 0.44 0.74 0.20

• u

Rank-1 submatrix
27
Modified power iteration Demo
v
0.0 0.0 0.60 0.28 0.59
Rank-1 submatrix
0.0 0.0 0.0 0.44 0.74 0.20

• u

Rank-1 submatrix
Zero-out!
28
Modified power iteration Demo
Rank-1 submatrix
Anew
29
Rank-one Downdata (R1D)
30
A simple model for text
31
Generating a corpus in the model
32
Theorem about text
33
LSI
34
R1D
35
Theorem about images
36
Experimental results
37
LSI
38
NMF-DIV
39
R1D
40
LSI
41
NMF_DIV
42
R1D
43

• Thank you!