Advances in Random Matrix Theory (stochastic eigenanalysis)

1
Advances in Random Matrix Theory(stochastic
eigenanalysis)

• Alan Edelman
• MIT Dept of Mathematics,
• Computer Science AI Laboratories

2
Stochastic Eigenanalysis

• Counterpart to stochastic differential equations
• Emphasis on applications to engineering finance
• Beautiful mathematics
• Random Matrix Theory
• Free Probability
• Raw Material from
• Physics
• Combinatorics
• Numerical Linear Algebra
• Multivariate Statistics

3
Scalars, Vectors, Matrices

• Mathematics Notation power less ink!
• Computation Use those caches!
• Statistics Classical, Multivariate, ?
• Modern Random
  Matrix Theory
• The Stochastic Eigenproblem
• Mathematics of probabilistic
  linear algebra
• Emerging Computational
  Algorithms
• Emerging Statistical Techniques

Ideas from numerical computation that stand the
test of time are right for mathematics!
4
Open Questions

• Find new applications of spacing (or other)
  statistics
• Cleanest derivation of Tracy-Widom?
• Finite free probability?
• Finite meets infinite
• Muirhead meets Tracy-Widom
• Software for stochastic eigen-analysis

5
Wigners Semi-Circle

• The classical most famous rand eig theorem
• Let S random symmetric Gaussian
• MATLAB Arandn(n) S( AA)/2
• S known as the Hermite Ensemble
• Normalized eigenvalue histogram is a semi-circle
• Precise statements require n?? etc.

6
Wigners Semi-Circle

• The classical most famous rand eig theorem
• Let S random symmetric Gaussian
• MATLAB Arandn(n) S( AA)/2
• S known as the Hermite Ensemble
• Normalized eigenvalue histogram is a semi-circle
• Precise statements require n?? etc.

n x n iid standard normals
7
Wigners Semi-Circle

• The classical most famous rand eig theorem
• Let S random symmetric Gaussian
• MATLAB Arandn(n) S( AA)/2
• S known as the Hermite Ensemble
• Normalized eigenvalue histogram is a semi-circle
• Precise statements require n?? etc.

8
Wigners original proof

• Compute E(tr A2p) as n?8
• Terms with too many indices, have some element
  with power 1. Vanishes with mean 0.
• Terms with too few indices not enough to be
  relevant as n?8
• Leaves only a Catalan number left Cp(2p)/(p1)
  for the moments when all is said and done
• Semi-circle only distribution with Catalan number
  moments

p
9
Finite Versions of semicircle

• n2 n4
• n3 n5

10
Finite Versions

• n2 n4
• n3 n5

Area under curve (-8,x) Can be expressed as sums
of probabilities that certain tridiagonal
determinants are positive.
11
Wigners Semi-Circle

• Real Numbers x ß1
• Complex Numbers xiy ß2
• Quaternions xiyjzkw ß4
• ß2½? xiyjz ß2½?

Defined through joint eigenvalue density
const x ?xi-xjß ?exp(-xi2 /2) ßrepulsion
strength ß0 no interference spacings are
Poisson Classical research only ß1,2,4 missing
the link to Poisson, continuous techniques, etc
12
Largest eigenvalue
convection-diffusion?
13
Haar or not Haar?
Uniform Distribution on orthogonal
matrices Gram-Schmidt or Q,RQR(randn(n))
14
Haar or not Haar?
Uniform Distribution on orthogonal
matrices Gram-Schmidt or Q,RQR(randn(n))
?
Eigenvalues Wrong
15
Longest Increasing Subsequence(n4)
(Baik-Deift-Johansson) (Okounkovs proof)
Green 4 Yellow 3 Red 2 Purple 1
1 2 3 4 2 1 3 4 3 1 2 4 4 1 2 3
1 2 4 3 2 1 4 3 3 1 4 2 4 1 3 2
1 3 2 4 2 3 1 4 3 2 1 4 4 2 1 3
1 3 4 2 2 3 4 1 3 2 4 1 4 2 3 1
1 4 2 3 2 4 1 3 3 4 1 2 4 3 1 2
1 4 3 2 2 4 3 1 3 4 2 1 4 3 2 1
16
Bulk spacing statistics
convection-diffusion?

• Bus wait times in Mexico
• Energy levels of heavy atoms
• Parked Cars in London
• Zeros of Riemann zeta
• Mice Brain Wave Spikes

Telltale Sign Repulsion optimality
17
whats my ß?web page

• Cys tricks
• Maximum Likelihood Estimation
• Bayesian Probability
• Kernel Density Estimation
• Epanechnikov kernel
• Confidence Intervals

http//people.csail.mit.edu/cychan/BetaEstimator.h
tml
18
Open Questions

• Find new applications of spacing (or other)
  distributions
• Cleanest derivation of Tracy-Widom?
• Finite free probability?
• Finite meets infinite
• Muirhead meets Tracy-Widom
• Software for stochastic eigen-analysis

19
Everyones Favorite Tridiagonal
-2 1
1 -2 1

1
1 -2





20
Everyones Favorite Tridiagonal
-2 1
1 -2 1

1
1 -2
G
G


G
1 (ßn)1/2







21
Stochastic Operator Limit



Cast of characters Dumitriu, Sutton, Rider
22
Open Questions

• Find new applications of spacing (or other)
  distributions
• Cleanest derivation of Tracy-Widom?
• Finite free probability?
• Finite meets infinite
• Muirhead meets Tracy-Widom
• Software for stochastic eigen-analysis

23
Is it really the random matrices?

• The excitement is that the random matrix
  statistics are everyhwere
• Random matrices properly tridiagonalized are
  discretizations of stochastic differential
  operators!
• Eigenvalues of SDOs not as well studied
• Deep down this is what I believe is the important
  mechanism in the spacings, not the random
  matrices! (See Brian Sutton thesis, Brian Rider
  papersconnection to Schrodinger operators)
• Deep down for other statistics, though its the
  matrices

24
Open Questions

• Find new applications of spacing (or other)
  distributions
• Cleanest derivation of Tracy-Widom?
• Finite free probability?
• Finite meets infinite
• Muirhead meets Tracy-Widom
• Software for stochastic eigen-analysis

25
From Stochastic Differential Operators to Sturm
Sequences

• Recent results (Rider and Ramirez) have shown
  that we can recast the stochastic eigenvalue
  problem as a diffusion process governed by a 1-D
  Schrödinger equation
• In the language of the diffusion process
• If the eigenfunction of the operator has a k
  roots when shifted by ?, we know there are k
  eigenvalues greater than ?
• The equivalent statement in the language of Sturm
  sequences is
• If there are k roots (sign changes) in the
  continuous limit of the Sturm sequence of a ?
  shifted matrix, we know there are k eigenvalues
  less than ?

26
Open Questions

• Find new applications of spacing (or other)
  distributions
• Cleanest derivation of Tracy-Widom?
• Finite free probability?
• Finite meets infinite
• Muirhead meets Tracy-Widom
• Software for stochastic eigen-analysis

27
Free Probability

• Free Probability (name refers to free algebras
  meaning no strings attached)
• Gets us past Gaussian ensembles and Wishart
  Matrices

28
The flipping coins example

• Classical Probability Coin 1 or -1 with p.5

50
50
50
50
y
x
-1 1
-1 1
xy
-2 0
2
29
The flipping coins example

• Classical Probability Coin 1 or -1 with p.5

Free
50
50
50
50
eig(B)
eig(A)
-1 1
-1 1
eig(AQBQ)
-2 0
2
30
From Finite to Infinite
31
From Finite to Infinite
? Gaussian (m1)
32
From Finite to Infinite
? Gaussian (m1)
Wiggly
33
From Finite to Infinite
? Gaussian (m1)
Wiggly
Wigner?
34
Semi-circle law for different betas
35
Open Questions

• Find new applications of spacing (or other)
  distributions
• Cleanest derivation of Tracy-Widom?
• Finite free probability?
• Finite meets infinite
• Muirhead meets Tracy-Widom
• Software for stochastic eigen-analysis

36
Matrix Statistics

• Many Worked out in 1950s and 1960s
• Muirhead Aspects of Multivariate Statistics
• Are two covariance matrices equal?
• Does my matrix equal this matrix?
• Is my matrix a multiple of the identity?
• Answers Require Computation of
• Hypergeometrics of Matrix Argument
• Long thought Computationally Intractible

37
The special functions of multivariate statistics

• Hypergeometric Functions of Matrix Argument
• ß2 Schur Polynomials
• Other values Jack Polynomials
• Orthogonal Polynomials of Matrix Argument
• Begin with w(x) on I
• ? p?(x)p?(x) ?(x)ß ?i w(xi)dxi d??
• Jack Polynomials orthogonal for w1 on the unit
  circle. Analogs of xm
• Plamen Koev revolutionary computation
• Dumitrius MOPS symbolic package

38
Multivariate Orthogonal PolynomialsHypergeometr
ics of Matrix Argument

• The important special functions of the 21st
  century
• Begin with w(x) on I
• ? p?(x)p?(x) ?(x)ß ?i w(xi)dxi d??
• Jack Polynomials orthogonal for w1 on the unit
  circle. Analogs of xm

39
Smallest eigenvalue statistics
Arandn(m,n) hist(min(svd(A).2))
40
Multivariate Hypergeometric Functions
41
Multivariate Hypergeometric Functions
42
Open Questions

• Find new applications of spacing (or other)
  distributions
• Cleanest derivation of Tracy-Widom?
• Finite free probability?
• Finite meets infinite
• Muirhead meets Tracy-Widom
• Software for stochastic eigen-analysis

43
Plamen Koevs clever idea
44
Symbolic MOPS applications
Arandn(n) S(AA)/2 trace(S4)
det(S3)
45
Mops (Ioana Dumitriu) Symbolic
46
Random Matrix Calculator
47
Encoding the semicircleThe algebraic secret

• f(x) sqrt(4-x2)/(2p)
• m(z) (-z isqrt(4-z2))/2
• L(m,z) m2zm10
• m(z) ? (x-z)-1f(x) dx Stieltjes transform

Practical encoding Polynomial L whose root m
is Stieltjes transform
48
The Polynomial Method

• RMTool
• http//arxiv.org/abs/math/0601389
• The polynomial method for random matrices
• Eigenvectors as well!

49
Plus

X randn(n,n) AXX m2zm10
Yrandn(n,2n) BYY zm2(2z-1)m20
AB m3(z2)m2(2z-1)m20
50
Times

X randn(n,n) AXX m2zm10
Yrandn(n,2n) BYY zm2(2z-1)m20
AB m4z2-2m3zm24mz40
51
Open Questions

• Find new applications of spacing (or other)
  distributions
• Cleanest derivation of Tracy-Widom?
• Finite free probability?
• Finite meets infinite
• Muirhead meets Tracy-Widom
• Software for stochastic eigen-analysis

52
Matrix Versions of Classical Stats
Orthog Matrix MATLAB (Arandn(n)
Brandn(n))
Hermite Sym Eig eig(AA) Normal
Laguerre SVD eig(AA) Chi-squared
Jacobi GSVD gsvd(A,B) Beta
Fourier Eig U,Rqr(AiB)
53
The big structure
Orthog Matrix Weight Stats
Graph Theory SymSpace
Hermite Sym Eig exp(-x2) Normal Complete Graph non-compact A,AI,AII
Laguerre SVD xae-x Chi-squared Bipartite Graph non-compact AIII,BDI,CII
Jacobi GSVD (1-x)a x (1x)ß Beta Regular Graph compact A, AI, AII, C, D, CI, D, DIII
Fourier Eig ei? compact AIII, BDI, CDI
54
Summary

• Stochastic Eigenanalysis
• Emerging Techniques
• Open Problems