Why are random matrix eigenvalues cool?

1
Why are random matrix eigenvalues cool?

• Alan Edelman
• MIT Dept of Mathematics,
• Lab for Computer Science
• MAA Mathfest 2002
• Thursday, August 1

2
Message

• Ingredient Take Any important mathematics
• Then Randomize!
• This will have many applications!

3
Some fun tidbits

• The circular law
• The semi-circular law
• Infinite vs finite
• How many are real?
• Stochastic Numerical Algorithms
• Condition Numbers
• Small networks
• Riemann Zeta Function
• Matrix Jacobians

4
Girkos Circular Law, n2000
Has anyone studied spacings?
5
Wigners Semi-Circle

• The classical most famous rand eig theorem
• Let S random symmetric Gaussian
• MATLAB Arandn(n) S(AA)/2
• Normalized eigenvalue histogram is a semi-circle
• Precise statements require n?? etc.

6
Wigners Semi-Circle

• The classical most famous rand eig theorem
• Let S random symmetric Gaussian
• MATLAB Arandn(n) S(AA)/2
• Normalized eigenvalue histogram is a semi-circle
• Precise statements require n?? etc.

n20 s30000 d.05 matrix size, samples,
sample dist e gather up
eigenvalues im1 imaginary(1) or
real(0) for i1s, arandn(n)imsqrt(-1)randn(
n)a(aa')/(2sqrt(2n(im1))) veig(a)'
ee v end hold off m xhist(e,-1.5d1.5)
bar(x,mpi/(2dns)) axis('square') axis(-1.5
1.5 -1 2) hold on t-1.011
plot(t,sqrt(1-t.2),'r')
7
Elements of Wigners Proof

• Compute E(A2k)11 mean(?2k) (2k)th moment
• Verify that the semicircle is the only
  distribution with these moments
• (A2k)11 ?A1xAxyAwzAz1 paths of length 2k
• Need only count number of special paths of length
  2k on k objects (all other terms 0 or
  negligible!)
• This is a Catalan Number!

8
Catalan Numbers

• ways to parenthesize (n1)
  objects
• Matrix Power Term Graph
• (1((23)4)) A12A23A32A24A42A21
• (((12)3)4) A12A21A13A31A14A41
• (1(2(34))) A12A23A34A43A32A21
• ((12)(34)) A12A21A13A34A43A31
• ((1(23))4) A12A23A32A21A14A41
• number of special paths on n departing from 1
  once
• Pass 1, (loadadvance, multiplyretreat), Return
  to 1

9
Finite Versions

• n2 n4
• n3 n5

10
How many eigenvalues of a random matrix are real?
gtgt eeig(randn(7)) e 1.9771
1.3442 0.6316 -1.1664
1.3504i -1.1664 - 1.3504i -2.1461 0.7288i
-2.1461 - 0.7288i
gtgt eeig(randn(7)) e -2.0767 1.1992i
-2.0767 - 1.1992i 2.9437 0.0234
0.4845i 0.0234 - 0.4845i 1.1914 0.3629i
1.1914 - 0.3629i
gtgt eeig(randn(7)) e -2.1633
-0.9264 -0. 3283 2.5242 1.6230
0.9011i 1.6230 - 0.9011i 0.5467
3 real 1 real
5 real
7x7 random Gaussian
11
How many eigenvalues of a random matrix are real?
n7
0.00069 0.08460 0.57727 0.33744
7 reals 5 reals 3 reals 1 real
12
How many eigenvalues of a random matrix are real?
n7
0.00069 0.08460 0.57727 0.33744
7 reals 5 reals 3 reals 1 real
These are exact but hard to compute! New research
suggests a Jack polynomial solution.
13
How many eigenvalues of a random matrix are real?

• The Probability that a matrix has all real
  eigenvalues is exactly
• Pn,n2-n(n-1)/4
• Proof based on Schur Form

14
Gram Schmidt (or QR) Stochastically

• Gram Schmidt
• Orthogonal Transformations to Upper
  Triangular Form
• A Q R (orthog upper triangular)

15
Orthogonal Invariance of Gaussians
G
G
G
G
G
G
G
Qrandn(n,1) ? randn(n,1) If Q orthogonal
Q?
16
Orthogonal Invariance
G
G
G
G
G
G
G
G
G
G
G
G
G
G
Qrandn(n,1) ? randn(n,1) If Q orthogonal
Q?
17
Chi Distribution
G
G
G
G
G
G
G
norm(randn(n,1)) ? ?n
?n
18
Chi Distribution
G
G
G
G
G
G
G
norm(randn(n,1)) ? ?n
?n
19
Chi Distribution
G
G
G
G
G
G
G
norm(randn(n,1)) ? ?n
?n
n need not be integer
20
G G G G G G G
G G G G G G G
G G G G G G G
G G G G G G G
G G G G G G G
G G G G G G G
G G G G G G G
21
G G G G G G G
G G G G G G G
G G G G G G G
G G G G G G G
G G G G G G G
G G G G G G G
G G G G G G G
22
?7 G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
23
?7 G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
24
?7 G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
25
?7 G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
26
?7 G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
27
?7 G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
28
?7 G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
29
?7 G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
30
?7 G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
31
?7 G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
O G G G G G G
32
?7 G G G G G G
O ?6 G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
33
?7 G G G G G G
O ?6 G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
34
?7 G G G G G G
O ?6 G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
35
?7 G G G G G G
O ?6 G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
36
?7 G G G G G G
O ?6 G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
37
?7 G G G G G G
O ?6 G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
38
?7 G G G G G G
O ?6 G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
39
?7 G G G G G G
O ?6 G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
40
?7 G G G G G G
O ?6 G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
O O G G G G G
41
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O G G G G
O O O G G G G
O O O G G G G
O O O G G G G
42
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O G G G G
O O O G G G G
O O O G G G G
O O O G G G G
43
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O G G G G
O O O G G G G
O O O G G G G
O O O G G G G
44
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O G G G G
O O O G G G G
O O O G G G G
O O O G G G G
45
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O G G G G
O O O G G G G
O O O G G G G
O O O G G G G
46
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O G G G G
O O O G G G G
O O O G G G G
O O O G G G G
47
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O G G G G
O O O G G G G
O O O G G G G
O O O G G G G
48
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O G G G G
O O O G G G G
O O O G G G G
O O O G G G G
49
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O G G G
O O O O G G G
O O O O G G G
50
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O G G G
O O O O G G G
O O O O G G G
51
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O G G G
O O O O G G G
O O O O G G G
52
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O G G G
O O O O G G G
O O O O G G G
53
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O G G G
O O O O G G G
O O O O G G G
54
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O G G G
O O O O G G G
O O O O G G G
55
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O G G G
O O O O G G G
O O O O G G G
56
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O G G
O O O O O G G
57
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O G G
O O O O O G G
58
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O G G
O O O O O G G
59
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O G G
O O O O O G G
60
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O G G
O O O O O G G
61
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O G G
O O O O O G G
62
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O ?2 G
O O O O O O G
63
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O ?2 G
O O O O O O G
64
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O ?2 G
O O O O O O G
65
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O ?2 G
O O O O O O G
66
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O ?2 G
O O O O O O G
67
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O ?2 G
O O O O O O ?1
68
?7 G G G G G G
O ?6 G G G G G
O O ?5 G G G G
O O O ?4 G G G
O O O O ?3 G G
O O O O O ?2 G
O O O O O O ?1
69
Same idea sym matrix to tridiagonal form
G ?6
?6 G ?5
?5 G ?4
?4 G ?3
?3 G ?2
?2 G ?1
?1 G
Same eigenvalue distribution as GOE O(n)
storage !! O(n) computation (potentially)
70
Same idea General beta
G ?6?
?6? G ?5?
?5? G ?4?
?4? G ?3?
?3? G ?2?
?2? G ??
?? G
beta 1 reals 2 complexes 4 quaternions
Bidiagonal Version corresponds To Wishart
matrices of Statistics
71
Numerical Analysis Condition Numbers

• ?(A) condition number of A
• If AU?V is the svd, then ?(A) ?max/?min .
• Alternatively, ?(A) ?? max (AA)/?? min (AA)
• One number that measures digits lost in finite
  precision and general matrix badness
• Smallgood
• Largebad
• The condition of a random matrix???

72
Von Neumann co.

• Solve Axb via x (AA) -1A b
• M ?A-1
• Matrix Residual AM-I2
• AM-I2lt 200?2 n ?
• How should we estimate ??
• Assume, as a model, that the elements of A are
  independent standard normals!

?
73
Von Neumann co. estimates (1947-1951)

• For a random matrix of order n the expectation
  value has been shown to be about n
• Goldstine, von Neumann
• we choose two different values of ?, namely n
  and ?10n
• Bargmann, Montgomery, vN
• With a probability 1 ? lt 10n
• Goldstine, von Neumann

X ?
74
Random cond numbers, n??
Distribution of ?/n
Experiment with n200
75
Finite n

• n10
  n25
• n50
  n100

76
Small World Networks 6 degrees of separation

• Edelman, Eriksson, Strang
• Eigenvalues of ATPTP, Prandperm(n)
• Incidence
  matrix of graph with two
• superimposed cycles.

77
Small World Networks 6 degrees of separation

• Edelman, Eriksson, Strang
• Eigenvalues of ATPTP, Prandperm(n)
• Incidence
  matrix of graph with two
• superimposed cycles.
• Wigner style derivation counts number of paths on
  a tree starting and ending at the same point
  (tree no accidents!) (McKay)
• We first discovered the formula using the
  superseeker
• Catalan number answer d2n-1-? d2j-1(d-1)n-j1Cn-j

78
The Riemann Zeta Function
On the real line with xgt1, for example
May be analytically extended to the complex
plane, with singularity only at x1.
79
The Riemann Hypothesis
-3 -2 -1 0 ½ 1
2 3
All nontrivial roots of ?(x) satisfy
Re(x)1/2. (Trivial roots at negative even
integers.)
80
The Riemann Hypothesis
Zeros .5i 14.134725142 21.022039639
25.010857580 30.424876126 32.935061588
37.586178159 40.918719012 43.327073281
48.005150881 49.773832478 52.970321478
56.446247697 59.347044003
?(x) along Re(x)1/2
-3 -2 -1 0 ½ 1
2 3
All nontrivial roots of ?(x) satisfy
Re(x)1/2. (Trivial roots at negative even
integers.)
81
Computation of Zeros

• Odlyzkos fantastic computation of 10k1
  through 10k10,000 for k12,21,22.
• See http//www.research.att.com/amo/zeta_tables/
• Spacings behave like the eigenvalues of
• Arandn(n)irandn(n) S(AA)/2

82
Nearest Neighbor Spacings Pairwise Correlation
Functions
83
Painlevé Equations
84
Spacings

• Take a large collection of consecutive
  zeros/eigenvalues.
• Normalize so that average spacing 1.
• Spacing Function Histogram of consecutive
  differences (the (k1)st the kth)
• Pairwise Correlation Function Histogram of all
  possible differences (the kth the jth)
• Conjecture These functions are the same for
  random matrices and Riemann zeta

85
Some fun tidbits

• The circular law
• The semi-circular law
• Infinite vs finite
• How many are real?
• Stochastic Numerical Algorithms
• Condition Numbers
• Small networks
• Riemann Zeta Function
• Matrix Jacobians

86
Matrix Factorization Jacobians
General
ALU AU?VT AX?X-1
AQR AQS (polar)
? uiin-i
? riim-i
? (?i2- ?j2)
? (?i?j)
? (?i-?j)2
Sym
Orthogonal
?sin(?i ?j)sin (?i- ?j)
Tridiagonal
TQ?QT
? (ti1,i)/ ?qi
87
Why cool?

• Why is numerical linear algebra cool?
• Mixture of theory and applications
• Touches many topics
• Easy to jump in to, but can spend a lifetime
  studying researching
• Tons of activity in many areas
• Mathematics Combinatorics, Harmonic Analysis,
  Integral Equations, Probability, Number Theory
• Applied Math Chaotic Systems, Statistical
  Mechanics, Communications Theory, Radar Tracking,
  Nuclear Physics
• Applications
• BIG HUGE SUBJECT!!