Vine Review: High Dimensional Dependence Modeling and Model Learning

1
Vine ReviewHigh Dimensional Dependence Modeling
andModel Learning

• Roger M. Cooke, Dorota Kurowicka, Tim Bedford
• Resources for the Future Dept Math TU Delft
  Univ. Strathclyde
• Nov 19, 2007

2
Outline

• Correlations copulae
• Joints from bivariates pieces trees vines
• Sampling
• Model Inference
• Bayesian Belief Nets

3
Correlations and Copulae
4
Correlations

5

6

7
Copula
X,Y with cdf s FX, FY are joined by copula C if
joint can be written
FXY(,y) C(FX(x), FY(y))
8
Diagonal Band Copula
density of diagonal band copula with
correlation
?0.8
MI(f) -ln(21-ß ß)
generalized diagonal band (Bajorski 01)
9
Elliptical Copula
MI(f) 1ln(2) ln(pv(1-?2))
10
Franks copula
Normal copula
11
Joints from Bivariate PiecesMarkov
TreesandRegular Vines
12
Markov Trees
r35
Sampling Us U0,1, independent, Xs
U0,1
(F,T, B) is a Bivariate tree Specification if F
is a set of univatiate cdfs, T is a Tree with
edge set E, and B is a consistent assignment of
bivariate distributions to the members of E.
13
Hammersley-Clifford theorem (Besag( JRSS 74))

14
Regular Vine System of Bivariate Constraints

• Regular vine (conditional) copula gt sampling
  algorithm

• Constraints on edges ? ?
• ? doubleton
• Every pair occurs once as ?,
• Copulae can be chosen arbitrarily, typically
  indexed by (rank) correlation
• Setting ij K Independent gt max. Inf
  realization given rest

4
34
23
12
453
1
2
3
35
132
253
5
1523
2435
14235
15
Non Regular Vine
4

34
12
23
453
1
2
3
35
1235
253
5
1235
2435
14235
16
(No Transcript)
17
Regular Vine Conditional Copulae
18
Mutual Information Decomposition
Definition A Regular Vine Specification is a set
(F,V, B) where F is a set of univariate cdfs, V
is a regular vine, and B is a set of
(conditional) bivariate distributions
corresponding to the edges of V .

19
(No Transcript)
20

21

• Sampling

22
D-Vine
Sampling algorithm U1U4 independent Uniform
0,1 X1X4 Uniform 0,1
23
Canonical Vine
24
X3 x1
x2

X1 U1
U2
X2
X3
U3
25

• Model Inference

26

• Why where R is multiple correlation
• D (1-R212..n)(1-R223..n)...(1-R2n-1n).
• Where ? is partial correlation
• (1-R212..n) (1- ?2123..n)(1-
  ?2134..n)...(1- ?21n)
• C / C11

27

• D(1-R241235)(1-R25123)(1-R2312)(1-R221)
• (1-R241235) (1- ?241235)(1- ?24235)(1-
  ?2453)(1- ? 43)
• (1-R25123) (1- ?25123)(1- ?2523)(1- ?253)
• (1-R2312) (1- ?2132)(1- ?232)
• (1-R221) 1- ?212

28
For Joint normal, MI(f) -½ln(D) D
Det(Corr.Mtrx)

• Definition (Multiple infomation) The multiple
  information of X1 w.r.t. X2...Xn is
• MI12n I( f f1 f2n )

29

30
Idea of Model Inference

• Find a regular vine such that the terms
• bijK(ij)
• are as spread out as possible. Eg decompose
  the sum MI(f) in terms which are very large or
  very small.

31
Majorization Schur convexity

32
Find right model capture most dependence in
fewest vbls Find the Vine whose MI majorizes

33
Example German weather stations
Compare model inference method of
Speed/Whittaker and Vine inference

34
Speed / Whittaker -log(D) 11.4406
Independence graph, 11 interactions log(D)
10.7763
Vine interaction graph 11 interactions log(D)
11.0970
35
Optimal Vine

36
Optimal Vine Partial Correlations
Graphics problem cant leave out zero edges!!!
37

• Bayesian Belief Nets

38
BBN directed acyclic graph specifying joint
distribution via conditional densities

f(1,2,8) (in general) f(3)f(53)f(13,5)f(23,
5,1) f(43,5,1,2)f(63,5,1,2,4)f(73,5,1,2,4,6)f(8
3,5,1,2,4,6,7) (this bbn) f(3)f(5)f(15)f(23)
f(43)f(62)f(76,2,5)f(84,5,6,7)
39
Causal Model Schiphol Airport

40
detail

41
Bayesian Belief Nets

These graphs describe the same distribution
42
Sampling order 1, 2, 3, 4 Factorization P(1)P(2
1)P(321)P(4321)
43
Nodes can be added (and deleted) without
reassessing
r52
1
3
2
5
r45234
4
44
Arcs of bbn not always edges of a vine

1
3
5
4
2
1
3
2
4
5
Cond. Corr from bbn Cond.independent Bivariate
distn must be calculated
45
BBN model inference

• Same idea as regular vine
• D ?arcs (ij) of BBN (1 ?2ijK(ij) )
• Heuristics
• Conditional rank correlation partial rank
  correlation
• Joint normal copula
• Determinant instead of Mutual Information

46

47
Refrences Cont