Algebraic Tools for Analyzing Preconditioners

1

• Algebraic Tools for Analyzing Preconditioners
• Bruce Hendrickson
• Erik Boman
• Sandia National Labs

2
Long History

• Many have worked on algebraic analysis methods
• Abridged history of this line of work
• Beauwens, Notay, Axelsson others (80s)
• Vaidya (91)
• Miller, Gremban, Guattery (mid 90s)
• Gilbert, Boman, Toledo, Chen, H., etc. (current)

3
Outline

• Definitions and concepts
• Basic tools
• Rectangular factorizations
• Special cases
• Vaidyas spanning trees
• Recent extensions
• Approximate inverse preconditioning

4
Starting PointPreconditioned CG

• Solving system Axf with preconditioner B
• For now, focus on symmetric positive definite
  matrices
• General systems later in the talk
• Iterations of preconditioned CG bounded by
  spectral condition number of matrix pencil

5
Support Number

• Support number s(A,B)
• s(A,B) min t xT(t B-A)x ? 0 ?x
• Closely related to largest eigenvalue
• lmax(A,B) s(A,B)
• lmin(A,B) 1/ lmax(B,A) 1/ s(B,A)
• (equality if full rank)
• So, k2(A,B) s(A,B) s(B,A)

6
Why Support Numbers?

• s(A,B) is largest generalized eigenvalue
  projected onto range space of B
• Equals lmax(A,B) when B full rank
• Remains well defined when B rank deficient
• Robust extension of largest eigenvalue
• Easier to work with s(A,B) than with l(A,B)

7
Properties ofSupport Numbers

• Splitting Lemma If A åi1,k Ai and Båi1,k
  Bi
• Then s(A,B) maxi s(Ai,Bi)
• Used in finite elements and domain decomposition
• Our interest is algebraic
• How to split?
• Triangle Inequality If B, C positive
  semidefinite
• Then s(A,C) s(A,B)s(B,C)
• When A, B are psd, then s(A,B) 1/1 - s(A-B,
  A)
• Useful for analysis of incomplete Cholesky

8
More Properties ofSupport Numbers

• Symmetric Product Support If AUUT and BVVT
• Then s(A,B) minW W22 such that VWU
• Special case
• s(uuT,VVT) minw wTw, where Vwu
• (Many more properties in our papers)

9
M-Matrices, Graphs Rectangular Factorizations

• Consider a simple Laplacian

10
Rectangular Factorizations

• A UUT, where U is arbitrary
• Interesting special cases
• Columns of U have 1 nonzero
• Non-negative diagonal matrices
• columns with 2 nonzeros, same magnitude,
  opposite sign
• Symmetric, diagonally dominant M-matrices
• columns with 2 nonzeros, same magnitude
• Symmetric, diagonally dominant matrices
• columns with 2 nonzeros
• Symmetric H-matrices with non-negative diagonal

11
Rectangular Factorizations and Finite Elements

• Factor each element matrix before assembly
• A UUT, where U rectangular, and few nonzeros per
  column
• Block column of U for each element
• Natural Factor Argyris Bronlund

12
Very Special CaseVaidyas Spanning Trees

• Let u and columns of V look like
• a(1, 0, , 0, -1)T, or b(, 0, 1, 0, )T
• Matrices representable as UUT
• Symmetric, diagonally-dominant, M-matrices
• Note vectors with 2 nonzeros can be thought of
  as edges in a (weighted) graph

13
Support Paths

• u Vw, s(uuT,VVT) wTw
• Support number length-of-path

14
Vaidyas Spanning Tree Preconditioners

• Vaidya used this to analyze preconditioners built
  from max-weight spanning trees
• Using support-path analysis, easy to show that
• Worst case condition number O(nm)
• n matrix size, m number of nonzeros
• Exact factorization of an incomplete matrix.
  (J. Gilbert)

15
Matrix Interpretation

• Given rectangular U (mltn)
• Find subset of columns V that makes a good basis
• That is, VWU, where W has small 2-norm
• Vaidyas max-weight spanning tree ensures
• Entries of V-1U are all of magnitude no more than
  1
• O(nm) condition number follows

16
Vaidyas Augmentation

• Can add edges in special way
• Reduce condition number, but increase
  factorization cost
• O(n1.75) runtime for solving general problems
• O(n1.2) runtime to solve for planar graphs
• Bounds independent of sparsity pattern
  numerical values!

17
How does Vaidya work in practice?

• Chen Toledo, ETNA 2003
• Sensitive to structure, not numerical values
• Competitive with relaxed Modified Incomplete
  Cholesky on 2D problems
• Sometimes worse, sometimes much better on 3D
  problems
• Interesting convergence behavior

18
Vaidya on Easy Problem
19
Vaidya on Harder Problem
20
Beyond VaidyaOther Spanning Trees

• Max-weight spanning tree might have bad topology
  (long support paths)
• Trade worse numerics for better structure
• MASST (min average stretch spanning tree)
• Alon/Karp/Peleg/West(95) for networks
• Gives condition number bound of O(m lg n) for
  general graphs

21
Hybrid Idea

• Augmenting MASST trees
• Spielman Teng (03)
• Add extra edges to improve condition number
• Solve general diagonally dominant M-matrices in
  O(n1.31) time
• No implementation yet

22
Beyond VaidyaBroader Matrix Classes

• Allow columns of U of the form
• a(1, 0, , 0, 1)T
• Now, UUT
• all symmetric, diagonally dominant matrices (SDD)
• Two types of graph edges Signed Graphs
• Max spanning tree becomes max-weight basis of
  matroid
• With Chen/Toledo, devised efficient algorithms
  Vaidya-like analysis
• O(nm) condition number bound for all SDD matrices
• Can augment and get better bounds for planar
  graphs

23
Factorized Approximate Inverses

• A UUT
• What if we have V, an approximate inverse of U?
• Could use VTV as a preconditioner
• Possible advantages
• For some matrices, U is cheap to compute
• Symmetric diagonally dominant
• Finite elements
• Columns of U capture natural structure
• E.g. few columns one finite element
• Allows preconditioner to focus on bad elements

24
Inverting Rectangular Factors

• Let AU D UT, and let V be a solution of
• Or alternatively, A VT U D
• Then, A-1 VT D-1 V

25
Nonsymmetric Systems

• Let AE FT, and X and Y solve
• And
• Then A-1 XT Y

26
Status

• Can solve KKT-like systems approximately
• Use few steps of iterative method, or
• Specify sparsity and minimize Frobineus norm
• Empirical testing underway

27
Other Ongoing Work

• Finite elements
• Use splitting lemma to decompose into elements
• Use symmetric-product lemma to approximate each
  element
• Assemble approximations and approximate the
  result
• Incomplete factorizations
• Simple proof of model problem results
• Suggests alternative dropping strategies
• Domain decomposition
• Easy proof of known results for block Jacobi on
  model problem
• Can generalize to some unstructured grids
• Generalizing tools to handle nonsymmetric matrices

28
Conclusions

• Support numbers are nice analytical tool
• Easy to prove algebraic properties
• Rectangular factorizations are useful for
  analyzing and constructing preconditioners
• Lots of open questions opportunities for new
  insights

29
Acknowledgements

• Sivan Toledo
• Marshall Bern
• Darin Diachin
• Edmond Chow
• Alex Pothen

• Collaborators
• Doron Chen
• John Gilbert
• Steve Guattery
• Ojas Parekh
• Clark Dohrmann
• Nhat Nguyen
• Work supported by DOEs Applied Mathematical
  Science Program
• Sandia is a multiprogram laboratory operated by
  Sandia Corporation, a Lockheed-Martin Company,
  for the U.S. DOE under contract DE-AC-94AL85000.

30
For More Information

• www.cs.sandia.gov/bahendr/support.html
• bah_at_cs.sandia.gov