Solving Specific Classes of Linear Equations using Random Walks

1
Solving Specific Classes of Linear Equations
using Random Walks

• Haifeng Qian
• Sachin Sapatnekar

2
Definition of diagonal dominance

• Matrix A is diagonally dominant if
• aii ? ?i?jaij
• Consider a system of linear equations
• Can solve A x b rapidly using a random walk
  analogy if A is diagonally dominant (but not
  singular, of course!)

(abbreviated as A x b)
3
Why do I care?

• Diagonally dominant systems arise in several
  contexts in CAD (and in other fields) for
  example,
• Power grid analysis
• VLSI Placement
• ESD analysis
• Thermal analysis
• FEM/FDM analyses
• Potential theory
• The idea for random walk-based linear solvers has
  been around even in the popular literature
• R. Hersh and R. J. Griego, Brownian motion and
  potential theory, Scientific American, pp.
  67-74, March 1969.
• P. G. Doyle and J. L. Snell, Random walks and
  electrical networks, Mathematical Association of
  America, Washington DC, 1984.
• http//math.dartmouth.edu/doyle/docs/walks/walks
  .pdf

4
Dirichlets problem

• Dirichlet problem an example thermal analysis
• Given a body of arbitrary shape, and complete
  information about temperature on the surface
  find termperature at an internal point
• Temperature is a harmonic function temperature
  at a point depends on average temperature of
  surrounding points
• Shizuo Kakutani (1944) solution of Dirichlet
  problem
• Brownian motion starting from a point (say, a)
• Take an award T temperature of first boundary
  point hit
• Find ET

a
b
5
A Direct Solver
6
Stochastic solvers
Stochastic solver methodology
7
Example Power grid analysis

• Network of resistors and current sources
• The equation at node x is

• Alternatively

8
Mapping this to the random walk game

• Solving a grid amounts to solving, at each
  nodeor

9
Random walk overview

• Given
• A network of roads
• A motel at eachintersection
• A set of homes
• Random walk
• Walk one (randomlychosen) road every day
• Stay the night at a motel
• (pay for it!)
• Keep going until reaching home
• Win a reward for reaching home!

• Problem find the expected amount of money in the
  end as a function of the starting node x

10
Random walk overview

• For every node

11
Random walk overview
12
Example

• Solution xA0.6 xB0.8
• xC0.7 xD0.9

13
Play the game
1
Pocket
-0.2
1
1
-0.05
-0.04
-0.022

• Walk results

0.578
0.728
0.8
0.444

0.382
0.738
14
Reusing computations avoiding repeated
walks(When solving for all xi values)
New home
Previously calculated node
Benefit more and more homes
shorter and shorter walks one walk
average of multiple walks
Qian et al., DAC2003
15
Reusing computations Journey record(when
solving for multiple right hand sides)
Keep a record motel/award list
New RHS Ax b2
Update motel prices, award values
Use the record pay motels, receive awards
New solution
Benefit no more walks only
feasible after trick1
Qian et al., DAC2003
16
Error vs. runtime tradeoffs

• Industrial circuit
• 70729 nodes, 31501 bottom-layer nodes
• VDD net true voltage range 1.13241.1917

17
Random walk overview

• Advantage
• Locality solve single entry
• Weakness
• Error M-0.5
• 3 error to be faster than direct/iterative
  solvers

18
Example application Power grid analysis

• Exact DC analysis Solve G X E
• Very expensive to solve for millions of nodes,
  eventually prohibitive
• Simple observations
• VDD and GND pins all over chip surface (C4
  connections)
• Most current drawn from nearby connections

19
A preconditioned iterative solver
20
Current techniques

• Preconditioning
• Popular choice Incomplete LU
• Placement/thermal matrices
• Symmetric positive definite
• Popular choice ICCG with
• Different ordering
• Different dropping rules

21
Current techniques
done
done

• Rules
• Pattern
• Min value
• Size limit

22
Stochastic preconditioning

• Special case today
• Symmetric
• Positive diagonal entries
• Negative off-diagonal entries
• Irreducibly diagonally dominant
• These are sufficient, NOT necessary, conditions

23
Sequential Monte Carlo
Stochastic Solver
approx. solve
error residual
approx. solve
Benefit r2ltltb2 y2ltltx2
same relative error lower absolute error
A. W. Marshall 1956, J. H. Halton 1962
24
This is computationally easy!
Can show that this amounts to preconditioned
Gauss-Jacobi So - why G-J? Why not
CG/BiCG/MINRES/GMRES
25
Whats on the journey record?
Row i
These are UL factors. Relation to LU factors?
Just a matrix reordering!
26
LDL factorization

• What we need for symmetric A
• What we have
• How to find ?
• Easy details omitted here
• Easy extension to asymmetric matrices exists

27
Compare to existing ILU

• Existing ILU
• Gaussian elimination
• Drop edges by pattern, value, size
• Error propagation
• A missing edge affects subsequent computation
• Exacerbated for larger and denser matrices

b2
b2
b1
b1
a
b3
b3
b5
b5
b4
b4
28
Superior because

• Each row of L is independently calculated
• No knowledge of other rows
• Responsible for its own accuracy
• No debt from other steps

29
Backup two types of error propagation
No Escape!
30
Test setup

• Quadratic placement instances
• Set 1 matrices and rhss by an industrial
  placer
• Set 2 matrices by UWaterloo placer on ISPD02
  benchmarks, unit rhss
• LASPACK ICCG with ILU(0)
• MATLAB ICCG with ILUT
• Approx. Min. Degree ordering
• Tuned to similar factorization size
• Same accuracy
• Complexity metric double-precision
  multiplications
• Solving stage only

31
Comparison
32
Physical runtimes on P4-2.8GHz

• Reasonable preconditioning overhead
• Less than 3X solving time
• One-time cost, amortized over multiple solves

33
Scaling trend
34
Newer results

• Examples generated from Sparskit
• Finite difference discretization of 3D Laplaces
  equation under Dirichlet boundary conditions
• S nonzeros in preconditioner (similar in all
  cases)
• C1 Condition of original matrix
• C2 Condition after split preconditioning

Ex1
Ex2
Ex3
Ex4
Ex5
Ex6
35
Conclusion

• Direct solver
• Locality property
• Reasonable for approximate solutions
• Hybrid solver
• Stochastically preconditioned iterative solver
• Independent row/column estimates for LU factors
• Better quality than same-size traditional ILU
• Extension to non-diagonally dominant matrices?
• Some pointers exist (see Haifeng Qians thesis)
• Needs further work

36

• Thank you!
• Downloads
• Solver package
• http//www.ece.umn.edu/users/qianhf/hybridsolver
• Thesis
• http//www-mount.ece.umn.edu/sachin/Theses/Haifen
  gQian.pdf

37
Thanks also (and especially) to

• Haifeng Qian
• (He will pick up the 2006 ACM Oustanding
  Dissertation Award in Electronic Design
  Automation in San Diego next week)