Parallel Implementation of InteriorPoint Methods of SemiDefinite Program

1
Parallel Implementation of Interior-Point
Methods of SemiDefinite Program

• Kanagawa University
• Dept. of Industrial Eng. and Management
• Makoto Yamashita
• Katsuki FujisawaMasakazu Kojima
• Kazuhide Nakata

2
Motivation

• SDP (SemiDefinite Program) has many applications
• SDPs become extremely large
• We want to solve large SDPs fast
• Parallel Computation is utilized
• Two parallel software were developed
• SDPARA (SDPA parallel version)
• SDPARA-C (SDPARA with the completion method)

3
Contents of this talk

• Definition of SDP and Applications
• Interior-Point Methods and Bottlenecks
• SDPARA and Concepts of Parallel-ism
• Principal Features of SDPARA-C
• Conclusion

4
Standard form of SDP
? SDPARA
? SDPARA-C
5
SDPs arise from

• Control Theory
• Combinatorial Optimization
• Quantum Chemistry
• Successive Relaxation Methods for Non-convex
  optimization problem
• and more direct/indirect applications

6
Quantum Chemistry
Nakata et al (2002), Zhao et al (2004)

• We want to compute the ground state energy of an
  molecular
• Prepare basic wave functions
• Find a wave function which minimizes the ground
  state energy

Hamiltonian
7
SDP from Quantum Chemistry

• The ground state energy is an optimal value of
  the SDP

Total electron numbers
Each basic wave function contains an
electronwith probability between 0,1
where
8
Existing Methods for SDPs

• Nonlinear Programming Methods
• Lagrangian Methods (PENNON)
• Spectral Bundle Methods (SBMethod)
• Low accuracy (large scale)
• Interior-Point Methods
• Primal-Dual (SDPA,SDPT3,SeDuMi)
• Dual-Scaling (DSDP)
• High stability (medium scale)

9
Primal-Dual Interior-Point Methods
10
Computation for Search Direction
2.CHOLESKY
Schur complement matrix ? Cholesky Factorizaiton
Schur complement equation
1.ELEMENTS
11
Bottlenecks on Single Processor
Parallelize in SDPARA
Computation time in second
12
Parallel implementation

• SDPARA (SDPA paRAllel version)
• Yamashita-Fujisawa-Kojima, 2002
• http//sdpa.is.titech.ac.jp/
• With libraries
• MPI-CH (An implementation of MPI)
• ScaLAPACK (Scalable Linear Algebra PACKage)
• Parallel computation for
• Elements of Schur complement matrix
• Cholesky Factorization
• on distributed memory
• Cannot store Schur complement matrix (m x m) on
  memory space of single processor

13
Parallel evaluation for the Schur complement
matrix

• .
• Each element can be computed independently if
  each processor stores
• All elements in i-th row shares
• Row-wise distribution is reasonable.

14
Row-wise distribution for evaluation of the Schur
complement matrix
Symmetric matrix 4 Processors No
Communication Simple memory distribution
15
Redistribution of Schur complement matrix for
parallel Cholesky factorization
16
PC-Cluster

• Presto III _at_ Tokyo-Tech
• Athlon 1900 MHz
• Myrinet (High-Speed Network)
• MPICH-GM
• 760 GFLOPS

17
Numerical results ontheta 6 Lovászs theta
function(m 4375, n300)
15
5
18
Numerical results oncontrol11 by
scalability(m1596, n165)
19
SDP from Quantum Chemistry

• SDPARA can solve an extremely large SDP (m ?
  24,000) in 600 second.
• Without distributed memory, we cannot solve the
  SDP.
• For larger problems, the scalability becomes
  better.

20
Weak point of SDPARA

• SDPARA can solve SDP with large number of
  equality constraints
• SDPARA is not suited forlarge variable matrices
  .
• Primal variable matrix is dense.
• The completion methodinto SDPARA ? SDPARA-C

21
Completion Method

• Instead of dense matrices
• Store such that
• The sparse structure of is determined by
  Chordal Graph
• Modify the Schur complement

22
Break-down of Row-wise Distribution

• Computation cost concentrates on multiplication
  of inverse matrices
• Strong dependency on the number of non-zero
  vectors
• In Max Clique Problem
• has non-zero vectors
• Each have only one nonzero
  vectors
• works of processor assigned to 1st row is heavy

23
Simple row-wise and Hashed row-wise
Communication
24
Main features of SDPARA-C

• The completion method
• Sparsity in primal matrix
• Hashed distribution of ELEMENTS
• Parallel computation for
• Reduction of memory space
• Solve extremely large and sparse SDPsm large, n
  large (m, n ? 40,000)

25
Conclusion

• How to solve extremely large SDPs fast
• Parallel for ELEMENTS and CHOLESKY
• SDPARA solves SDPs with many equality constraints
  very fast
• Satisfying scalability and load-balance

26
Future Directions

• Combination of SDPARA, SDPARA-C
• SDPARA Dense or Medium
• SDPARA-C Sparse or Large
• Distribution of variable matrices