Parallel Implementation for SemiDefinite Programming with Positive Matrix Completion Method

1
Parallel Implementation for SemiDefinite
Programming withPositive Matrix Completion Method

• Kanagawa University
• Dept. of Industrial Eng. and Management
• Makoto Yamashita
• Tokyo-Tech Masakazu Kojima Tokyo Denki Univ.
  Katsuki Fujisawa Tokyo-Tech Kazuhide Nakata

2
Motivation

• SDP (SemiDefinite Program) has many applications
• SDPs become extremely large
• Solve larger SDPs in shorter time
• Utilization of Parallel Computation
• SDPARA-C SDPARA (SemiDefinite Programming
  Algorithm paRAllel version) with the completion
  method

3
Contents of this talk

• Definition of SDP
• Concepts of Parallel-ism
• The Completion Method and SDPARA-C
• Conclusion

4
SDPs arisen from

• Control Theory
• Combinatorial Optimization
• Data Mining
• Quantum Chemistry
• Successive Relaxation Methods for Non-convex
  optimization problem
• and more direct/indirect applications

The Completion Method
5
Standard form of SDP
6
Primal-Dual Interior-Point Methods
7
Computation for Search Direction
2.CHOLESKY
Schur complement matrix ? Cholesky Factorizaiton
Schur complement equation
3.PMATRIX
1.ELEMENTS
4.DENSE (dense matrix computation)
8
Bottlenecks on Single Processor
Parallelize in SDPARA
SDPARA-C
Computation time in second
9
Row-wise distribution for evaluation of the Schur
complement matrix
Symmetric Matrix4 Processors No
Communication Simple memory distribution
10
Redistribution of Schur complement matrix for
parallel Cholesky factorization
11
Numerical results ontheta 6 Lovászs theta
function(m 4375, n300)
15
5
12
LiF Quantum Chemistry(m15313, n496)
13
maxG51 Max-Cut Problem(m 2000, n2000)
14
Drawbacks of SDPARA

• Ineffective on Max-Cut Problem
• PMATRIX,DENSE are bottlenecks
• Full-dense in primal variable matrix
• Inheritance of sparsity in dual
• m large, n medium size

15
Sparsity in Dual
For example, Primal is
Dual is
16
The Completion Method

• Introduce Sparsity in Primal Variable
• Nakata-Fujisawa-Fukuda-Kojima-Murota
• Positive Definite Matrix Completion
• Chordal Graph
• Sparse Cholesky Factorization

2003
17
Aggregate Sparsity Pattern
All elements are required for PSD
Enough for objective function and equality
constraints
Aggregate Sparsity Pattern
18
Positive Definite Matrix Completion
Positive Definite Matrix Completion Assign values
not in to be positive definite
19
Chordal Graph
1 2 3 4 5 6 7
1 2 3 4 5 6 7
Chordal Graph
Length of minimal cycle ? 3
Aggregate Sparsity Pattern
Insufficient for Positive Definite Matrix
Completion
20
Extended Sparsity Pattern
Chordal Graph
21
Sparse Cholesky Factorization

• Instead of dense matrices
• Store such that
• The sparse structure of is determined by
  Chordal Graph
• Modify the Schur complement

22
Performance of Completion Methods
PARALLEL
PARALLEL
PARALLEL
BETTER
WORSE
PARALLEL
Max Clique Problem m1891, n 1000, 64 processors
23
SDPARA-C

• SDPARA Completion Method
• Nakata-Yamashita-Fujisawa-Kojima 2003
• Parallel Computation for
• ELEMENTS
• CHOLESKY (same as SDPARA)
• PMATRIX

24
Based on Row-wise distributionfor ELEMENTS
In SDPARA-C, Evaluation of Schur complement
matrix is based on Row-wise distribution
Sometime, Load-balance becomes worse in the
combination with the completion methods
25
Break-down of Row-wise Distribution

• Computation cost concentrates on inverse
  multiplications
• Strong dependency on the number of non-zero
  column vectors
• In Max Clique Problem
• has non-zero vectors
• Each have only one nonzero
  vectors
• Work of the Processor for 1st row is heavy

26
Hashed row-wise distribution

• Divide non-zero vectors into subsets
• th processor computes
• Accumulate results with network communication

27
Simple row-wise and Hashed row-wise
Communication
28
Parallel Computation for PMATRIX

• Column-wise computation of
• Each column requires same computation cost
• Non-cyclic distribution

29
Performance of SDPARA-C
PARALLEL
Completion
Max Clique Problem m1891, n 1000, 64 processors
30
SDPARA,SDPARA-C,PDSDP for norm.10.990(m11,n1000,
?21)
PDSDP (Benson,2002)
31
Memory Reduction in SDPARA-C

• Schur complement matrix on distributed memory
• Memory reduction for owing to the
  completion method
• SDPARA-C can handle extremely large SDPs
  involving both many equality constraints and
  large-scale matrix (m,n ? 40000 on Max-Cut)

32
SDPs from SDPLIB and DIMACS
2017
1958
X
X
memoryover
memoryover
33
Conclusion

• How to solve large SDPs in a short time
• The Completion Method
• Hashed Row-wise for ELEMENTS
• Parallel for PMATRIX
• SDPARA-C solves extremely large SDPs
• Satisfying scalability and load-balance

34
Future Directions

• Combination of SDPARA, SDPARA-C
• SDPARA Dense or Medium
• SDPARA-C Sparse and Large
• Numerical Stability
• Application
• Quadratic Assignment Problem
• Network Location Problem

35
Optimization in Quantum Chemistry
Nakata et al (2003), Zhao et al (2004)

• Compute the ground-state energy of some molecules
  
• N-representability and Reduced Density Matrix
  (Kernel Matrix)
• Hilbert Space
• Ray-Leigh ratio

Energy
Excited states
n4
n3
n2
n1
Stable state
ground state energy
36
Quantum Chemistry

• Prepare basic wave functions
• Find a wave function which minimizes the ground
  state energy

Hamiltonian
37
SDP from Quantum Chemistry

• The ground state energy is an optimal value of
  the SDP

Total electron numbers
Each basic wave function contains an
electronwith probability between 0,1
where
38
Methods for the ground state energy

• Self Consistent Field
• Configuration Interaction
• Full CI
• Single-Double CI
• SDP relaxation

Energy
Single-Double CI
Full CI
The ground state energy
SDP relaxation
39
Numerical Results of Accuracyfor SDPs arisen
from Quantum Chemistry
N of electron, 2K of orbits ?SDP
difference bet. SDP and Full-CI ?SDCI difference
bet. Singly-Double excited CI and
Full-CI Full-CI Accurate value with enormous
computation time