ML: A Multilevel Preconditioning Package

1
ML A Multilevel Preconditioning Package

• Copper Mountain Conference on Iterative Methods
• March 29-April 2, 2004
• Jonathan Hu
• Ray Tuminaro
• Marzio Sala

Sandia is a multiprogram laboratory operated by
Sandia Corporation, a Lockheed Martin
Company,for the United States Department of
Energy under contract DE-AC04-94AL85000.
2
Outline

• Overview
• Multigrid basics
• Available user options
• Configuring building
• Interoperability with other packages
• Example program
• Conclusions

3
ML Package

• Provides parallel multigrid preconditioning for
  linear solver methods
• Current developers Ray Tuminaro, Jonathan Hu,
  Marzio Sala
• Former Charles Tong (LLNL)
• Main methods
• Geometric
• Grid refinement hierarchy
• 2-level FE basis function domain decomposition
• AMG (algebraic multigrid based on aggregation)
• Smoothed aggregation
• Edge-element AMG for Maxwells equations
• n-level (smoothed) aggregation domain
  decomposition
• Classical AMG
• Written primarily in C
• C interfaces to various Trilinos packages
• 102,411 lines of code (as of this afternoon)
• 26 example programs
• Downloadable as part of Trilinos
• CVS, bugzilla

4
Recursive MG Algorithm (V Cycle)

• MG(f, u, k)
• if (k 1) u1 (A1)-1f1
• else
• Sk(Ak, fk,uk) //pre-smooth
• rk fk Ak uk
• fk-1 Rk-1 rk uk-1 0 //restrict
• MG(fk-1, uk-1, k-1)
• uk uk Ik-1 uk-1 //interpolate
  correct
• Sk(Ak, fk,uk) //post-smooth
• Ak, Rk, Ik, Sk required on all levels
• Sk smoothers
• Rk restriction operators
• Ik interpolation operators

V Cycle to solve A4u4f4
k4
k1
5
ML Capabilities

• MG cycling V, W, full V, full W
• Grid Transfers
• Several automatic coarse grid generators
• Several automatic grid transfer operators
• Coarse grid visualization capabilities
• Smoothers
• Jacobi, Gauss-Seidel, Hiptmair, Krylov methods,
  sparse approximate inverses, Chebyshev
• Variety of Serial Parallel Direct Solvers
• SuperLU, Umfpack, KLU, MUMPS, etc.
• Kernels matrix/matrix multiply, etc.

6
Smoothed Aggregation

• Developed for linear elasticity (Vanek, Mandel,
  Brezina)
• Construct tentative prolongator tP
• Aggregation group unknowns together
• Augment interpolate null space (e.g., rigid body
  modes)
• Construct final prolongator P
• Smooth P (I ? D-1A)TP
• lower energy in basis functions

7
Smoothed Aggregation
8
Uncoupled / MIS Aggregation

• Greedy algorithm

9
Graph partitioning aggregation

• Global graph partitioning
• Operates on global domain
• ParMETIS
• Aggregates can span processors

• Local graph partitioning
• Processors work independently
• METIS
• ? 1 aggregate per processor

10
ML Smoother Choices

• Jacobi, Point/Block Gauss Seidel
• MLS
• Based on Chebyshev polynomials of smoothed
  operator.
• Serial competitive with true Gauss-Seidel
• Parallel Performance is independent of
  processors
• Hiptmair
• Distributed relaxation smoother for Maxwells
  Eqns.
• Other smoothers used within each projection step
• Aztec solvers
• Krylov methods, incomplete factorizations
• Direct solution (via Amesos interface)
• UMFPACK, KLU (serial)
• SuperLU
• MUMPS

11
ML and Other Packages
12
Configuring and Building ML

• Builds by default when you configure build
  Trilinos
• By default, you get
• Epetra Aztecoo support
• Example suite (Trilinos/packages/ml/examples)
• Some options of interest (off by default)
• MPI support
• Graph partitioning aggregation (METIS, ParMETIS)
• Direct solvers (Amesos)
• Profiling

configure --with-mpi-compilers/usr/local/mpich/bi
n --with-ml_amesos \ --with-libs-lamesos
lsuperlu
13
A Small Exampleml/examples/ml_example_epetra_pre
conditioner.cpp
Solve Axb A advection/diffusion
operator Linear solver gmres Precond. 2-level
AMG, graph-partitioning
aggregation
Solution component
Example methods
Packages used
Linear Solver
AztecOO (Epetra)
gmres
ML multi- grid pre- cond.
ML (Teuchos)
AMG
14
ml_example_epetra_preconditioner.cpp

• Trilinos_Util_CrsMatrixGallery
  Gallery(recirc_2d", Comm)
• Gallery.Set("problem_size", 10000)
• // linear system matrix linear problem
• Epetra_RowMatrix A Gallery.GetMatrix()
• Epetra_LinearProblem Problem
  Gallery.GetLinearProblem()
• // Construct outer solver object
• AztecOO solver(Problem)
• // Set some solver options
• solver.SetAztecOption(AZ_solver, AZ_gmres)
• solver.SetAztecOption(AZ_output, 10)
• solver.SetAztecOption(AZ_kspace, 160)

15
example (contd.)

• // Set up multilevel preconditioner
• ParameterList MLList // parameter list for ML
  options
• MLList.set("max levels",2)
• MLList.set("aggregation type", "METIS")
  // graph partitioning
• MLList.set("aggregation nodes per aggregate",
  16)
• // set up aztecoo smoother
• MLList.set("smoother type","aztec")
• int optionsAZ_OPTIONS_SIZE double
  paramsAZ_PARAMS_SIZE
• AZ_defaults(options,params)
• optionsAZ_precond AZ_dom_decomp
• optionsAZ_subdomain_solve AZ_ilut
• MLList.set("smoother aztec options", options)
• MLList.set("smoother aztec params", params)
• MLList.set("coarse type","Amesos_Superludist")
  
• MLList.set("coarse max processes", 4)

16
AMG for Common Problem Types

• Trilinos_Util_CrsMatrixGallery
  Gallery(laplace_3d", Comm)
• Gallery.Set("problem_size", 100100100)
• // linear system matrix linear problem
• Epetra_RowMatrix A Gallery.GetMatrix()
• Epetra_LinearProblem Problem
  Gallery.GetLinearProblem()
• // Construct outer solver object
• AztecOO solver(Problem)
• solver.SetAztecOption(AZ_solver, AZ_cg)
• // Set up multilevel precond. with smoothed
  aggr. defaults
• ParameterList MLList //
  parameter list for ML options
• ML_EpetraSetDefaults(SA,MLList)
• ML_EpetraMultiLevelPreconditioner MLPrec
  new // create preconditioner
• ML_EpetraMultiLevelPrecondition
  er(A, MLList, true)
• solver.SetPrecOperator(MLPrec) // tell solver
  to use ML preconditioner
• solver.Iterate(500, 1e-12) // iterate at
  most 500 times

17
Collaborations

• Within Sandia
• ALEGRA
• Radiation
• Maxwell
• Electrostatic potential
• MPSalsa (Shadid, et al.)
• CEPTRE
• External users
• EM3D (Lawrence Berkeley NL)
• P. Arbenz (ETH Zurich)
• R. Geus (PSI)
• J. Fish, H. Waisman (RPI)
• Potential Users
• PREMO (Sandia)

18
Getting Help

• Website www.cs.sandia.gov/tuminaro/ml
• See Trilinos/packages/ml/examples
• See guide
• ML Users Guide, ver. 3.0 (in ml/doc)
• Mailing lists
• ml-users_at_software.sandia.gov
• ml-announce_at_software.sandia.gov
• Bug reporting, enhancement requests via bugzilla
• http//software.sandia.gov/bugzilla
• Email us directly
• jhu_at_sandia.gov
• rstumin_at_sandia.gov
• msala_at_sandia.gov

19
Conclusions

• ML 3.0 release this May
• Part of next Trilinos release
• Existing options
• Uncoupled / MIS aggregation
• Smoothers
• SuperLU
• New options with ML 3.0
• Graph-based aggregation
• parameter list setup
• Amesos interface to direct solvers
• Visualization, aggregate statistics
• Use ML with Trilinos!
• ML as preconditioner is trivial
• Rich set of external libraries