Department Review 2002

1
NOX An Object-Oriented Nonlinear Solver
Package Roger Pawlowski, Tamara Kolda, Russell
Hooper, and John Shadid Sandia National
Laboratories Albuquerque, NM Trilinos User
Group Meeting October 16th, 2003
Sandia is a multiprogram laboratory operated by
Sandia Corporation, a Lockheed Martin
Company,for the United States Department of
Energys National Nuclear Security
Administration under contract DE-AC04-94AL85000.
2
Background

• NOX is a Trilinos solver package built to
• enable robust and efficient solutions to systems
  of nonlinear equations.
• provide a broad array of nonlinear algorithms
  through a single interface.
• rapid deployment of new solver technology into
  ASCI codes.
• eliminate redundancies in ASCI code development.
• Started 2.5 years ago.
• Funding ASCI Algorithms and LDRD (2.5 FTE).
• Consists of two libraries
• NOX nonlinear algorithms
• LOCA v2.0 continuation and bifurcation
  algorithms (Eric Phipps)
• Website software.sandia.gov/nox

3
In-NOX-ulated Applications
Primary NOX Team Tammy Kolda, Roger Pawlowski,
Russ Hooper
4
Nonlinear Equations

• Given
• find for which
• F is a system of n nonlinear equations with n
  unknowns.
• x is the unknown or solution vector of size n.

5
Nonlinear Solution Algorithms
Globalizations
Iterative Linear Solvers, Adaptive Forcing Terms
User Norms and Merit Functions
6
Example Newtons Method for F (x) 0
Building Blocks of NOX

• Choose an initial guess x0
• For k 0,1,2,...
• Compute Fk F (xk)
• Compute Jk where (Jk )ij F i(xk)/x j
• Let dk -Jk-1 Fk
• (Optional) Let lk be a calculated step length
• Set xk1 xk lkdk
• Test for Convergence or Failure

7
NOX Solvers(Iterate Control)
Derived from NOXSolverGeneric bool 
reset(NOXAbstractGroup grp,
NOXStatusTestGeneric tests,
NOXParameterList params)   bool 
reset(NOXAbstractGroup grp,
NOXStatusTestGeneric tests)   NOXStatusT
estStatusType getStatus()   NOXStatusTestSta
tusType iterate()   NOXStatusTestStatusType so
lve()   const NOXAbstractGroup  getSolutionGr
oup() const   const NOXAbstractGroup  getPrev
iousSolutionGroup() const   int getNumIterations()
const   const NOXParameterList  getParameter
List() const

• LineSearchBased
• Compute Direction
• Compute Step Length
• Scale direction by step length
• Update solution
• Check Convergence
• TrustRegionBased (dogleg)
• Computes Newton Direction
• Computes Cauchy Direction
• Adjust trust region radius
• Update solution
• Check Convergence
• InexactTrustRegionBased (Prerelease)
• TensorBased (Prerelease)
• UserDefined

Solver is determined by passing a parameter list
to a handle for the class NOXSolverManager
solver(group,
StatusTests,

ParameterList) int status solver.solve()
8
NOX Directions

• Steepest Descent
• Newton
• Requires a linear solve
• Adaptive forcing terms
• Broyden
• Modified Newton with rank-1 updates to the most
  recently computed Jacobian
• Nonlinear-CG (Prerelease)
• Tensor (Prerelease)

Direction is determined and constructed in a
handle called a Manager NOXDirectionManage
r dir(PrintParams, DirectionParams)
User Defined Directions Derived from
NOXDirectionGeneric Passed into the solver
via parameter list using a direction template
ctor.
User Defined Merit Function Derived from
NOXParameterMeritFunction Passed into the
solver via parameter list using the
NOXParameterArbitrary entry
9
NOX Line Searches
Line Searches are determined and constructed in a
handle called a Manager NOXLineSearchManag
er ls(PrintParams, LineSearchParams)

• Full Step
• Constant step size.
• Defaults to 1.0.
• Backtrack
• interval halving
• Polynomial Interpolation
• Quadratic and Cubic interpolation
• Requires a Sufficient Decrease Condition
• More-Thuente
• Polynomial Interpolation
• Sufficient Decrease and Curvature Conditions
• Nonlinear-CG (Prerelease)
• Curvilinear (Prerelease)

User Defined Line Searches Derived from
NOXLineSearchGeneric Passed into the solver
via parameter list using a template constructor.
User Defined Merit Function Derived from
NOXParameterMeritFunction Passed into the
solver via parameter list using the
NOXParameterArbitrary entry
User Defined Norms Derived from
NOXParameterUserNorm Passed into the solver
via parameter list using the NOXParameterArbit
rary entry
10
Mix-n-Match Solver Algorithms

• Higly versatile code environment

• Solver
• Line Search Based
• Trust Region Based
• Tensor Based
• Direction
• Newton
• Broyden
• Steepest Descent
• Tensor
• User-Defined
• Line Search / Damping
• Full Step
• Backtrack
• Polynomial/Quadratic
• More-Thuente
• Curvilinear
• User-Defined

11
Stopping Criteria (StatusTests)

• Highly Flexible Design Users build a
  convergence test hierarchy and registers it with
  the solver (via solver constructor or reset
  method).
• Norm F Inf, One, Two absolute, relative
• Norm Update DX Inf, One, Two
• Norm Weighted Root Mean Square (WRMS)
• Max Iterations Failure test if solvers reaches
  max iters
• FiniteValue Failure test that checks for NaN
  and Inf on
• Stagnation Failure test that triggers if the
  convergence rate fails a tolerance check for n
  consecutive iterations.
• Combination AND, OR
• Users Designed Derive from NOXStatusTestGener
  ic

Return Types Unconverged Converged Failed Uneval
uated
12
Building a Status Test

• Converge if both

• Fail if value of becomes Nan or Inf

• Fail if we reach maximum iterations

13
Status Tests Continued
-- Status Test Results --...........OR
Combination -gt ...........AND Combination -gt
...........F-Norm 5.907e-01 lt 1.000e-08
(Length-Scaled Two-Norm, Absolute
Tolerance) ...........WRMS-Norm 4.794e01
lt 1 (Min Step Size 1.000e00
gt 1) (Max Lin Solv Tol
1.314e-15 lt 0.5) ...........Finite Number
Check (Two-Norm F) Finite ...........Number
of Iterations 2 lt 200
-- Final Status Test Results --Converged....OR
Combination -gt Converged....AND Combination -gt
Converged....F-Norm 3.567e-13 lt 1.000e-08
(Length-Scaled Two-Norm, Absolute
Tolerance) Converged....WRMS-Norm 1.724e-03
lt 1 (Min Step Size 1.000e00
gt 1) (Max Lin Solv Tol
4.951e-14 lt 0.5) ??...........Finite Number
Check (Two-Norm F) Unknown ??...........Number
of Iterations -1 lt 200
User Defined are Derived from NOXStatusTestGen
eric NOXStatusTestStatusType checkStatus(const
NOXSolverGeneric problem)   NOXStatusTest
StatusType  checkStatusEfficiently(const
NOXSolverGeneric problem,
NOXStatusTestCheckType
checkType)   NOXStatusTestStatusType getStatus
() const   ostream print(ostream stream, int
indent0) const
14
NOX Framework
SolverLayer

• Dont need to directly access the vector or
  matrix entries, only manipulate the objects.
• NOX uses an abstract interface to manipulate
  linear algebra objects.
• Isolate the Solver layer from the linear algebra
  implementations used by the application.
• This approach means that NOX does NOT rely on any
  specific linear algebra format.
• Allows the apps to tailor the linear algebra to
  their own needs!
• Serial or Parallel
• Any Storage format LAPACK, PETSc, Epetra, User
  Defined

15
NOXAbstractVector

• Initialization
• x y
• x y
• xi 1/yi for i 1 to n
• xi g for i 1 to n
• Length of Vector
• Scaling
• x a x
• xi xiyi for i 1 to n
• Update
• x a a g x,
• x a a b b g x

• Norm
• k x k1, k x k2 , k x k1
• k x kw (weighted norm)
• Dot
• x y
• Clone (create a copy)
• y x

The solver is not allowed nor does it need
explicit access to the vector just the ability
to manipulate it.
16
NOXAbstractGroup

• x Iterate
• Initialize
• Update
• Access
• F F (x)
• Compute
• Access
• J Jacobian of F at x
• Compute
• Apply / Apply Transpose
• Apply Inverse
• Apply Preconditioning
• n Newton Vector
• Compute n -J-1F to specified tolerance
• Access

• g Gradient of kF (xk)k2
• g JTF
• Access
• Clone

The linear solver and application interface are
combined into the Group. Vectors x, F, n, g are
accessed as NOXAbstractVectors. Matrix J is
never directly accessed!
17
Code Demonstration
18
NOX Framework
SolverLayer
19
Linear Algebra Support Features

• NOXs support libraries define a concrete
  implementation of the Abstract Vector and Group.
• We only require the user to implement a minimal
  interface derived from NOXltPackagegtInterface.

bool computeF(const Epetra_Vector x,
Epetra_Vector f, FillType flagF) bool
computeJacobian(const Epetra_Vector x,
Epetra_Operator Jac)   bool computePrecMatrix(con
st Epetra_Vector x, Epetra_RowMatrix M)   bool
computePreconditioner(const Epetra_Vector x,
Epetra_Operator M)
Group(NOXParameterList printingParams,
NOXParameterList linearSolverParams,
NOXEpetraInterface i, NOXEpetraVector
x, Epetra_Operator J) Group(NOXParameterList
printingParams, NOXParameterList
linearSolverParams, NOXEpetraInterface i,
NOXEpetraVector x, Epetra_Operator J,
Epetra_Operator M)
20
The Epetra Goodies

• Matrix-Free Newton-Krylov Operator
• Derived from Epetra_Operator
• Can be used to estimate Jacobian action on a
  vector
• NOXEpetraMatrixFree
• Finite Difference Jacobian
• Derived from an Epetra_RowMatrix
• Can be used as a preconditioner matrix
• NOXEpetraFiniteDifference
• Graph Colored Finite Difference Jacobian
• Derived from NOXEpetraFiniteDifference
• Fast Jacobian fills need connectivity/coloring
  graph
• (NOXEpetraFiniteDifferenceColoring)
• Full interface to AztecOO using NOX parameter
  list
• Preconditioners internal AztecOO, Ifpack, User
  defined
• Scaling object

21
Homotopy Algorithms

• LOCA Library of Continuation Algorithms
• Bifurcation and stability analysis package.
• Tightly coupled to NOX to reuse application
  interface.
• Provided as a single package.
• Stepper object provides a robust and intelligent
  step control for parameter continuation.
• Eric Phipps

22
NOX Specific Configure Options

• Compiling NOX library in Trilinos
• --enable-nox
• Compiling prerelease code
• --enable-prerelease
• Using nox built-in linear algebra support
• LAPACK --enable-nox-lapack
• --enable-nox-lapack-examples
• Epetra --enable-nox-epetra
• --enable-nox-epetra-examples
• PETSc --enable-nox-petsc
• --enable-nox-petsc-examples

• Compiling the test suite
• --enable-tests
• --enable-nox-tests
• Compiling LOCA library in Trilinos/NOX
• --enable-loca

23
Future Work

• Test Suite Development Darin Diachin
• Re-work of the NOXEpetra support library for
  efficiency - Pawlowski
• TSF support library Pawlowski, Hooper
• Tensor algorithm development Brett Bader, Tammy
  Kolda
• Broyden algorithm development Kolda, Pawlowski
• Multi-Physics Solvers Strong code coupling

24
Multi-Physics Solvers

• MultiPhysics Bill Spotz and Alfred Lorber
• rapid prototyping environment to explore solver
  coupling algorithms.
• PyNOX - Python interface to call NOX solver.
• SIERRA Russell Hooper, John Shadid and Roger
  Pawlowski
• Implementation of a Jacobian-Free coupling
  algorithm in SIERRA.

Brusselator Example in NOX (Russell Hooper)
Loose Coupling
Full Coupling
Jacobian-Free Coupling
with preconditioner
Analytic OR FD Coloring (Epetra)
25
Summary

• Key Features
• Object-oriented C library
• Large number of cutting-edge algorithms
• Can use any linear algebra implementation
• Easy interface
• User Flexibility Add solvers, directions, line
  searches, convergence tests
• NOX Support libraries
• Fast integration reduced interface
• parallel iterative solvers (AztecOO, PETSc)
• preconditioners (Ifpack, ML, Aztec, PETSc)
• Matrix-Free Newton-Krylov
• Matrix Estimation Finite Difference/Coloring

26
NOX Contributors (Ideas, Code, and Testing)

• Sandia
• Tammy Kolda
• Roger Pawlowski
• Russ Hooper
• John Shadid
• Todd Coffey
• Andy Salinger (LOCA)
• Eric Phipps (LOCA)
• Bill Spotz
• Mike Heroux (Trilinos)
• Scott Hutchinson (Xyce)
• Rob Hoekstra (Xyce)
• Eric Keiter (Xyce)
• Alan Williams (SIERRA)
• Tom Smith (Premo)
• Alfred Lorber (Premo)
• Tom Brunner (Alegra)

• Academics
• Homer Walker, WPI
• Joe Simonis, WPI
• Tim Kelley, NCSU
• Bobby Schnabel, UCB
• Richard Byrd, UCB
• Ryan McKenzie, Kentucky
• Craig Douglas, Kentucky

NOX Developer
NOX Information Tammy Kolda tgkolda_at_sandia.gov
Roger Pawlowski rppawlo_at_sandia.gov Download
http//software.sandia.gov/nox