Parallel Optimization Methods for SimulationBased Problems

1
Parallel Optimization Methods for
Simulation-Based Problems
in Nanoscience

• Juan Meza, Michel van Hove, Zhengji Zhao
• Lawrence Berkeley National Laboratory
• Berkeley, CA
• http//hpcrd.lbl.gov/meza
• Supported by DOE/MICS

SIAM CSE Conference, Orlando, FL, Feb. 12-15,2005
2
Many scientific applications require the solution
of an optimization problem
3
Low-energy electron diffraction (LEED)

• Goal is to determine surface structure through
  low energy electron diffraction (LEED)
• Inverse problem consists of minimizing so-called
  R-factor - a measure of fitness between
  experiment and theory
• Combination of global/local optimization
• Inherently noisy optimization problem

Low-energy electron diffraction pattern due to
monolayer of ethylidyne attached to a rhodium
(111) surface
4
Surface structure determination from experiment

• Forty five structural models were proposed in a
  complex surface structure determination by low
  energy electron diffraction experiments
• Lattice sites can be occupied by Ni or Li atoms,
  or have a vacancy. In addition continuous fit
  parameters corresponding to local relaxation of
  positions are also allowed here.
• Many arrangements of Ni atoms (light and dark
  green) and Li atoms (yellow and orange) are
  possible within the outlined 2-dimensional square
  unit cell.

5
Surface structure determination from experiment

• Electron diffraction determination of atomic
  positions in a surface
• Li atoms on a Ni surface

Global optimization of structure type which
of these 45 structure types best fits
experiment?
Local optimization of structure parameters
which are the best interatomic distances and
angles?
6
Low Energy Electron Diffraction
R-Factors
7
Characteristics of optimization problem

• Inverse problem
• minimize R-factor - defined as the misfit between
  theory an experiment
• Several ways of computing the R-factor
• Combination of continuous and categorical
  variables
• Atomic coordinates, i.e. x, y, z
• Ni, Li
• No derivatives available - standard issue with
  black-box simulations
• Invalid structures lead to function being
  undefined in certain regions and/or discontinuous

8
Pendry R-factor
where the intensity curve, I, is computed by the
LEED code
9
Previous Work

• Previous attempt used genetic algorithms to solve
  the global optimization method.
• Large number of invalid structures generated
  (more on this later).
• Overall, a solution was found - after adding
  sufficient constraints.
• Global Optimization in LEED Structure
  Determination Using Genetic Algorithms, R. Döll
  and M.A. Van Hove, Surf. Sci. 355, L393-8 (1996).
  
• A Scalable Genetic Algorithm Package for Global
  Optimization Problems with Expensive Objective
  Functions, G. S. Stone, M.S. dissertation,
  Computer Science Dept., San Francisco State
  University, 1998.

10
Pattern search methods

• Pattern search methods, Torczon, Lewis Torczon,
  Lewis, Kolda, Torczon (2004), etc.
• Extension to mixed variable problems by Audet and
  Dennis (2000).
• Case of nonlinear constraints studied in
  Abramsons PhD dissertation (2002).
• A frame-based Mesh Adaptive Direct Search (MADS)
  method proposed by Audet and Dennis (2004) that
  removes restriction of a finite number of poll
  directions.
• Good software available - APPSPACK (Kolda),
  NOMADm (Abramson), OPT (Hough, Meza, Williams)

11
NOMADm

• Variables can be continuous, discrete, or
  categorical
• General constraints (bound, linear, nonlinear)
• Nonlinear constraints can be handled by either
  filter method or MADS-based approach for
  constructing poll directions
• Objective and constraint functions can be
  discontinuous, extended-value, or nonsmooth.
• Available at http//en.afit.edu/ENC/Faculty/MAbra
  mson/NOMADm.html

12
MVP Algorithm

• Initialization Given D? , x0 , M0, P0
• For k 0, 1,
• SEARCH Evaluate f on a finite subset of trial
  points on the mesh Mk
• POLL Evaluate f on the frame Pk
• If successful - mesh expansion
• xk1 xk Dk dk
• Otherwise contract mesh

Global phase can include user heuristics or
surrogate functions
Local phase more rigid, but necessary to ensure
convergence
13
Convergence properties

• Assuming f(x) is suitably smooth...
• For unsuccessful iterations, k rf(xk) k is
  bounded as a function of the step length Dk
• Via globalization, lim inf Dk 0
• Conclude lim inf k rf(xk) k 0

14
Test problem

• Model contains three layers of atoms
• Using symmetry considerations we can reduce the
  problem to 14 atoms
• 14 categorical variables
• 42 continuous variables
• Positions of atoms constrained to lie within a
  box
• Best known previous solution had R-factor .24

Model 31 from set of TLEED model problems
15
GA results - atomic coordinates constrained to ?
0.4 Angstrom
16
NOMAD results for minimization with respect to
the continuous variables
Best known solution R-factor 0.24
17
NOMAD results for minimization with respect to
the continuous variables

Best known solution R-factor 0.24
18
GA results - categorical variable search with
fixed atomic positions
best known solution 11111222222222
Li Ni
Remark population size 10 / Generation
19
NOMAD results for categorical variables with
fixed atomic positions
R 0.2387 of func call 49
20
Robustness of NOMAD 15 of 20 initial guesses
(poll step only)
21
Five of 20 trials are trapped in local minima
using only poll step
22
LHS search GSS poll escapes from local minima
(R 0.24) 11111222222222
Li Ni
New minimum found (R 0.1184) 22222112111111
N
23
NOMAD results for 20 trials using LHS GSS
20 trials of identity search
AVG intial R 0.5243
R 0.2387 AVG of func call 73
R 0.1184 AVG of func call 152
Best known solution (R 0.24) 11111222222222
New minimum found (R 0.1184) 22222112111111
24
Minimization with respect to both continuous and
categorical variables
Simultaneous relaxation of both continuous and
categorical variables removes restriction on
coordinates
R-factor 0.24 of func call 212
Best known solution R-factor 0.24
R-factor 0.2151 of func call 1195
25
Minimization with respect to both types of
variables removes coordinate constraints
Penalty R-factor 1.6 (invalid structures)
Best known solution R-factor 0.24
26
LEED Chemical Identity Search Ni (100)-(5x5)-Li
New structure found (R 0.1184)
Best known solution (R 0.24)
27
Conclusions

• GSS methods for mixed variable problems were
  successful in solving the surface structure
  determination problem
• On average NOMAD took 60 function evaluations
  versus 280 for previous solution (GA)
• Improved solutions from previous best known
  solutions found in all cases
• Generation of far fewer invalid structures
• Algorithm appears to be fairly robust, with a
  better structure found in all 20 trial points
• Ability to minimize with respect to both
  categorical and continuous variables a critical
  advantage for these types of problems

28
Future work

• Implement parallel version of algorithms
• Improve the probability of not being trapped in
  local minima
• Develop new SEARCH strategies, especially for
  categorical variables
• Develop automatic strategies for switching
  between different structure models
• Improve objective function call
• Develop new validity check
• Experiment with other R-factor formulations to
  increase the sensitivity
• Implement simultaneous minimization of other
  physical quantities (e.g., energy)

29
Acknowledgements

• Chao Yang
• Lin-Wang Wang
• Xavier Cartoxa
• Andrew Canning
• Byounghak Lee

30
Questions