Gradient Penalty Based Search

1
Gradient Penalty Based Search

• Exterior Penalty
• Slides taken from Numerical Optimization
  Techniques for Engineering Design by Vanderplaats
• 6/16/05

2
Review Key Goals

• Understanding the client side iSIGHT Optimization
  Model Abstraction
• Understanding the internal representation of the
  iSIGHT Optimization Model
• Understanding the transformation between iSIGHT
  Model and Third Party Optimization Packages
• Give you conceptual background to understand
  options of each technique

Bracketing, golden section, polynomial
interpolation Steepest descent, fletcher reeves,
BFGS
3
Walk Away Info

• Problem Formulation is critical
• Scaling
• Good initial starting point
• Use multiple starting points
• Read the optimizers diagnostic output

4
Nonlinear Constrained Optimization

• First look at constrained optimization.
• Simplest and most natural approach is to turn a
  constrained problem into an unconstrained
  problem.
• Here we are dealing with nonlinear objectives and
  nonlinear constraints.
• We can then use gradient techniques conveed in
  last chapter to solve. (Steepest Descent,
  Fletcher-Reeves, BFGS)
• In this chapter, we will look at gradient based,
  exterior penalty approach.

5
Exterior Penalty

• I like it. It is undervalued
• Use it with an initial infeasible design to get
  feasible or close to feasible.
• Great in a Task Plan when complemented by a GRG
  or SQP. (Sandgren PhD Thesis)
• Insure you have scaled/normalized design
  variables, constraints and objective.
• Universal point give a good starting value if
  you have one. Some people just give all zeros or
  ones for convenience but this is not working with
  the numerical based optimizers.

6
Caution

• Difference between ADS exterior penalty and the
  ObjectiveAndPenalty data field kept by iSIGHT.
• ObjectiveAndPenalty is always calculated for
  every optimization technique even though only a
  few directly use it.
• ObjectiveAndPenalty IS USED by Genetic
  Algorithms, Simulated Annealing, Hooke Jeeves,
  Downhill Simplex.
• ObjectiveAndPenalty is similar to that used in
  exterior penalty techniques except it uses a
  penalty base.
• ObjectiveAndPenalty is NOT used in exterior
  penalty method of ADS

7
(No Transcript)
8
iSIGHT ObjectiveAndPenalty
9
ADS Exterior Penalty
Minimize objective function as unconstrained
function but imposing a penalty to limit
constraint violations
Penalty can lead to numerical ill-conditioned
problem
Exterior penalty imposes a moderate penalty
solves the problem,increases the penalty solves
the problem, increases the penaltysolves the
problem,
10
ADS Exterior Penalty
Solving a series of unconstrained problems rp is
the penalty multiplier. It is constant for the
solving of one unconstrained optimization. It
has a default value of 10.
After solving the problem, rp is multiplied by a
penalty multiplier, rmult. It has a default
value of 5. The unconstrainedproblem is then
solved from the previous best point.
The sequence of unconstrained optimizations will
continueuntil convergence. The user also has a
parameter to limit howbig the penalty multiplier
can get. The limit is rpmax and itsdefault value
is 1.0E10
11
(No Transcript)
12
Example of impact of rp
Min F(X) X1 X2s.t. g1(X)-2X1-2X2lt0
g2(X)86X1X12-X2lt0
Vary rp from .05, .1, 1.0 to see effect on new
combined objectivepenalty pseudo objective
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
ADS Exterior Penalty istrat 1
18
Exterior Penalty Choices 1, 2, 3
19
(No Transcript)
20
ADS Parameters
Number of Iterations itmax enables us to limit
number ofunconstrained problems solved.
Parameter is called MaxStrategy iterations in
iSIGHT GUI
Default in iSIGHT is a 1,3,3 or BFGS
Note We can run a unconstrained Fletcher
Reevespicking a 1,1,3 by using a
SetTechniqueOption call in prologue.
21
Select Exterior Penalty
22
Exterior Penalty is an ADS Technique
Max Strategy controls the number of constrained
subproblemsto solve Max number of iterations
controls number of iterationsper subproblem
23
Key Penalty Parameters in ADS
rp initial penalty rmult multiplier of rp
for each newunconstrained problem rpmax is
largest value of rp
24
Lab

• Objectives
• Apply exterior penalty beam problem.
• Use Fletcher Reeves and BFGS for search
  directions.
• Reinforce the importance of scaling.
• Insure you appreciate the field Internal in db
  file.
• Lab taken from Numerical Optimization Techniques
  in Engineering Design by Vanderplaats (see
  description on next page.
• Initial design is infeasible.
• Problem coded in Tcl, load beam_Tcl.desc
• Remember to consider multiple starting points.

25
(No Transcript)
26
Lab

• Task 1 Using Exterior Penalty
• Create an optimization plan called EPenalty. The
  plan should be made up of a single step,
  ExteriorPenalty.
• Change the print level to 3552. Set up the
  iSIGHT log to filter out all message to the log
  window and log file to be only All other types
  and allow 30000 lines to be displayed (hint Edit
  Properties from main tool bar).
• Execute the optimization and answer the following
  questions
• What is the optimum objective value?
• Is it feasible?
• How many function evaluations did it take?
• Which type of optimizer (iopt) did ADS use? Look
  at log.
• How many subproblems were run?
• What was the value of rp for the first
  subproblem?
• What was the value of rp for the last subproblem?
• In the Look at the initial values of the design
  variables, constraints and objectives. Look at
  the value of the gradients for the objective and
  constraints for the first iteration. Note,that
  they are of different orders of magnitudes. How
  do you suggest changing theformulation?

27
Lab continued

• Task 2
• Scaling is probably the most important thing
  that you can do to aid an optimizer. Reload the
  description file. Set up the exterior penalty in
  the same manner as the last run.
• Set the run mode to Single. Execute the task a
  single time in order to get the parameters
  (inputs and outputs) initialized. Bring up the
  Parameters Form. Use the Scaling to set the
  scalefactors for the objective and the design
  variables to the current value (see figure
  below.) This will set design variables (b1-b5 to
  be scaled by 5 h1-h5 to be scaled by 40 and the
  vol objective to be scaled by 100000. Save your
  changes.
• Rerun the task. What is the optimal objective
  value? How many function evaluations did
  it take? How many subproblems were run?

28
Lab

• Task 3
• In Task 1 of this lab, you used the default
  optimizer of BFGS. In this lab, use the exact
  same set up that you did for Task 1 (No scaling)
  except customize your setup to use a
  FletcherReeves to calculate the search direction
  (i.e. iopt 1). Execute the problem and answer
  the following questions.
• What is the optimal value?
• How many function evaluations did it take?
• Bring up solution monitor and open the datafile
  for your db file. Scroll over to the field called
  Internal. Recall that a value of 1 indicates a
  gradient calculation. Look at how many of the
  total evaluations are for gradients. (This is why
  automatic differentiationis such a valuable new
  technique. Ask Therese when it is coming ??