ENGINEERING OPTIMIZATION

1
ENGINEERING OPTIMIZATION Methods and Applications
A. Ravindran, K. M. Ragsdell, G. V. Reklaitis
Book Review
2
Chapter 5 Constrained Optimality Criteria
Part 1 Ferhat Dikbiyik Part 2Yi Zhang
Review Session July 2, 2010
3
Constraints Good guys or bad guys?
4
Constraints Good guys or bad guys?
reduces the region in which we search for optimum.
5
Constraints Good guys or bad guys?
makes optimization process very complicated
6
(No Transcript)
7
Outline of Part 1

• Equality-Constrained Problems
• Lagrange Multipliers
• Economic Interpretation of Lagrange Multipliers
• Kuhn-Tucker Conditions
• Kuhn-Tucker Theorem

8
Outline of Part 1

• Equality-Constrained Problems
• Lagrange Multipliers
• Economic Interpretation of Lagrange Multipliers
• Kuhn-Tucker Conditions
• Kuhn-Tucker Theorem

9
Equality-Constrained Problems
GOAL
solving the problem as an unconstrained problem
by explicitly eliminating K independent variables
using the equality constraints
10
Example 5.1
11
What if?
12
Outline of Part 1

• Equality-Constrained Problems
• Lagrange Multipliers
• Economic Interpretation of Lagrange Multipliers
• Kuhn-Tucker Conditions
• Kuhn-Tucker Theorem

13
Lagrange Multipliers
Converting constrained problem to an
unconstrained problem with help of certain
unspecified parameters known as Lagrange
Multipliers
14
Lagrange Multipliers
Lagrange function
15
Lagrange Multipliers
Lagrange multiplier
16
Example 5.2
17
(No Transcript)
18
Test whether the stationary point corresponds to
a minimum
positive definite
19
(No Transcript)
20
Example 5.3
21
(No Transcript)
22
(No Transcript)
23

max
positive definite
negative definite
24
Outline of Part 1

• Equality-Constrained Problems
• Lagrange Multipliers
• Economic Interpretation of Lagrange Multipliers
• Kuhn-Tucker Conditions
• Kuhn-Tucker Theorem

25
Economic Interpretation of Lagrange Multipliers
The Lagrange multipliers have an important
economic interpretation as shadow prices of the
constraints, and their optimal values are very
useful in sensitivity analysis.
26
Outline of Part 1

• Equality-Constrained Problems
• Lagrange Multipliers
• Economic Interpretation of Lagrange Multipliers
• Kuhn-Tucker Conditions
• Kuhn-Tucker Theorem

27
Kuhn-Tucker Conditions
28
NLP problem
29
Kuhn-Tucker conditions (aka Kuhn-Tucker Problem)
30
Example 5.4
31
Example 5.4
32
Example 5.4
33
Outline of Part 1

• Equality-Constrained Problems
• Lagrange Multipliers
• Economic Interpretation of Lagrange Multipliers
• Kuhn-Tucker Conditions
• Kuhn-Tucker Theorem

34
Kuhn-Tucker Theorems

1. Kuhn Tucker Necessity Theorem
2. Kuhn Tucker Sufficient Theorem

35
Kuhn-Tucker Necessity Theorem
36
Kuhn-Tucker Necessity Theorem

• Let
• f, g, and h be differentiable functions x be a
  feasible solution to the NLP problem.
• and for
  k1,.,K are linearly independent at the optimum
• If x is an optimal solution to the NLP problem,
  then there exists a (u, v) such that (x,u,
  v) solves the KTP given by KTC.

Constraint qualification
! Hard to verify, since it requires that the
optimum solution be known beforehand !
37
Kuhn-Tucker Necessity Theorem

• For certain special NLP problems, the constraint
  qualification is satisfied
• When all the inequality and equality constraints
  are linear
• When all the inequality constraints are concave
  functions and equality constraints are linear

! When the constraint qualification is not met at
the optimum, there may not exist a solution to
the KTP
38
Example 5.5
x (1, 0)
39
Example 5.5
x (1, 0)
No Kuhn-Tucker point at the optimum
40
Kuhn-Tucker Necessity Theorem
Given a feasible point that satisfies the
constraint qualification
not optimal
optimal
If it does not satisfy the KTCs
If it does satisfy the KTCs
41
Example 5.6
42
Kuhn-Tucker Sufficiency Theorem

• Let
• f(x) be convex
• the inequality constraints gj(x) for j1,,J be
  all concave function
• the equality constraints hk(x) for k1,,K be
  linear

If there exists a solution (x,u,v) that
satisfies KTCs, then x is an optimal solution
43
Example 5.4

• f(x) be convex
• the inequality constraints gj(x) for j1,,J be
  all concave function
• the equality constraints hk(x) for k1,,K be
  linear

44
Example 5.4

• f(x) be convex

semi-definite
45
Example 5.4

• f(x) be convex
• the inequality constraints gj(x) for j1,,J be
  all concave function

v
g1(x) linear, hence both convex and concave
negative definite
46
Example 5.4

• f(x) be convex
• the inequality constraints gj(x) for j1,,J be
  all concave function
• the equality constraints hk(x) for k1,,K be
  linear

v
47
Remarks

• For practical problems, the constraint
  qualification
• will generally hold. If the functions are
  differentiable, a KuhnTucker point is a possible
  candidate for the optimum. Hence, many of the
  NLP methods attempt to converge to a KuhnTucker
  point.

48
Remarks

• When the sufficiency conditions of Theorem 5.2
  hold, a KuhnTucker point automatically becomes
  the global minimum. Unfortunately, the
  sufficiency conditions are difficult to verify,
  and often practical problems may not possess
  these nice properties. Note that the presence of
  one nonlinear equality constraint is enough to
  violate the assumptions of Theorem 5.2

49
Remarks

• The sufficiency conditions of Theorem 5.2 have
  been generalized further to nonconvex inequality
  constraints, nonconvex objectives, and nonlinear
  equality constraints. These use generalizations
  of convex functions such as quasi-convex and
  pseudoconvex functions