Lecture 9

1
Lecture 9 Nonlinear Programming Models

• Topics
• Convex sets and convex programming
• First-order optimality conditions
• Examples
• Problem classes

2
General NLP
Minimize f(x)
s.t. gi(x) (?, ?, ) bi, i 1,,m
x (x1,,xn)T is the n-dimensional vector of
decision variables f (x) is the objective
function gi(x) are the constraint functions bi
are fixed known constants
3
Convex Sets
Definition A set S ? ?n is convex if every point
on the line segment connecting any two points x1,
x2 Î S is also in S. Mathematically, this is
equivalent to x0 lx1 (1l)x2 Î S for all l
such 0 l 1.
?
x1
x2
x1
x1
?
?
x2
?
x2
?
?
4
(Nonconvex) Feasible Region
S (x1, x2) (0.5x1 0.6)x2 1 2(x1)2
3(x2)2 27 x1, x2 0
5
Convex Sets and Optimization
Let S x Î ?n gi(x) bi, i 1,,m Fact
If gi(x) is a convex function for each i 1,,m
then S is a convex set.
Convex Programming Theorem Let x ? ?n and let f
(x) be a convex function defined over a convex
constraint set S. If a finite solution exists to
the problem Minimizef (x) x Î S then all
local optima are global optima. If f (x) is
strictly convex, the optimum is unique.
6
Convex Programming
Max f (x1,,xn) s.t. gi(x1,,xn) bi
i 1,,m x1 ³ 0,,xn ³ 0 is a convex program
if f is concave and each gi is convex.
Min f (x1,,xn) s.t. gi(x1,,xn) bi
i 1,,m x1 ³ 0,,xn ³ 0 is a convex program
if f is convex and each gi is convex.

7
Linearly Constrained Convex Function with Unique
Global Maximum
Maximize f (x) (x1 2)2 (x2 2)2 subject
to 3x1 2x2 6 x1 x2 3 x1
x2 7 2x1 3x2 4
8
(Nonconvex) Optimization Problem
9
First-Order Optimality Conditions

• Minimize f (x) gi(x) ? bi, i 1,,m
• Lagrangian

10
Importance of Convex Programs
Commercial optimization software cannot guarantee
that a solution is globally optimal to a
nonconvex program.
NLP algorithms try to find a point where the
gradient of the Lagrangian function is zero a
stationary point and complementary slackness
holds. Given L(x,m) f(x) m(g(x) b) we
want ?L(x,m) ?f(x) m?g(x) 0 m(g(x) b)
0 g(x) b 0, m ³ 0
For a convex program, all local solutions are
global optima.
11
Example Cylinder Design
We want to build a cylinder (with a top and a
bottom) of maximum volume such that its surface
area is no more than s units.
12
Solution by Substitution
s - 2pr 2
s - 2pr 2
rs
Volume V pr2
- pr 3
?


h
p
2
2
r
2pr
1/2
dV
s
s
s
1/2
0 ?
-
r (
)
h

r
2(
)
p
2pr
p
dr
6
6
s
3/2
s
s
1/2
1/2
(
)
V pr 2h 2p
r (
)
)
h 2(
p
6
p
p
6
6
Is this a global optimal solution?
13
Test for Convexity
dV(r)
s
d2V(r )
rs
-6pr
- 3pr 2 ?

- pr 3 ?
V(r )
dr
2
2
dr 2
2
d
V
0 for all r ³ 0
dr 2
Thus V(r ) is concave on r ³ 0 so the solution is
a global maximum.
14
Advertising (with Diminishing Returns)

• A company wants to advertise in two regions.
• The marketing department says that if x1 is
  spent in region 1, sales volume will be 6(x1)1/2.
• If x2 is spent in region 2, sales volume will be
  4(x2)1/2.
• The advertising budget is 100.

15
Excel Add-in Solution
16
Portfolio Selection with Risky Assets (Markowitz)

• Suppose that we may invest in (up to) n stocks.
• Investors worry about (1) expected gain (2) risk.

Let mj expected return sjj variance of return
We are also concerned with the covariance
terms sij cov(ri, rj) If sij gt 0 then
returns on i and j are positively correlated. If
sij lt 0 returns are negatively correlated.
17
Decision Variables xj of shares of stock j
purchased
n j 1
R(x) å mjxj
Expected return of the portfolio
n j 1
n i 1
V(x) å å sijxixj
Variance (measure of risk)
18
(No Transcript)
19
(No Transcript)
20
Hanging Chain with Rigid Links
What is equilibrium shape of chain?
21
Objective Minimize chains potential
energy Assuming that the center of the mass of
each link is at the center of the link. This is
equivalent to minimizing
22
Is a local optimum guaranteed to be a global
optimum?
No!
23
Direct Current Network
Problem Determine the current flows I1, I2,,I7
so that the total content is minimized Content G(
I) ?0I v(i)di for I 0 and G(I) ?0I
v(i)di for I lt 0
24
Solution Approach
Electrical Engineering Use Kirchoffs laws to
find currents when power source is
given. Operations Research Optimize performance
measure in network taking flow balance into
account.
Linear resistor Voltage, v(I ) IR Content
function, G(I ) I 2R/2
Battery Voltage, v(I ) E Content function,
G(I ) EI
25
Network Flow Model
Minimize Z 100I1 5I22 5I32 10I42
10I52 subject to I1 I2 0, I2 I3 I4
0, I5 I6 0, I5 I7 0, I3 I6 I7
0, I1 I6 0
Solution I1 I2 50/9, I3 40/9, I4 I5
10/9, I6 50/9, I7 10/9
26
NLP Problem Classes

• Constrained vs. unconstrained
• Convex programming problem
• Quadratic programming problem
• f (x) a cTx ½ xTQx, Q ? 0
• Separable programming problem
• f (x) ?j1,n fj(xj)
• Geometric programming problem
• g(x) ?t1,T ctPt(x), Pt(x) (x1at1) . . .
  (xnatn), xj gt 0
• Equality constrained problems

27
What You Should Know About Nonlinear Programming

• How to identify a convex program.
• How to write out the first-order optimality
  conditions.
• The difference between a local and global
  solution.
• How to classify problems.