ESI 4313 Operations Research 2

1
ESI 4313Operations Research 2

• Nonlinear Programming
• One Dimensional Problems

2
Recap

• Nonlinear programming
• Modeling
• Structure of optimal solutions
• Local and global optimal solutions
• Role of convexity/concavity
• Can guarantee that a local optimal solution is
  globally optimal

3
Recap

• One-dimensional optimization
• Recognize convexity and concavity
• Use derivative information to characterize local
  optima
• Find all local optima analytically/ graphically

4
Whats next?

• One-dimensional optimization
• Numerically find a local optimum for a
  one-dimensional optimization problem
• Multi-dimensional optimization
• Recognize convexity and concavity
• Use derivative information to characterize local
  optima
• Numerically find a local optimum for a
  one-dimensional optimization problem

5
Numerical optimization

• If we cannot solve the problem analytically, how
  do we find local optima numerically?
• Many numerical methods use the theoretical
  characterization of local optima via the first
  and/or second order derivatives
• First derivative will tell us if we have obtained
  a stationary point
• Second derivative will (usually) tell us the
  nature of this stationary point

6
Numerical optimization

• We will make some regularity assumptions
• Objective function f is continuous and
  differentiable
• Feasible region is of the form a,b
• We will assume that we want to maximize f(x)
• Otherwise, simply replace the objective function
  by f
• We are interested in finding a local maximum

7
Numerical optimization

• The two algorithms that we will consider in more
  detail are
• Bisection
• Golden section search (treated only in the text)
• The bisection algorithm was originally developed
  for finding the root of a function
• that is, for finding a value of x where the
  function of interest takes on the value 0

8
Numerical optimization

• Consider the optimization problem
• We can find a local maximum by finding a root in
  a,b of f(x) 0

9
Bisection

• We assume that we have a pair of points in a,b,
  say x lt x, such that
• f(x) gt 0
• f(x) lt 0
• That is, the interval x,x contains a root of
  f(x) 0
• Moreover, this root will correspond to a local
  maximum!!

10
Example

• Consider the optimization problem

11
Example

• We wish to find a root of the equation

12
Example

• Clearly,
• f(-1½) 4.25 gt 0
• f(0) -1 lt 0
• So we can choose x-1½ and x0

13
Bisection

• The goal is to move x and x closer together
  while maintaining the property that f(x) gt 0
  and f(x) lt 0
• Idea
• Pick the midpoint of x and x, i.e.,
• x ½x ½x
• Evaluate f(x)
• If f(x) gt 0, replace x by x
• If f(x) lt 0, replace x by x

14
Example
x -0.75 f(x) -1.1875 x -0.75
x -1.125 f(x) 0.828 x -1.125
Etc.
15
Bisection

• We continue the process until
• x and x are close enough
• that is, we stop when x -x lt ? for some
  prespecified tolerance ?
• or, either f(x) or f(x) is small enough
• that is, we stop when f(x) lt ? or f(x)
  lt ? for some prespecified tolerance ?
• How many iterations (function evaluations) are
  needed to find an approximate local maximum?