Optimization Linear Programming and Simplex Method

1
OptimizationLinear Programming andSimplex
Method
2
Optimization Methods

• One-Dimensional Unconstrained Optimization
• Golden-Section Search
• Quadratic Interpolation
• Newton's Method
• Multi-Dimensional Unconstrained Optimization
• Non-gradient or direct methods
• Gradient methods
• Linear Programming (Constrained)
• Graphical Solution
• Simplex Method

3
Standard Form Linear Programming Problem

• Basic linear programming (LP) problem consists of
  two major parts
• The objective function
• A set of constraints
• For maximization problem, the objective function
  is generally expressed as

cj payoff of each unit of the jth activity that
is undertaken xj magnitude of the jth
activity Z total payoff due to the total number
of activities
4
Standard Form Linear Programming Problem

• The constraints can be represented generally as
• where
• aij amount of the ith resource that is consumed
  for each unit of the jth activity
• bi amount of the ith resource that is
  available
• The general second type of constraints specifies
  that all activities must have a positive value.
  i.e,

5
Problem

• How many tonnes of regular and premium gas to
  produce in order to maximize weekly profit?

6
Developing LP Formulation

• Let x1 1 tonne of regular gas
• x2 1 tonne of premium gas
• Maximize Z 150x1 175x2
• subject to
• 7x1 11x2 77 (material constraint) (1)
• 10x1 8x2 80 (time constraint) (2)
• x1 9 (regular gas storage) (3)
• x2 6 (premium gas storage) (4)
• x1, x2 0 (positivity) (5, 6)

7
Graphical Solution

• Feasible solution space (the shaded area)
  contains all points that obey the constraints or
  points that represent the feasible solutions.
• Points located outside of this area are known as
  infeasible solution.

8
Graphical Solution

• We can adjust the line representing the objective
  function Z 150x2 175x2 to locate the optimum
  point.
• One of the corner points (points where two or
  more lines intersect) will be an optimum point.
• Corner points can be feasible or infeasible

9
Graphical Solution

• Some insights
• Increasing storages ( constraints (3) and (4))
  does not improve profit.
• Raising either materials or production time can
  improve profit.

10

• Possible outcomes that can generally be obtained
  in a LP problem
• Unique solution. The maximum objective function
  intersects a single point.
• Alternate solutions. Problem has an infinite
  number of optima corresponding to a line segment.
• No feasible solution.
• Unbounded problems. Problem is under-constrained
  and therefore open-ended.

11

• Unique solution represented by a corner point
• Alternate solutions represented by points
  on one of the constraint line
• (b) No feasible solution
• (c) Unbounded problems

12

• Key properties of LP that drives the design of
  Simplex Method (An algorithm for solving LP
  problems)
• The optimum point is always at a feasible corner
  point. (Why?)
• We only need to check the corner points.
• If a corner point feasible solution has an
  objective function value that is better that or
  equal to all adjacent corner point feasible
  solutions, then it is optimal.
• We don't necessarily need to search all corner
  points.

13

• Key properties of LP that drives the design of
  Simplex Method
• There are finite number of corner point feasible
  solutions.
• Any method that checks only corner points will
  terminate eventually.

14

• Basic idea behind the Simplex method
• Starting at a feasible corner point solution.
• Repeatedly move to a better adjacent corner point
  feasible solution until an optimum point is found.

15
Finding Corner Points Algebraically

• How to find corner points algebraically?
• How to locate adjacent corner points?
• How to decide which adjacent point to move to
  next?
• Simplex method offers a neat solution (involves
  the use of slack variables) to address these
  questions. ?

16

• First, constraint equations are reformulated as
  equalities by introducing slack variables
  variables that measures how much of a constrained
  resource is available.

Slack variables
Si gt 0 means the corresponding resource is not
fully consumed. Si lt 0 means the corresponding
resource is over-consumed.
17
Augmented System

• Adding slack variables results in an augmented
  system of linear equations which is under
  specified (has more unknowns than equations).
• 6 unknowns ( 2 original variables 4 slack
  variables)
• 4 equations
• In general, if a LP problem has n variables and m
  constraints, the resulting augmented system will
  have a total of (n original variables m slack
  variables) and m equations.

Continue
18
Augmented System

• To solve an augmented system with
• 6 unknowns ( 2 original variables 4 slack
  variables)
• 4 equations
• We need to set 2 of the variables constants
  before we can solve the system of equations.
• e.g., Setting x1 0, x2 0 yields

Continue
19
Augmented System

• Solving the resulting system yields x10, x20,
  S177, S280, S39, S46, which represent one of
  the augmented solutions values of all
  (original slack) variables are given.
• For this example, the values tell us that if we
  don't produce any regular (x1) or premium (x2)
  gas, we would have the following amount of
  unconsumed resources
• 77 m3 of raw gas,
• 80 production hours
• 9 tonne of regular gas storage
• 6 tonne of premium gas storage

20
Augmented System

• An augmented solution which corresponds to a
  corner point is also known as a basic solution.
• The augmented solution x10, x20, S177, S280,
  S39, S46 is a basic solution. It corresponds to
  corner point A.
• In addition, it is a basic feasible solution
  because all variables 0.
• So far, you know how to find an augmented
  solution.
• How should we set the variables so that the
  augmented solution is a basic solution?

Continue
21
Characteristics of Slack Variables

• The equations (or constraints) have a one-to-one
  relationship with the slack variables.
• For example, S1 relates to the 1st equation, S4
  relates to the 4th equation.
• If a point is on the line 7x1 11x2 77 (line
  labeled "1"), what's the value of S1?
• If a point is on the line 10x1 8x2 80 (line
  labeled "2"), which variable has the value zero?
• At the corner point D, which variables have the
  value zero?

Show Graph Equations
22
Characteristics of Slack Variables

• Setting two variables to zero has the effect of
  "selecting two lines", and solving the resulting
  system means finding the intersecting point (a
  corner point) of the selected lines.

For example, let S10, S20. Then the system
becomes
The value of x1 and x2 of this system is the
coordinate of the corner point C (the point where
line "1" and line "2" intersect.)
23
Finding Corner Points Algebraically

• We now know how to find the corner points.
• Next
• How to find an adjacent corner point?
• How to decide which adjacent corner point to move
  to?

24
Basic and Non-Basic Variables
Do you see a pattern between any pair of
adjacent corner points?
Variables in which their values are set to zero
are called non-basic variables. Remember
Non-basic, variable set to zero, corresponding
constraint is active
25
Basic and Non-Basic Variables

• If two points are adjacent corner points, then
• Their basic sets are identical except for one
  member.
• Their non-basic sets are identical except for one
  member.
• The reverse of the above statement is not always
  true. (e.g., F and A)
• Questions
• To find an adjacent corner point, can we just
  switch one variable between the basic and
  non-basic sets?
• How should we decide which corner point to move
  to next?

26
Basic and Non-Basic Variables

• How do we make sure the following conditions are
  satisfied?
• The corner points are adjacent.
• The corner points are feasible.
• The new corner point gives better value for the
  objective function than the current corner point.

27
Which variables to switch?

• All three conditions from the previous slide can
  be satisfied if the following steps are taken
• Selecting an entering basic variable a
  non-basic variable that will increase the
  objective function value the most if allowed to
  take on a positive value. Move this variable from
  the non-basic set to the basic set.
• Finding the leaving basic variable a basic
  variable that is forced to a value zero first
  after allowing the entering basic variable to
  increase. Move the variable that is forced to
  zero from the basic set to the non-basic set.

28
Which variables to switch?

• Suppose we are currently at point A.
• Basic set S1, S2, S3, S4 , non-basic set
  x1, x2
• Which variable should be the entering basic
  variable?
• Which variable should be the leaving basic
  variable?

29
Which variables to switch?

• x2 has a larger coefficient in the objective
  function, so we should make x2 the entering basic
  variable. (x1 remains a non-basic variable and
  has the value zero)
• If we keep increasing x2, other basic variables
  will decrease. Which basic variable will become
  zero first?

30
Which variables to switch?
Show Graph Equations
Minimum ratio test

• If we take the ratio of the RHS value to the
  coefficient of the entering basic variable (x2),
  the smallest non-negative ratio can tell us which
  basic variable will reach zero first if we keep
  increasing the value of x2.
• So S4 should be made the leaving basic variable.

31
Which variables to switch?

• So setting x10, S40 and solve the system of
  equations would yield an augmented solution
  correspond to point E.
• Solving the system of equations at every corner
  point is not very efficient. Simplex method
  offers an efficient way to compute the value of
  each variable by updating the equations
  incrementally.

32
F
4
E
D
1
C
3
5
2
6
A
B