LINEAR PROGRAMMING

1
LINEAR PROGRAMMING
2

• A certain wide class of practical problems
  appears to be just beyond the range of modern
  computing machinery. These problems occur in
  everyday life they run the gamut from some very
  simple situations that confront an individual to
  those connected with the national economy as a
  whole. Typically, these problems involve a
  complex of different activities in which one
  wishes to know which activities to emphasize in
  order to carry out desired objectives under known
  limitations.

George B. Dantzig, 1948
3
LP History

• George Dantzig, 1947
• Introduced LP and recognized it as more than a
  conceptual tool Computing answers important.
• Invented primal simplex algorithm.
• First LP solved Laderman, 9 cons., 77 vars.,
  120 MAN-DAYS.
• First computer code 1951
• LP used commercially Early 60s
• Powerful mainframe codes introduced Early 70s
• Computational progress stagnated Mid 80s
• Remarkable progress last 15 years (PCs, new
  computer science and mathematics)

4
Diet Problem

• Bob wants to plan a nutritious diet, but he is on
  a limited budget, so he wants to spend as little
  money as possible. His nutritional requirements
  are as follows

• 2000 kcal
• 55 g protein
• 800 mg calcium

From Linear Programming, by Vasek Chvátal
5
Diet Problem
Nutritional values
Bob is considering the following foods
6
Diet Problem
Variables
We can represent the number of servings of each
type of food in the diet by the variables
x1 servings of oatmeal x2 servings of chicken x3
servings of eggs x4 servings of milk x5 servings
of cherry pie x6 servings of pork and beans
7
Diet Problem
Nutritional values
Bob is considering the following foods
x1 x2 x3 x4 x5 x6
KCAL constraint 110x1 205x2 160x3
160x4 420x5 260x6 ? 2000 (110x1 kcals
in oatmeal)
8
Diet Problem
LP formulation
Minimize
Cost
0.3x1 2.40x2 1.30x3 0.90x4 2.0x5 1.9x6

subject to
Nutritional requirements
110x1 205x2 160x3 160x4 420x5
260x6 ? 2000 4x1 32x2 13x3
8x4 4x5 14x6 ? 55 2x1
12x2 54x3 285x4 22x5 80x6 ? 800
Bounds
9
Diet Problem
Solution
When we solve the preceding LP (using CPLEX, of
course) we get a solution value of 6.71, which
is achieved with the following menu
14.24 servings of oatmeal 0 servings of
chicken 0 servings of eggs 2.71
servings of milk 0 servings of cherry pie
0 servings of pork and beans
10

• A linear program (LP) is an
• optimization problem of the form

11
Solving Linear Programs
12
y
A Graphical Solution
Maximize 0.90 x 0.73 y (OBJECTIVE) Subject
To Constraint 1 0.42 x 0.07 y lt 4200000
Constraint 2 0.13 x 0.39 y lt 3900000
Constraint 3 0.35 x 0.44 y lt 7000000
x gt 0
y gt 0
x
(0,0)
13
The Simplex Algorithm
A graphical representation
We now look at a graphical representation of the
simplex method as it solves the following problem
Maximize 3x1 2x2 2x3 Subject to x1
x3 ? 8 x1
x2 ? 7 x1 2x2
? 12 x1, x2, x3
? 0
14
The Simplex Algorithm
Maximize z 3x1 2x2 2x3
x3
Optimal!
(0,0,8)
(0,6,8)
(2,5,6)
(0,6,0)
x2
(2,5,0)
(7,0,1)
(7,0,0)
x1
15
A Real LP Sample Run
Log started (V8.2.0a1) Tue Apr 8 215946
2003 Problem name supply_chain.lp.gz Constraints
10810259 Variables
19916187 Reduced LP has 3140439 rows, and
7314332 variables. Presolve time 339.43
sec. Iteration log . . . Iteration 0
Infeasibility 1534390803.049594 Iterati
on 1247301 Infeasibility
437397607.938387 ... Iteration 2348954
Infeasibility 10965.599999 Elapsed
time 3776.48 sec. (2349000 iterations) Iteratio
n 2349222 Objective 1775629743606097400.
000000 ... Elapsed time 5396.65 sec. (2687000
iterations) Iteration 2687616 Objective
1403833763253068800.000000 Removing shift
(10). Primal simplex - Optimal Objective
1.4038337633e18 Solution time 5410.00 sec.
Iterations 2687616 (2348954)
16
MIXED INTEGERPROGRAMMING
17
Mixed-Integer Programming (MIP)( linear
programming restriction that some MIP or all
variables must take on integer values.)
18
Manufacturing Cereal
19
Cereal ProblemWith setup cost
Minimize Total_Cost 283 x1 266 x2 187 x3
188 x4 182 x5 181 x6 200 y1
250 y2 400 y3 350 y4 800 y5 1000
y6 Subject To Boxes x1 x2 x3 x4 x5 x6
200 S1 - 50 y1 x1 lt 0 S2 - 70 y2
x2 lt 0 S3 - 40 y3 x3 lt 0 S4 - 70 y4
x4 lt 0 S5 - 100 y5 x5 lt 0 S6 - 90
y6 x6 lt 0 Bounds x1, x2, x3, x4, x5, x6 gt 0
y1, y2, y3, y4, y5, y5 0 or 1 (binary) End
20
Cereal ProblemWith setup cost Solution log
Problem 'cereal.lp' read. Nodes
Cuts/ Node
Left Objective IInf Best Integer Best
Node ItCnt 0 0 38220.0000 1
38220.0000 6 0 0
0 38520.0000 38220.0000
6 38410.0000 2
38520.0000 Cuts 4 8
38495.7143 2 38520.0000 Cuts 4
12 0 0 0
38510.0000 38495.7143 12
cutoff 38510.0000 Flowcuts 2
13 Cover cuts applied 1 Flow cuts applied
5 Gomory fractional cuts applied 2 Integer
optimal solution Objective
3.8510000000e004 Solution time 0.01 sec.
Iterations 13 Nodes 0 Variable Name
Solution Value x3
40.000000 x4
60.000000 x5
100.000000 y3
1.000000 y4
1.000000 y5
1.000000 All other variables in the range 1-12
are zero.
21
y
objective
feasible solutions
x
Linear Program
22
y
objective
feasible solutions
x
Integer Program
23
Solving MIPs Branch and Bound

• Consider the following integer program

Maximize x y 2z Subject to 7x 2y 3z
? 36 5x 4y 7z ? 42
2x 3y 5z ? 28 x,
y, z ? 0, integer
(IP0)
24
Branch Bound Example
Best IP value
Best IP solution
25
(No Transcript)
26
Diet Problem
The Pill Salesman
A pill salesman offers Bob energy, protein, and
calcium pills to fulfill his nutritional needs.
We will represent the costs of each of the pills
as follows
y1 cost (in dollars) of 1 kcal energy pill y2
cost (in dollars) of 1 g protein pill y3 cost (in
dollars) of 1mg calcium pill
27
Diet Problem
LP constraints
Nutritional requirements
110x1 205x2 160x3 160x4 420x5
260x6 ? 2000 y1 4x1 32x2 13x3
8x4 4x5 14x6 ? 55 y2
2x1 12x2 54x3 285x4 22x5
80x6 ? 800 y3
28
Diet Problem
LP constraints
Minimize
Cost
0.3x1 2.40x2 1.30x3 0.90x4 2.0x5 1.9x6

Nutritional requirements
110x1 205x2 160x3 160x4 420x5
260x6 ? 2000 y1 kcal 4x1 32x2
13x3 8x4 4x5 14x6 ? 55
y2 protein 2x1 12x2 54x3 285x4
22x5 80x6 ? 800 y3 calcium
x1 servings of oatmeal The cost of the
nutrients in one serving of oatmeal shouldnt
exceed the cost of just buying one serving of
oatmeal 110y1 4y2 2y3 ? 0.3 (4 y2
cost of protein in oatmeal)
29
Diet Problem
The salesmans requirements
The pill salesman wants to make as much money as
possible, given Bobs constraints. He knows Bob
wants 2000 kcal, 55g protein, and 800 mg calcium,
so his problem is as follows
Maximize 2000y1 55y2 800y3 Subject to
110y1 4y2 2y3 ? 0.3
205y1 32y2 12y3 ? 2.4
160y1 13y2 54y3 ? 1.3
160y1 8y2 285y3 ? 0.9
420y1 4y2 22y3 ? 2.0
260y1 14y2 80y3 ? 1.9
y1, y2, y3 ? 0
30
Diet Problem
Solution
Solving this LP gives us the following pill
prices
0.27 for 1 kcal energy pill 0.00 for 1 g
protein pill 0.16 for 1mg calcium pill
Total cost 0.27 (2000) 0.16 (800)
6.71
THE SAME AS THE LOWEST COST DIET!
31
Diet Problem
Weak Duality Cost of any feas. diet ? Price of
any feas. pills Strong Duality (Von Neumann,
1947) Optimal cost of diet Optimal cost of pills
32
Diet Problem
MORAL The values of the dual variables, which
are always available as part of LP solution,
equal the marginal value of corresponding
resources.
33
The Simplex Algorithm

• Consider the following linear program

Maximize 5x1 4x2 3x3 Subject to 2x1
3x2 x3 ? 5 4x1 x2
2x3 ? 11 3x1 4x2 2x3 ?
8 x1, x2, x3 ? 0.
34
The Simplex Algorithm
Slack Variables
We add slack variables to each constraint to get
equations
Maximize 5x1 4x2 3x3 Subject to 2x1
3x2 x3 x4 5
4x1 x2 2x3 x5
11 3x1 4x2 2x3
x6 8 x1,
x2, x3, x4, x5, x6 ? 0.
35
The Simplex Algorithm
Getting started
Rewriting we can express the slack variables and
the objective, denoted z, in terms of the
original variables. Setting x1x2x30, we
easily read off the solution x45, x511, x68,
z0.
z 5x1 4x2 3x3
x4 5 - 2x1 - 3x2 - x3
x5 11 - 4x1 - x2 - 2x3 x6
8 - 3x1 - 4x2 - 2x3
36
The Simplex Algorithm
1st Iteration
We want to increase the value of z from its
current value of 0. Clearly, increasing the
value of x1 will increase the value of z. By how
much can we increase it?
z 5x1 4x2 3x3
x4 5 - 2x1 - 3x2 - x3
x5 11 - 4x1 - x2 - 2x3 x6
8 - 3x1 - 4x2 - 2x3
x4 ? 0 ? 5 2x1 ? 0 ? 5 ? 2x1 ? x1 ? 5/2 x5
? 0 ? 11 4x1 ? 0 ? 11 ? 4x1 ? x1 ? 11/4 x6 ? 0
? 8 3x1 ? 0 ? 8 ? 3x1 ? x1 ? 8/3
37
The Simplex Algorithm
2nd Iteration
Now we see that increasing the value of x3 will
improve the value of z.
z 25/2 - 7/2x2 1/2x3 -
1/2x4 x1 5/2 - 3/2x2 -
1/2x3 - 1/2x4 x5 1
5x2 2x4 x6
1/2 1/2x2 - 1/2x3 3/2x4
x1 ? 0 ? 5/2 1/2x3 ? 0 ? 5/2 ? 1/2x3 ? x3 ?
5 x5 ? 0 ? x3 unbounded x6 ? 0 ? 1/2 1/2x3
? 0 ? 1/2 ? 1/2x3 ? x1 ? 1
38
The Simplex Algorithm
Optimal!!!
z 13 - 3x2 - x4 -
x6 x1 2 - 2x2 - 2x4
x6 x5 1 5x2 2x4
x3 1 x2 3x4 - 2x6
39
Barrier Algorithms
Simplex solution path
Optimum