Title: UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2004
1 UMass Lowell Computer Science 91.503 Analysis
of Algorithms Prof. Karen Daniels Fall, 2004
- Lecture 9
- Wednesday, 11/17/04
- Linear Programming
2Overview
- Motivation Basics
- Standard Slack Forms
- Formulating Problems as Linear Programs
- Simplex Algorithm
- Duality
- Initial Basic Feasible Solution
- Literature Case Study
3Motivation Basics
4Motivation A Political Problem
Goal Win election by winning majority of votes
in each region.
100,000 voters
200,000 voters
50,000 voters
Thousands of voters who could be won with 1,000
of ads
Subgoal Win majority of votes in each region
while minimizing advertising cost.
source 91.503 textbook Cormen et al.
5Motivation A Political Problem (continued)
Thousands of voters representing majority.
urban
suburban
rural
source 91.503 textbook Cormen et al.
6General Linear Programs
real numbers
Linear function
variables
Linear constraints
Linear inequalities
source 91.503 textbook Cormen et al.
7Overview of Linear Programming
Objective function
Convex feasible region
Objective value
source 91.503 textbook Cormen et al.
8Standard Slack Forms
9Standard Form
objective function
constraints
source 91.503 textbook Cormen et al.
10Standard Form (compact)
n-dimensional vectors
m-dimensional vector
mxn matrix
Can specify linear program in standard form by
(A,b,c).
source 91.503 textbook Cormen et al.
11Converting to Standard Form
source 91.503 textbook Cormen et al.
12Converting to Standard Form (continued)
Transforming minimization to maximization
Negate coefficients
source 91.503 textbook Cormen et al.
13Converting to Standard Form (continued)
Giving each variable a non-negativity constraint
New non-negativity constraints
If xj has no non-negativity constraint, replace
each occurrence of xj with xj xj.
source 91.503 textbook Cormen et al.
14Converting to Standard Form (continued)
Transforming equality constraints to inequality
constraints
source 91.503 textbook Cormen et al.
15Converting to Standard Form (continued)
Changing sense of an inequality constraint
Rationale
source 91.503 textbook Cormen et al.
16Converting Linear Programs into Slack Form
for algorithmic ease, transform all constraints
except non-negativity ones into equalities
slack variable
for inequality constraint
define slack
instead of s
non-basic variables
basic variables
source 91.503 textbook Cormen et al.
17Converting Linear Programs into Slack
Form (continued)
objective function
source 91.503 textbook Cormen et al.
18Converting Linear Programs into Slack
Form (continued)
set of indices of basic variables
set of indices of non-basic variables
Compact Form (N, B, A, b, c, v)
Slack Form Example
Compact Form
negative of slack form coefficients
source 91.503 textbook Cormen et al.
19 Formulating Problems as Linear Programs
20Shortest Paths
Single-pair shortest path minimize distance
from source s to sink t.
Can we replace maximize with minimize here? Why
or why not?
source 91.503 textbook Cormen et al.
21Maximum Flow
source 91.503 textbook Cormen et al.
22Minimum Flow
source 91.503 textbook Cormen et al.
23Multicommodity Flow
should be si
source 91.503 textbook Cormen et al.
24Simplex Method
25Solving a Linear Program
- Simplex algorithm
- Geometric interpretation
- Visit vertices on the boundary of the simplex
representing the convex feasible region - Transforms set of inequalities using process
similar to Gaussian elimination - Run-time
- not polynomial in worst-case
- often very fast in practice
- Ellipsoid method
- Run-time
- polynomial
- slow in practice
- Interior-Point methods
- Run-time
- polynomial
- for large inputs, performance can be competitive
with simplex method - Moves through interior of feasible region
source 91.503 textbook Cormen et al.
26Simplex Algorithm ExampleBasic Solution
Standard Form
Slack Form
Basic Solution
Basic Solution set each nonbasic variable to 0.
source 91.503 textbook Cormen et al.
27Simplex Algorithm Example Reformulating the LP
Model
Main Idea In each iteration, reformulate the LP
model so basic solution has larger objective value
Select a nonbasic variable whose objective
coefficient is positive x1
Increase its value as much as possible.
Identify tightest constraint on increase.
For basic variable x6 of that constraint, swap
role with x1.
Rewrite other equations with x6 on RHS.
new objective value
entering variable
PIVOT
source 91.503 textbook Cormen et al.
leaving variable
28Simplex Algorithm Example Reformulating the LP
Model
Next Iteration select x3 as entering variable.
new objective value
entering variable
PIVOT
leaving variable
New Basic Solution
source 91.503 textbook Cormen et al.
29Simplex Algorithm Example Reformulating the LP
Model
Next Iteration select x2 as entering variable.
new objective value
entering variable
PIVOT
leaving variable
New Basic Solution
source 91.503 textbook Cormen et al.
30Simplex Algorithm Issues to be Addressed
source 91.503 textbook Cormen et al.
31Simplex Algorithm Pivoting
entering variable
leaving variable
Rewrite the equation that has xl on LHS to have
xe on LHS
Update remaining equations by substituting RHS of
new equation for each occurrence of xe.
Do the same for objective function.
Update sets of nonbasic, basic variables.
source 91.503 textbook Cormen et al.
32Simplex Algorithm Pivoting (continued)
source 91.503 textbook Cormen et al.
33Simplex Algorithm Pseudocode
Feasibility initial basic solution
to be defined later (detects infeasibility)
optimal solution
source 91.503 textbook Cormen et al.
34Simplex Algorithm Correctness Proof Strategy
- If SIMPLEX has initial feasible solution and
eventually terminates, then it either - returns a feasible solution or
- determines that the LP is unbounded.
- SIMPLEX terminates.
- SIMPLEXs solution is optimal.
source 91.503 textbook Cormen et al.
35Simplex Algorithm Correctness Proof Approach,
Part I
- If SIMPLEX has initial feasible solution and
eventually terminates, then it either - returns a feasible solution or
- determines that the LP is unbounded.
source 91.503 textbook Cormen et al.
36Simplex Algorithm Correctness Proof Approach,
Part II
source 91.503 textbook Cormen et al.
37Duality
- Simplex Algorithm Correctness Proof Approach,
Part III
38Linear Programming Duality
max becomes min
x variables go away
RHS coefficients swap places with objective
function coefficients
y variables appear
sense changes
source 91.503 textbook Cormen et al.
39Duality Example
source 91.503 textbook Cormen et al.
40Weak Linear Programming Duality
Any feasible solution to primal LP has value no
greater than that of any feasible solution to the
dual LP.
source 91.503 textbook Cormen et al.
41Weak Linear Programming Duality (continued)
source 91.503 textbook Cormen et al.
42Finding a Dual Solution
Finding a dual solution whose value is equal to
that of an optimal primal solution
source 91.503 textbook Cormen et al.
43Optimality
source 91.503 textbook Cormen et al.
44Initial Basic Feasible Solution
45Finding an Initial Solution
An LP model whose initial basic solution is not
feasible
source 91.503 textbook Cormen et al.
46Finding an Initial Solution(continued)
Auxiliary LP model Laux
source 91.503 textbook Cormen et al.
47Finding an Initial Solution(continued)
source 91.503 textbook Cormen et al.
48Finding an Initial Solution(continued)
Original LP model
Laux
Laux in slack form
source 91.503 textbook Cormen et al.
49Finding an Initial Solution(continued)
PIVOT
PIVOT
source 91.503 textbook Cormen et al.
50Finding an Initial Solution(continued)
source 91.503 textbook Cormen et al.
51Fundamental Theorem of Linear Programming
source 91.503 textbook Cormen et al.