Introduction to Algorithms

1
Introduction to Algorithms

• Linear Programming
• My T. Thai _at_ UF

2
New crop problem

• A farmer
• Has 10 acres to plant in wheat and rye
• Has Only 1200 to spend
• Has to plant at least 7 acres
• Has to get the planting done in 12 hours
• Each acre of wheat costs 200 and takes an hour
  to plant and gives 500 profit
• Each acre of rye costs 100 and takes 2 hours to
  plant and gives 300 profit
• Goal Compute the numbers of acres of each should
  be planted to maximize profits

x of wheat acres y of rye acres
3
Linear program

• A linear programming problem is the problem of
  maximizing or minimizing a linear function
  subject to linear constraints (equalities and
  inequalities)
• E.g.

Objective function
Constraints
4
Formulate shortest path as an LP

• Triangle inequalities
• At the source vertex ds 0
• dt is less than weights of all paths from s to t
  gt dt is the maximum value that less than weights
  of all paths from s to t
• We have the following

5
Max Flow

• Capacity constraints
• Conservation constraints
• We have the following

6
Minimum-cost flow

• Pay cost to transmit flow fuv
  through edge (u, v)
• Wish to send d units with the minimized cost

7
Multicommodity flow

• Has k commodities ith commodity is defined by a
  triple
• Total flow of all commodities through an edge
  does not exceed its capacity
• Find a feasible flow

8
Some Notations

• Minimization (maximization) linear program the
  value of objective function is minimized
  (maximized)
• Feasible (infeasible) solution a setting of
  variables satisfies all the constraints
  (conflicts at least one constraint)
• Feasible region set of feasible solutions
• Objective value value of the objective function
  at a particular point
• Optimal solution objective value is maximum over
  all feasible solutions
• Optimal objective value objective value of
  optimal solution
• The linear program is infeasible if it has no
  feasible solutions otherwise it is feasible
• The linear program is unbounded if the optimal
  objective value is infinite

9
Standard form

• Concise representation

10
Convert into standard form
11
Example
(negate objective function)
(replace equality)
(replace x2)
(negate constrain and change variable name)
12
Geometry of Linear Programming

• Theorem 1
• Feasible region of an LP is convex
• Proof
• Feasible region is the intersection of half
  spaces defined by the constraints. Each half
  space is convex.

13
Example
14
Feasibility and Infeasibility

• Simple solution try all vertices of polyhedron ?
  running time
• ? How to refine this approach?

15
Simplex method

• Start off from a vertex, which is called a basic
  feasible solution
• Iteratively move along an edge of the polyhedron
  to another vertex toward the direction of
  optimization
• For each move, need to make sure that the
  objective function is not decreased
• Observation when moving from a vertex to another
  vertex, an inequality achieves equality

16
Questions Arise
17
Slack form

• All inequality constraints are non-negativity
  constraints
• Convert by introducing slack variables

nonbasic variables
Basic variables
18
Concise representation of slack form

• z the value of the objective function
• N the set of indices of the nonbasic variables
  (N n)
• B the set of indices of the basic variables (B
  m)
• v an optional constant term in the objective
  function
• A tuple (N, B, A, b, c, v) represents the slack
  form

maximize
19
Simplex algorithm

• Basic solution all nonbasic variables equals 0
• Each iteration converts one slack form into an
  equivalent slack form s.t. the objective value is
  not decreased
• Choose a nonbasic variable xe (entering variable)
  such that its increase makes the objective value
  increase
• Keeping all constraint satisfied, raise the
  variable raise it until some basic variable xl
  (leaving variable) becomes 0
• Exchange the roles of that basic variable and the
  chosen nonbasic variable

20
Example
21
Substitute x_2 x_3 0 to the slack variables,
we have
22
(No Transcript)
23
Cycling

• SIMPLEX may run forever if the slack forms at two
  different iterations of SIMPLEX are identical
  (cycling phenomenon)
• We need specific rule of picking the entering and
  leaving variables
• There are quite a few methods to prevent cycling.
  The one we just used is called Blands pivoting
  rule

24
Other Pivoting Rules
25
Time Complexity
26
Duality

• Given a primal problem
• P min cTx subject to Ax b, x 0
• The dual is
• D max bTy subject to ATy c, y 0

27
An Example
28
Weak Duality Theorem

• Weak duality Theorem
• Let x and y be the feasible solutions for P and
  D respectively, then
• Proof Follows immediately from the constraints

29
Weak Duality Theorem

• This theorem is very useful
• Suppose there is a feasible solution y to D. Then
  any feasible solution of P has value lower
  bounded by bTy. This means that if P has a
  feasible solution, then it has an optimal
  solution
• Reversing argument is also true
• Therefore, if both P and D have feasible
  solutions, then both must have an optimal
  solution.

30
Hidden Message

Strong Duality Theorem If the primal P has an
optimal solution x then the dual D has an
optimal solution y such that cTx bTy
31
Complementary Slackness

• Theorem
• Let x and y be primal and dual feasible solutions
  
• respectively. Then x and y are both optimal iff
  two
• of the following conditions are satisfied
• (ATy c)j xj 0 for all j 1n
• (Ax b)i yi 0 for all i 1m

32
Proof of Complementary Slackness

• Proof
• As in the proof of the weak duality theorem, we
• have cTx (ATy)Tx yTAx yTb (1)
• From the strong duality theorem, we have

(2) (3)
33
Proof (cont)

• Note that
• and
• We have
• x and y optimal ? (2) and (3) hold
• ? both sums (4) and (5) are zero
• ? all terms in both sums are zero (?)
• ? Complementary slackness holds

(4)
(5)
34
Why do we care?

• Its an easy way to check whether a pair of
  primal/dual feasible solutions are optimal
• Given one optimal solution, complementary
  slackness makes it easy to find the optimal
  solution of the dual problem
• May provide a simpler way to solve the primal

35
Some examples

• Solve this system

36
Min-Max Relations

• What is a role of LP-duality
• Max-flow and Min-Cut

37
Max Flow in a Network

• Definition Given a directed graph G(V,E) with
  two distinguished nodes, source s and sink t, a
  positive capacity function c E ? R, find the
  maximum amount of flow that can be sent from s to
  t, subject to
• Capacity constraint for each arc (i,j), the flow
  sent through (i,j), fij bounded by its capacity
  cij
• Flow conservation at each node i, other than s
  and t, the total flow into i should equal to the
  total flow out of i

38
An Example
3
4
3
4
4
3
0
4
2
3
4
3
2
4
t
2
1
s
3
1
1
1
3
2
0
2
3
0
2
1
2
5
0
0
39
Formulate Max Flow as an LP

• Capacity constraints 0 fij cij for all (i,j)
• Conservation constraints
• We have the following

40
LP Formulation (cont)
3
4
3
4
4
3
0
4
2
3
4
3
2
4
3
t
2
1
1
s
1
1
2
3
2
0
3
1
0
2
2
5
0
0
8
41
LP Formulation (cont)
42
Min Cut

• Capacity of any s-t cut is an upper bound on any
  feasible flow
• If the capacity of an s-t cut is equal to the
  value of a maximum flow, then that cut is a
  minimum cut

43
Max Flow and Min Cut
44
Solutions of IP

• Consider
• Let (d,p) be the optimal solution to this IP.
  Then
• ps 1 and pt 0. So define X pi pi 1
  and \bar X pi pi 0. Then we can find
  the s-t cut
• dij 1. So for i in X and j in \bar X, define
  dij 1, otherwise dij 0.
• Then the object function is equal to the minimum
  s-t cut