Linear Programming and Simplex Algorithm

1
Linear Programming and Simplex Algorithm

• Reference
• Numerical Recipe Sec. 10.8

2
Content

• Linear programming problems
• Simplex algorithm
• Simplex tableau
• NR solver
• LP Applications
• Collision detection
• Melodic similarity
• Underdetermined linear systems

3
The Diet Problem

• Dietician preparing a diet consisting of two
  foods, A and B.

Minimum requirement protein 60g fat 24g
carbohydrate 30g Looking for minimum cost diet
4
Linear Programming
5
The Problem
Maximize
N dimension of x M number of constraints (M
m1m2m3)
Subject to
6
Theory of Linear Optimization
Terminology Feasible vector Feasible basic
vector Optimal feasible vector
7
Theory (cont)

• Feasible region is convex and bounded by
  hyperplanes
• Feasible vectors that satisfy N of the original
  constraints as equalities are termed feasible
  basic vectors.
• Optimal occur at boundary (gradient vector of
  objective function is always nonzero)
• Combinatorial problem determining which N
  constraints (out of the NM constraints) would be
  satisfied by the optimal feasible vector

8
Simplex Algorithm (Dantzig 1948)

• A series of combinations is tried to increase the
  objective function
• Number of iterations less than O(max(M,N))
• Worst case complexity is exponential (in number
  of variables) yet has polynomial smoothed
  complexity (and works well in practice)
• Explain in restricted normal form first

9
The Canonical Problem
Maximize
N dimension of x M number of constraints (M
m1m2m3)
Subject to
I.
II.
III.
10
Testing Problem
I.
II.
III.
Solved by NR (later)
11
NR Solver Interface

• On input
• a two-dimensional array in NR format
  (convert_matrix) a1..M21..N1 last row used
  internally (set to zero)
• On output
• icase (0 sol found 1 unbounded -1
  inconsistent constraints)
• izrov1..N zero (non-basic) variables
• iposv1..M positive (basic) variables

12
N4 M12, M21,M31
On Input
slack var for inequalities
bi ?0
13
On Output
iposv1..45,2,4,3 izrov1..4 1,6,8,7
N4 M4, M12,M21,M31 NM1M2 NM1M2 Any
variable gtNM1M2 is internal var (ignored)
A11 z_opt A21 xiposv1 ?x5 y1
?constraint 1 stays inequality A31
xiposv2 ?x2 A41 xiposv3 ?x4 A51
xiposv4 ?x3
Optimal solution x (0, 3.33, 4.73, 0.95), zopt
17.02
14
Duality

• Primal problem

• Dual problem

15
(No Transcript)
16
Duality

• If a linear program has a finite optimal solution
  then so does the dual. Furthermore, the optimal
  values of the two programs are the same.
• If either problem has an unbounded optimal
  solution, then the other problem has no feasible
  solutions.

17
Duality Example ref
Wholesalers perspective How can I set the
prices per ounce of chocolate, sugar, and cream
cheese so that the baker will buy from me, and so
that I will maximize my revenue?
Classic diet problem
x(2, 1.5)
u(10/3, 20, 0)
18
Diet Problem
19
Applications
20
The Problem
Under-determined system (infinite number of
solutions) Find the solution with minimum
magnitude (i.e., closest to origin)
21
Example
Approach 1 find complete solution
(-2,0,0) subtract its projection along (1,-1,1)
22
Linear Programming Solution

• Most simplex algorithm assume nonnegative
  variables
• Set

23
Example
24
Solved by QSopt
The real minimum (-4/3, -2/3, 2/3) The LP
constraint is not exactly x2 (but x?)
This problem can also be solved by SVD (singular
value decomposition)
25
LP in CD (narrow phase, ref)

• A pair of convex objects each facet represented
  by the plane inequality aixbiyciz ? di
• If two sets of inequality (from two convex
  objects) define a feasible region, then a
  collision is detected
• The type of optimization (minmax) and the
  objective function used is irrelevant.

26
LP for CD
27
Transportation Distance as a Measure to Melodic
Similarity

• Ref Typke et al. 2003

28
Example
Partially matched
29
Earth Mover Distance
30
EMD as Linear Program
Solve by simplex algorithm
31
EMD for Melodic Similarity

• Ground distance Euclidean distance
• Scale time coordinate (so that they are
  comparable)
• Transposition transpose one of the melodies so
  that the weighted average pitch is equal (not
  optimum but acceptable)

32
HW Verify planar CD using LP
Warning Non-negative assumption (for some
solvers)!!
33
Verify planar CD using LP.
34
LP Solvers

• Lp_solve MILP
• QSopt
• GLPK

35
QSopt ref

• Contains a GUI and a callable library

36
LP Format Example
Problem smallExample Minimize obj 0.60x1
0.40x2 Subject to c1 12x1 6x2 gt 24 30x1
15x2 gt 30 Bounds 0 lt x1 0 lt x2 End
Somehow x1 gt 0 x2 gt 0 Doesnt work!?
37
LP format BNF definition1
38
LP format BNF definition2
39
Scenario Read and Solve
40
Accessing a Solution

• In our applications, we dont really care about
  pi, slack, and rc.
• Set them to NULL

• value objective function
• x solution vector (columns)
• pi dual variables
• slack slack variables
• rc reduced cost

41
QS_MAX or QS_MIN
cmatind
of nonzeros in jth column
Location of start entry
42
Modify an LP Problem

• QSnew_row
• QSadd_rows
• QSnew_col
• QSadd_cols
• QSdelete_row
• QSdelete_col
• QSchange_coef
• QSchange_objcoef
• QSchange_rhscoef
• QSchange_sense

• See reference manuals for more details (Ch.5)
• Use QSwrite_prob to verify the change

43
The End
44
issues

• Min problem reverse the objective function and
  keep the constraint set? Or solve the dual
  problem
• LPIK

45
Restricted Normal Form

• Only equality and non-negativity constraints
• One additional requirement each equality
  constraint can identify one left-hand variable.

LH var. (basic) non-zero
RH var. (non-basic) zero
46
Simplex Tableau
Change x2 to basic (non-zero) is a good idea
Z row
Feasible vector (2,0,0,8)
In each step, a right-hand variable and a
left-hand variable change places to increase z
47
Simplex Tableau
Z row
pivot column
Which basic (x1 or x4) ?non-basic?
Setting x1 zero makes x2 1/3. OK!
Setting x4 zero makes x2 -8/3, violating
non-negative constraints
48
Simplex Tableau
In each step, a right-hand variable and a
left-hand variable change places.
Stop iteration! Opt.z 2/3
Feasible vector (0,1/3,0,9) Optimal z 2/3
observing z-row
49
Continue to simplex-tableau.ppt
50
Converting to Restricted Normal Form

• Slack variable convert inequality to equality
  constraints
• Artificial variable artificially left-hand
  variables

51
Example (slack artificial variables)
slack vars.
yi, zi ? 0
artificial vars.
52
Optimization
Phase I maximize the auxiliary objective function
Optimal sol all zis are zero produce a set of
left-hand variables from xi and yis only
If this cannot be done, no feasible basic vector
exist (constraints are inconsistent)
Phase II use the solution from first phase,
maximize the original objective function
53
On output