Simplex Algorithm NR

1
Simplex Algorithm (NR)

• Reference
• Numerical Recipe Sec. 10.8

2
The Diet Problem

• Dietician preparing a diet consisting of two
  foods, A and B.

Minimum requirement protein 60g fat 24g
carbohydrate 30g Looking for minimum cost diet
3
Linear Programming
4
The Problem
Maximize
N dimension of x M number of constraints
Subject to
5
Testing Problem
6
Theory of Linear Optimization
Terminology Feasible vector Feasible basic
vector Optimal feasible vector
7
Theory (cont)

• Feasible region is convex and bounded by
  hyperplanes
• Feasible vectors that satisfy N of the original
  constraints as equalities are termed feasible
  basic vectors.
• Optimal occur at boundary (gradient vector of
  objective function is always nonzero)
• Combinatorial problem determining which N
  constraints (out of the NM constraints) would be
  satisfied by the optimal feasible vector

8
Simplex Algorithm (Dantzig 1948)

• A series of combinations is tried to increase the
  objective function
• Number of iterations less than O(max(M,N))
• Explain in restricted normal form first
• Worst case complexity is exponential (in number
  of variables) yet has polynomial smoothed
  complexity (and works well in practice)

9
Restricted Normal Form

• Only equality and non-negativity constraints
• One additional constraint each equality
  constraint can identify one left-hand variable.

LH var. (basic) non-zero
RH var. (non-basic) zero
10
Simplex Tableau
Z row
Feasible vector (2,0,0,8)
In each exchange, a right-hand variable and a
left-hand variable change places to increase z
Here change x2 to basic (non-zero) is a good
idea
11
Simplex Tableau
Z row
pivot column
Which basic (x1 or x4) ?non-basic?
x2 violates non-negative constraints
12
Simplex Tableau
In each exchange, a right-hand variable and a
left-hand variable change places.
Feasible vector (0,1/3,0,9) Optimal z 2/3 by
observing z-row
13
Converting to Restricted Normal Form

• Slack variable convert inequality to equality
  constraints
• Artificial variable artificially left-hand
  variables

14
Example (slack artificial variables)
yi, zi ? 0
15
Optimization
Phase I maximize the auxiliary objective function
Optimal sol all zis are zero produce a set of
left-hand variables from xi and yis only
If this cannot be done, no feasible basic vector
exist (constraints are inconsistent)
Phase II use the solution from first phase,
maximize the original objective function
16
On output
17
NR Solver Interface

• On input
• a two-dimensional array in NR format
  (convert_matrix) a1..M21..N1 last row used
  internally
• On output
• icase (0 sol found 1 unbounded -1
  inconsistent constraints)
• izrov1..N zero (non-basic) variables
• iposv1..M positive (basic) variables

18
On Input
bi ?0
19
On Output
iposv1..45,2,4,3 izrov1..4 1,6,8,7
N4 M4, M12,M21,M31 NM1M2 NM1M2 Any
variable gtNM1M2 is internal var (ignored)
A11 z_opt A21 xiposv1 ?x5 y1
?constraint 1 stays inequality A31
xiposv2 ?x2 A41 xiposv3 ?x4 A51
xiposv4 ?x3
Optimal solution x (0, 3.33, 4.73, 0.95), zopt
17.02
20
Another Example (Diet)
21
LP in CD (narrow phase)

• A pair of convex objects each facet represented
  by the plane inequality aixbiyciz ? di
• If two sets of inequality (from two convex
  objects) define a feasible region, then a
  collision is detected
• The type of optimization and the objective
  function used is irrelevant.

22
LP for CD
23
Duality

• Primal problem

• Dual problem

numerical example
theory, interpretation
24
(No Transcript)
25
Duality

• If a linear program has an optimal solution then
  so does the dual. Furthermore, the optimal values
  of the two programs are the same.

Computational efficiency?! (if N M are quite
different)
26
Dual of the Diet Problem

• Interpretation!!

27
Transportation Distance as a Measure to Melodic
Similarity

• Ref Typke et al. 2003

28
Example
Partially matched
29
EMD Revisited
30
EMD as Linear Program
Solve by simplex algorithm
31
EMD for Melodic Similarity

• Ground distance Euclidean distance
• Scale time coordinate (so that they are
  comparable)
• Transposition transpose one of the melodies so
  that the weighted average pitch is equal (not
  optimum but acceptable)

32
HW Verify planar CD using LP
Warning Non-negative assumption!!
33
Verify planar CD using LP.