Chapter 3' Pitfalls

1
Chapter 3. Pitfalls

• Initialization
• Ambiguity in an iteration
• Finite termination?
• Resolving ambiguity in each iteration
• Selection of the entering variable
• Any nonbasic variable with can
  enter basis.
• (However, performance of the algorithm depends
  on the choice of the entering variable.
• examples of the rules largest coefficient
  rule, maximum increase rule, steepest ascent
  rule, )

2

• Selection of leaving variable
• No restriction in minimum ratio test can
  increase the value of the entering variable
  indefinitely while satisfying nonnegativity,
  hence problem is unbounded
• Ex)

Increase x3 while keeping x1 and x4 at 0, then
direction of movement should be d ( 0, 2, 1, 0,
0 ) ? 0 so that the new point satisfies the
equations. Hence the new point x0 ?d ? 0
satisfies nonnegativity for any ? ? 0. Also
objective value increases by 1? ( 1 is
coefficient of x3 in z row).
3

• Also note that

Note that if we want to move from x0 to x0 ?d,
?gt0 , then we must have A(x0 ?d)b so that the
new point satisfies the equations Axb. Then
A(x0 ?d)b, ?gt0 holds if and only if Ad0.
So the direction of movement d must satisfy Ad0
for the d vector to be a feasible direction of
movement ( d is in the null space of A).
4

• b) In case of ties in the minimum ratio test

5

In the next iteration,

( x1 entering, x5 leaving )
Though x1 cannot be increased, we perform the
pivot as usual, making x1 as basic and x5 as
nonbasic. No change in solution values, but
basic and nonbasic status changes for two
variables. Later, we will examine what this
means in geometry.
6

• Terminology
• degenerate solution (???) basic feasible
  solution with one or more basic variables having
  0 values.
• degenerate iteration (pivot) simplex iteration
  that does not change the basic solution values
  (only basis changes).
• Observations
• Given a nondegenerate b.f.s., we must have ties
  in the minimum ratio test so that we have a
  degenerate solution after pivot.
• A degenerate pivot occurs only if we have a
  degenerate solution, but the converse is not true
  (i.e. we may have a nondegenerate pivot although
  we have a degenerate solution).

7
Geometric meaning of a degenerate iteration
x20
x20
x50
x50
x10
x10
x60
x60
A
A
( x10, x20 used)
( x20, x50 used)
8

• (ex- continued)
• In the first dictionary, the point A is
  identified by using the three equations in Ax b
  and setting x1 ? 0, x2 ? 0 at equalities. The
  nonnegativity constraints for nonbasic variables
  (here x1, x2) are used to indentify point A
• ( provides n equations together with Ax b).
• After pivot, the solution point A is not
  changed. But it is identified now by using x2 ?
  0, x5 ? 0 at equalities together with Ax b.
• Hence degenerate iteration changes the defining
  equations one at a time, but the solution point
  is not changed.

9

• Getting out of degenerate iterations

x20
x50
x10
x60
A
x3?0
( x30, x50 used)
We changed the data in the previous dictionary a
little bit. Now, x2 enters (takes value 1) and
x3 leaves the basis, and it is a nondegenerate
pivot. Geometrically, we still satisfy the
equation x5 0, but do not need to satisfy x2
0. As we increase the value of x2 up to 1, x3
becomes 0, hence the equation x3 0 is now used
(together with x5 0 and Axb) to define a new
point.
10

• Degenerate pivot is the process of identifying
  the same point using different defining equations
  (nonnegativity constraints). If we are lucky
  enough to obtain defining equations that guides
  the moving direction, we move to a different
  point with a nondegenerate pivot.
• During the degenerate iterations, the algorithm
  stalls and it may hamper the performance of the
  algorithm. Such phenomenon is of practical
  concern and affects the performance of the
  algorithm (especially, for problems with some
  special structures and large problems).
• Later, we will see how to get out of degenerate
  iterations systematically.

11

• If we have a degenerate solution, pivot may
  continue indefinitely (Example in text p.31,
  pivoting rule is largest coefficient for entering
  variable and smallest subscript for leaving in
  case of ties. Then we have the initial
  dictionary again after 6 pivots.)
• Terminology
• Cycling appearance of the same dictionary again
  in the simplex iterations.
• Cycling is the only reason that simplex method
  may fail to terminate (i.e. simplex method
  terminates in a finite number of iterations as
  long as cycling is avoided).

12

• Thm) If simplex method fails to terminate, it
  must cycle.
• Pf) Number of ways to choose basic variables are
  finite. Hence, if simplex fails to terminate,
  same basis must appear again.
• Show same basis ? same dictionary, Hence same
  dictionary appears, i.e. cycles.
• (1) and (2) have the same set of solutions
  (ignore nonnegativity constraints)
• Let Consider a solution to (1)
  given as the following

13

• This solution satisfies (1). So it also
  satisfies (2), hence
• These equalities must hold for any real number t.
• Therefore, we have
• We can use any nonbasic variable in the above
  proof, so dictionaries are identical.
  ?
• The theorem can be proved easily if we use
  matrices, but the above proof uses only algebraic
  arguments.

14

• The proof of the theorem shows that if we have
  the same basis, then the dictionaries (tableaus)
  are the same. Since there are only a finite
  number of ways to choose the basis, the simplex
  method terminates finitely if the same basis (the
  same dictionary) does not appear again, i.e.
  cycling is avoided.

15
Avoid cycling

• Perturbation method, Lexicographic method
• Smallest-subscript rule (Blands rule)
• Smallest-subscript rule (Blands rule)
• Choose the variable having the smallest index
  among possible candidates as the
  entering variable.
• Also, if ties occur while choosing the leaving
  variable, select the smallest indexed variable
  among the candidates as the leaving variable.
• See the proof in the text if time permits.

16

• Examples of smallest subscript rule

x3 enters, x4 leaves.
x1 enters, x5 leaves.
17

• Perturbation method, lexicographic method
• Idea Avoid the appearance of degenerate solution
  since it is a precondition of cycling ( If the
  solution is not degenerate, the objective
  function value increases strictly in the next
  iteration, hence the same basis does not appear
  again). So add very small positive ? to the
  r.h.s of equations so that the solution values
  are unchanged practically but degeneracy is
  avoided.
• But the ?s added to the r.h.s may cancel out
  each other during elementary row operations,
  again causing degeneracy.
• Remedy Add different values of ?is to
  different r.h.s. so that cancellation does not
  occur.

18

• Add ?i to the i-th r.h.s., i 1, , m with the
  following property.
• Then, it can be shown that the values of the
  basic variables never become 0 in subsequent
  simplex iterations, hence no cycling occurs. (In
  practice, precision can cause problems.)
• (In actual implementation, ?1 ?, ?2 ?2, ?3
  ?3, may be used or random numbers in 0, ?
  for some fixed small ? are used. )
• We may be concerned about the correctness of the
  optimal solution, but in practice the revised
  simplex method is usually used instead of
  dictionary. It has the capability to recover the
  correct solution values if the current basis is
  known (Chapter 7). Then we can restart the
  algorithm again without adding ?Is, once we
  escape degenerate iterations. Hence the effect
  of adding ?Is to the r.h.s can be eliminated.

19

• Perturbation method usually refers to the methods
  actually adding small ?is to the right hand
  sides during simplex iterations.
• For lexicographic method, the idea is the same
  as the perturbation method. But we do not
  actually add ?is. However, we perform the
  pivots as if ?is are present.

20

• Lexicographic ordering of numbers
• Consider r r0 r1 ?1 . rm ?m, s
  s0 s1 ?1 . sm ?m
• If r ? s, there is the smallest subscript k
  such that rk ? sk .
• We say that r is lexicographically smaller than
  s if rk lt sk .
• (Similar terminology is used for vectors too)
• Then if
• r is smaller than s if r is lexicographically
  smaller than s.
• Ex) r 2 21?1 19 ?2 20000?3
• s 2 21?1 20?2 20?3 15?4 14?5
• r is lexicographically smaller than s.

21

• Lexicographic method Smart implementation of
  the perturbation method. We do not actually add
  epsilons to the r.h.s., but epsilons are treated
  as symbols (like variables) denoting very small
  positive numbers.
• Moreover, during the simplex iterations, the
  coefficients of epsilons can be read from the
  simplex tableau directly without actually
  expressing the epsilon variables in the tableau

• Note that ?i in the above form may be regarded as
  additional variables in the equations like x.
  Any solution x, ? (and z) need to satisfy the
  above equations.

22
Lexicographic method

(02?1 )
(02?2 )
(1?3 )
23

• Note that the coefficient matrix for the basic
  variables x5, x6, x7 and the coefficient matrix
  for ?1, ?2, ?3 are the same identity matrices
  in the beginning of the lexicographic method.
• Since we use the elementary row operations in the
  simplex iterations, those two coefficient
  matrices have the same elements in the following
  iterations. Hence, we can read the coefficients
  of ?1, ?2, ?3 from the coefficients of x5,
  x6, x7 . So we do not actually need to add ?1,
  ?2, ?3 to the tableau.
• Usually, lexicographic method means we read
  coefficients of ?is from the coefficients of
  corresponding x variables.
• Hence, in the example, we actually do not add
  ?is. We first perform ratio test using the
  entering column and the r.h.s. If ties occur,
  then perform ratio test (for tied rows) again
  using the entering column and the column for x5,
  etc.

24
Can cycling be prevented?

• Since the elementary row operation matrices are
  nonsingular, the two matrices for ? variables and
  corresponding x variables remain nonsingular
  after applying elementary row operations
  (elementary row operation is equivalent to
  premultiplying the corresponding nonsingular
  matrices. Since the matrix for ? variables is
  identity matrix initially, it is nonsingular.
  Hence we also obtain nonsingular matrix).
• In other words, no row with all 0 elements
  appear (in the ?-matrix). Hence the values of
  basic variables never become 0. (conceptually,
  after adding small ?i values.)

25

• Simplex pivot rule guarantees that all basic
  variables are nonnegative at any time (treating
  ?is as small positive values). Hence, combined
  with above, it guarantees that the values of
  basic variables are always positive
  (conceptually). So strict increase of the
  objective value is guaranteed. It guarantees
  that the same tableau (or dictionary) will not
  appear again.
• We can read the real solution value by ignoring
  the ? terms in the current dictionary.
• It is also observed that the lexicographic
  method can be started and stopped at any time
  during the simplex iterations. We just
  conceptually add or drop the ? terms.
• See the text for more rigorous proof. It proves
  that rows with all 0 elements never appear in
  ?-matrix, hence no solution will have 0 value,
  i.e. no degeneracy occurs in subsequent
  iterations.

26
Initialization (two-phase method)

• We need an initial b.f.s to start the simplex
  method.
• If we have bi lt 0 for some constraint i, the
  slack variable xni bi lt 0 is not feasible.
• Consider easily obtained solution xi 0 for all
  i in (1) and then subtract some positive number
  x0 from all l.h.s. so that it becomes feasible
  solution to (1),
• i.e. xi 0 for all i, x0
• Now, if we can find a feasible solution to (1)
  with x0 0, it is a feasible solution to (1)
  using original variables.

subject to
(1)
27

• Hence solve the following problem using the
  easily obtained initial feasible solution and
  simplex method to find an optimal solution with
  x0 0. Note that x0 is a nonnegative variable.
• (1) has a feasible solution ? (2) has an
  optimal solution with optimal value 0 (x0
  0)
• If we find an optimal solution with x0 0, we
  can obtain a feasible solution to (1) by
  disregarding x0.
• One point to be careful is that we need a b.f.s.
  to perform the simplex method.

subject to
(2)
28

• Example

We cannot perform simplex iteration in this
dictionary since the basic solution is not
feasible (nonnegativity violated) However a
feasible dictionary can be easily obtained by one
pivot.
29

• Current basic solution is not feasible. Let x0
  enter basis and the slack variable with most
  negative value leaves the basis (It is not a
  simplex iteration. Just perform the pivot, not
  considering obj. value. It does not change the
  solution set.)

Perform simplex method. After two iterations, we
get the optimal dictionary
30

• We obtained optimal solution with value 0.
  Hence the current optimal solution gives a b.f.s.
  to the original problem. Drop x0 (no more
  needed) and replace the objective function with
  the original one. z row is used to read the
  objective value of a given solution, hence it can
  be added or dropped without affecting the
  feasible solution set to the LP. Note that the
  b.f.s. is a b.f.s. to the original problem.

31

• Express it in dictionary form (only nonbasic
  variables appear in the r.h.s.) by substituting
  the basic variables in the objective function.
• Now, restart the simplex method with the current
  dictionary

32

• Algorithm strategy in phase one choose x0 as
  leaving variable in case of ties in the minimum
  ratio test.
• 2 cases possible in phase one optimal solution
• w nonzero ( w lt 0 ), x0 basic ? original
  problem is infeasible
• w 0 , x0 nonbasic ? drop w, express
  original objective function z in terms of
  nonbasic variables, continue the simplex method.
• (Note that w 0, x0 basic cant happen by our
  strategy)
• Similar idea can be used when the original LP is
  given in equality form. (Without converting a
  equality using two inequalities, simplex method
  can be used directly to solve the equality form.
  More in Chapter 8.)

33

• (Fundamental theorem of LP)
• Every LP in standard form has the following
  properties.
• No optimal solution ? either unbounded or
  infeasible
• ? feasible solution ? ? a basic feasible
  solution
• ? optimal solution ? ? a basic optimal
  solution