Chap 2. The Geometry of LP

1
Chap 2. The Geometry of LP

• In the text, polyhedron is defined as P x ?
  Rn Ax ? b . So some of our earlier results
  should be taken with modifications.
• Thm 2.1
• (a) The intersections of convex sets is convex.
• (b) Every polyhedron is a convex set.
• (c) Convex combination of a finite number of
  elements of a convex set also belongs to that
  set.
• (recall that S closed for convex combination of
  2 points.
• ? S closed for convex combination of a finite
  number of pts)
• (d) Convex hull of a finite number of vectors
  (polytope) is convex.

2

• Pf) (a) Let x, y ? ?i?I Si , Si convex ?
  x, y ? Si , ? i
• ? ?x (1-?)y ? Si ? i since Si convex
• ? ?x (1-?)y ? ?i?I Si , ? ? Si convex.
• (b) Halfspace x ax ? b is convex.
• P ? halfspaces ? From (a), P is convex
• ( or we may directly show A( ?x (1-?)y ) ? b.
  )
• (c) Use induction. True for k 2 by
  definition.
• Suppose statement holds for k. Suppose ?k1 ?
  1.
• Then ?i1k1 ?ixi ?k1xk1 (1-?k1) (
  ?i1k ( ?i / (1-?k1)) xi )
• ?i / (1-?k1) ? 0 and sum up to 1, hence ?i 1k
  ( ?i / (1-?k1)) xi ? S. ? ?i 1k1 ?ixi ? S.
• (d) Let S be the convex hull of vectors x1, ,
  xk and y, z ? S
• ? y ?i1k ?ixi , z ?i1k ?ixi for some
  ?i , ?i
• ?y (1-?)z ? ? ?ixi (1-?) ? ?ixi ?i1k
  (??i (1-?)?i) xi
• ??i (1-?)?i ? 0 and sum up to 1 ? convex
  comb. of xi
• ? ?y (1-?)z ? S. ?

3
Extreme points, vertices, and b.f.ss

• Def (a) Extreme point ( as we defined earlier)
• (b) x ? P is a vertex if ? c ? Rn such that
  cx lt cy ? y ? P and y ? x. ( x is unique
  optimal solution of min cx, x ? P )
• (c) Consider polyhedron P and x ? Rn. Then x
  is a basic solution if all equality constraints
  are active at x and ? n linearly independent
  active constraints among the constraints active
  at x.
• ( basic feasible solution if x is basic
  solution and x ? P )
• Note Earlier, we defined the extreme point same
  as in the text.
• Vertex as 0-dimensional face ( dim (P) rank
  (A, b) n ) which is the same as basic
  feasible solution in the text.
• We defined basic solution (and b.f.s) only for
  the standard LP. ( xB B-1b, xN 0 )
• Definition (b) is new. It gives an equivalent
  characterization of extreme point. (b) can be
  extended to characterize a face F of P.

4
x3
x2
A
C
P
E
x1
D
B

• Fig. 2.6 P (x1, x2, x3) x1x2x3 1, x1,
  x2, x3 ? 0
• Three constraints active at A, B, C, D. Only
  two constraints active at E. Note that D is not
  a basic solution since it does not satisfy the
  equality constraint. However, if P is denoted as
  P (x1, x2, x3) x1x2x3 ? 1, x1x2x3 ? 1,
  x1, x2, x3 ? 0, D is a basic solution by the
  definition in the text, i.e. whether a solution
  is basic depends on the representation of P.

5
A
E
D
P
F
B
C

• Fig. 2.7 A, B, C, D, E, F are all basic
  solutions. C, D, E, F are basic feasible
  solutions.

6

• Comparison of definitions in the notes and the
  text

Notes Text
Extreme point Geometric definition Geometric definition
Vertex 0-dimensional face Existence of c vector which makes x as the unique optimal solution for the LP
Basic solution, b.f.s. Defined for standard form. Set n-m variables at 0 and solve the remaining system. b.f.s. if nonnegative. Defined for general polyhedron. Satisfy equality constraints and n linearly independent constraints are active. ( 0-dimensional face if feasible)
7

• Thm 2.3 x ? P, then x vertex, extreme point,
  and b.f.s. are equivalent statements.
• Pf) We follow the definitions given in the
  text. We already showed in the notes that
  extreme point and 0-dimensional face ( AIx bI,
  AI rank n, b.f.s. in the text) are equivalent.
• To show all are equivalent, take the following
  steps
• x vertex (1) ? x extreme point (2) ? x
  b.f.s. (3) ? x vertex
• (1) x vertex ? x extreme point
• Suppose x is vertex, i.e. ? c ? Rn such that
  x is unique min of min cx, x ? P.
• If y, z ? P, y, z ? x, then cx lt cy and cx
  lt cz.
• Hence cx lt c( ?y (1-?)z ), 0 ? ? ? 1 ?
  ?y (1-?)z ? x
• Hence x cannot be expressed as convex
  combination of two other points in P
• ? x extreme point.

8

• (continued)
• (2) x extreme point ? x b.f.s.
• Suppose x is not a b.f.s.. Let I i aix
  bi
• Since x is not a b.f.s., the number of linearly
  independent vectors ai in I lt n.
• Hence ? nonzero d ? Rn such that aid 0, ? i ?
  I.
• Consider y x ?d, z x - ?d. But, y, z ?
  P for sufficiently small positive ?, and x
  (yz)/2, which implies x is not an extreme
  point.
• (3) x b.f.s. ? x vertex
• Let x be a b.f.s. and let I i aix bi
  
• Let c ?i?I ai . Then cx ?i?I aix ?i?I
  bi
• ? x ? P, we have cx ?i?I aix ? ?i?I bi
  cx, hence x optimal.
• For uniqueness, equality holds ? aix bi , i
  ? I.
• Since x is b.f.s., it is the unique solution of
  aix bi , i ? I
• Hence x is a vertex. ?

9

• Note Whether x is a basic solution depends on
  the representation of P. However, x is b.f.s.
  iff x extreme point and x being extreme point
  is independent of the representation of P. Hence
  the property of being a b.f.s. is also
  independent of the representation of P.
• Cor 2.1 For polyhedron P ? ?, there can be
  finite number of basic or basic feasible
  solutions.
• Def Two distinct basic solutions are said to be
  adjacent if we can find n-1 linearly independent
  constraints that are active at both of them. (
  In Fig 2.7, D and E are adjacent to B A and C
  are adjacent to D.)
• If two adjacent basic solutions are also
  feasible, then the line segment that joins them
  is called an edge of the feasible set ( one
  dimensional face).

10
2.3 Polyhedra in standard form

• Thm 2.4 P x Ax b, x ? 0 , A m ? n,
  full row rank.
• Then x is a basic solution ? x satisfies Ax
  b and ? indices B(1), , B(m) such that AB(1),
  , AB(m) are linearly independent and xi 0, i
  ? B(1), , B(m).
• Pf) see text.
• ( To find a basic solution, choose m linearly
  independent columns AB(1), , AB(m). Set xi 0
  for all i ? B(1), , B(m), then solve Ax b
  for xB(1), , xB(m). )
• Def basic variable, nonbasic variable, basis,
  basic columns, basis matrix B. (see text)
• ( BxB b ? xB B-1b )

11

• Def For standard form problems, we say that two
  bases are adjacent if they share all but one
  basic column.
• Note A basis uniquely determines a basic
  solution.
• Hence if have two different basic solutions ?
  have different basis.
• But two different bases may correspond to the
  same basic solution. (e.g. when b 0 )
• Similarly, two adjacent basic solutions ? two
  adjacent bases
• Two adjacent bases with different basic
  solutions ? two adjacent basic solutions.
• However, two adjacent bases only not necessarily
  imply two adjacent basic solutions. The two
  solutions may be the same solution.

12

• Check that full row rank assumption on A results
  in no loss of generality.
• Thm 2.5 P x Ax b, x ? 0 ? ?, A m ?
  n, rank is k lt m.
• Q x AIx bI, x ? 0 , I i1, , ik
  with linearly indep. rows.
• Then P Q.
• Pf) Suppose first k rows of A are linearly
  independent.
• P ? Q is clear. Show Q ? P.
• Every row ai of A can be expressed as ai
  ?j1k ?ijaj.
• Hence, for x ? P, bi aix ?j1k ?ij ajx
  ?j1k ?ij bj , i 1, , m
• i.e. bi is also linear combination of bj , j ?
  I.
• Suppose y ? Q, then ? i 1, , m,
• aiy ?j1k ?ij ajy ?j1k ?ij bj bi
• Hence, y ? P ? Q ? P ?

13
2.4 Degeneracy

• Def 2.10 A basic solution x ? Rn is said to be
  degenerate if more than n of the constraints are
  active at x.
• Def 2.11 P x ? Rn Ax b, x ? 0 , A m ?
  n, full row rank.
• Then x is a degenerate basic solution if more
  than n m of the components of x are 0 ( i.e.
  some basic variables have 0 value)
• For standard LP, if we have more than n m
  variables at 0 for a basic feasible solution x,
  it means that more than n m of the
  nonnegativity constraints are active at x in
  addition to the m constraints in Ax b.
• The solution can be identified by defining n-m
  nonbasic variables ( value 0). Hence,
  depending on the choice of nonbasic variables, we
  have different bases, but the solution is the
  same.

14
A
D
C
P
B
E

• Fig 2.9 A and C are degenerate basic feasible
  solutions. B and E are nondegenerate. D is a
  degenerate basic solution.

15
A
x40
x30
B
x50
P
x10
x60
x20

• Fig 2.11 (n-m)-dimensional illustration of
  degeneracy. Here, n6, m4. A is nondegenerate
  and basic variables are x1, x2, x3, x6. B is
  degenerate. We can choose x1, x6 as the nonbasic
  variables. Other possibilities are to choose x1,
  x5, or to choose x5, x6.

16

• Degeneracy is not purely geometric property, it
  may depend on representation of the polyhedrom
• ex) P x Ax b, x ? 0 , A m ? n
• P x Ax ? b, -Ax ? -b, x ? 0
• We know that P P, but representation is
  different.
• Suppose x is a nondegenerate basic feasible
  solution of P.
• Then exactly n m of the variables xi are
  equal to 0.
• For P, at the basic feasible solution x, we
  have n m variables set to 0 and additional 2m
  constraints are satisfied with equality. Hence,
  we have n m active constraints and x is
  degenerate.

17
2.5 Existence of extreme points

• Def 2.12 Polyhedron P ? Rn contains a line if
  ? a vector x ? P and a nonzero d ? Rn such that x
  ?d ? P for all ? ? R.
• Note that if d is a line in P, then A(x ?d) ?
  b for all ? ? R
• ? Ad 0
• Hence d is a vector in the lineality space S. (
  in P SKQ )
• Thm 2.6 P x ? Rn aix ? bi, i 1, , m
  ? ?, then the following are equivalent.
• (a) P has at least one extreme point.
• (b) P does not contain a line.
• (c) ? n vectors out of a1, , am, which are
  linearly independent.
• Pf) see proof in the text.

18

• Note that the conditions given in Thm 2.6 means
  that the lineality space S 0
• Cor 2.2 Every nonempty bounded polyhedron
  (polytope) and every nonempty polyhedron in
  standard form has at least one basic feasible
  solution (extreme point).

19
2.6 Optimality of extreme points

• Thm 2.7 Consider the LP of minimizing cx over
  a polyhedron P. Suppose P has at least one
  extreme point and there exists an optimal
  solution.
• Then there exists an optimal solution which is
  an extreme point of P.
• Pf) see text.
• Thm 2.8 Consider the LP of minimizing cx over
  a polyhedron P. Suppose P has at least one
  extreme point.
• Then, either the optimal cost is - ?, or there
  exists an extreme point which is optimal.

20

• (continued)
• Idea of proof in the text)
• Consider any x ? P. Let I i aix bi
• Then we move to y x ?d, where aid 0, i ?
  I and cd ? 0.
• Then either the optimal cost is - ? ( if the
  half line d is in P and cd lt 0 ) or we meet a
  new inequality which becomes active ( cost does
  not increase).
• By repeating the process, we eventually arrive
  at an extreme point which has value not inferior
  to x.
• Therefore, for any x in P, there exists an
  extreme point y such that cy ? cx. Then we
  choose the extreme point which gives the smallest
  objective value with respect to c.

21

• ( alternative proof of Thm 2.8)
• P S K Q. Pointedness of P implies S
  0.
• Hence x ? P ? x ?i qidi ?j rjuj , where
  dis are extreme rays of K and ujs are extreme
  points of P and qi ? 0, rj ? 0, ?j rj 1.
• Suppose ? i such that cdi lt 0, then LP is
  unbounded.
• ( For x?P, x ?di ? P for ? ? 0. Then c(
  x?di ) ? - ? as ? ? ?)
• Otherwise, cdi ? 0 for all i , take u such
  that cu minj cuj
• Then ? x ? P,
• cx ?i qi(cdi) ?j rj(cuj) ? ?j rj(cuj) ?
  (cu) ?j rj cu.
• Hence LP is either unbounded or ? an extreme
  point of P which is an optimal solution.
• Proof here shows that the existence of an
  extreme ray di of the pointed recession cone Ax ?
  0 ( if have min problem and polyhedron is Ax ?
  b) such that cdi lt 0 is the necessary and
  sufficient condition for unboundedness of the LP.
• ( If P has at least one extreme point, then LP
  unbounded
• ? ? an extreme ray di in recession cone K such
  that cdi lt 0) ?