ESI 6448 Discrete Optimization Theory

1
ESI 6448Discrete Optimization Theory

• Section number 5643

2
Course Administration

• Lectures MW (510640) _at_ FLG
  280
• Instructor Distinguished Professor Panos M.
  Pardalosemail pardalos_at_ufl.eduhttp//www.ise.uf
  l.edu/pardalos/
• TA Ashwin Arulselvanemail ashwin_at_ufl.edu
• Course webpage http//www.ise.ufl.edu/ESI6448

3
Course grading

• Homework assignments (30), and exams (60),
  class participation (10).
• Exam dates TBD
• Check class webpage
• Homework policy
• Discussion on problem allowed but you must write
  answer IN YOUR OWN WRITING

4
Applications of IP

• Train scheduling
• Airline crew scheduling
• Production planning
• Electricity generation planning
• Telecommunications
• Ground holding of aircraft
• Cutting problems

5
MIP, IP, BIP, COP

• LP max cx Ax b, x 0
• MIP max cx hy Ax Gy b
  x 0, y 0 and integer
• IP (BIP) max cx Ax b
  x 0 and integer (x ? 0, 1)
• COP minS?N ?j?S cj S ? F

6
Example

• Max 1.00x1 0.64x2 50x1 31x2 250
  3x1 2x2 -4 x1, x2 0 and integer

(376/193, 950/193)
Rounding?
(5, 0)
7
Formulations (IP, BIP)

• Define variables
• Define constraints using variables
• Define objective function using variables
• For COP
• S ? N
• Incidence vector xS of S s.t.xSj 1 if j ? S,
  xSj 0 otherwise.

8
The assignment problem

• Description
• n people to carry out n jobs
• Each person is assigned to exactly one job
• Estimated cost cij if person i is assigned to job
  j
• Find a minimum cost assignment

9
The assignment problem

• Formulation
• Variables
• xij 1 if person i does job j, 0 o.w.
• Constraints
• Snj1 xij 1 for i 1, , n
• Sni1xij 1 for j 1, , n
• xij ? 0, 1 for i, j 1, , n
• Objective
• min Sni1Snj1cijxij

10
0-1 knapsack problem

• Description
• Budget b available and n projects
• aj outlay for project j
• cj expected return of project j
• Choose projects to maximize return

11
0-1 knapsack problem

• Formulation
• Variables
• xj 1 if project j is selected, 0 o.w.
• Constraints
• Snj1 ajxj b
• xj ? 0, 1 for j 1, , n
• Objective
• max Snj1cjxj

12
Set covering problem

• Description
• m regions, n possible service centers
• For each possible center j
• Cost of installation cj
• Service area (regions) Sj
• Choose a minimum cost service centers

13
Set covering problem

• Formulation (COP)
• M 1, , m
• N 1, , n
• Sj ? M regions serviced by a center j
• cj installation cost for center j
• minT?N Sj?T cj ?j?T Sj M

14
Set covering problem

• Formulation (BIP)
• 0-1 incidence matrix A s.t.aij 1 if i ? Sj, 0
  o.w.
• Variables
• xj 1 if center j is selected, 0 o.w.
• Constraints
• Snj1 aijxj 1 for i 1, , m
• xj ? 0, 1 for j 1, , n
• Objective
• Min Snj1 cjxj

15
Traveling Salesman Problem (TSP)

• Description
• A salesman must visit n cities exactly once and
  return to his starting point
• cij travel time from city i to city j
• Find the quickest tour

16
Traveling Salesman Problem (TSP)

• Formulation
• Variables
• xij 1 if travel from city i to city j, 0 o.w.
• xii not defined for i 1, , n
• xij ? 0, 1 for i, j 1, , n, i ? j
• Constraints
• Sjj?i xij 1 for i 1, ..., n
• Sii?j xij 1 for j 1, ..., n
• Subtours?
• Objective
• min Sni1Snj1cijxij

17
Traveling Salesman Problem (TSP)

• To eliminate subtours
• cut-set constraints
• Si?SSj?S xij 1 for S ? N, S ? ?
• subtour elimination constraints
• Si?SSj?S xij S - 1 for S ? N, 2 S n 1

18
Combinatorial explosion

• Combinatorial problem solvable by enumeration
• Is it a feasible approach?depends on the number
  of possible solutions
• The assignment problem n!
• The knapsack and covering problem 2n-1
• TSP (n-1)!

19
Mixed integer formulations

• fixed charge cost function
• h(x) f px if 0 lt x ? C 0 if x 0,
  with f gt 0 and p gt 0
• additional variable
• y 1 if x gt 0 and y 0 o.w.
• replace h(x) by fy px and add constraintsx ?
  Cy, y ? 0, 1
• x0 and y1 ?

h(x)
fpx
0
C
x
20
Uncapacitated facility location (UFL)

• Description
• n potential depots and m clients
• fj fixed cost to use depot j
• cij transportation cost if all of clients is
  order is delivered from depot j
• Decide which depots to open and which depot
  serves each client so as to minimize the sum of
  costs

21
Uncapacitated facility location (UFL)

• Formulation
• Variables
• yj 1 if depot j is used, 0 o.w. (fixed cost
  variable)
• xij fraction of demand of client i satisfied
  from depot j
• Constraints
• Snj1 xij 1 for i 1, ..., m
• Smi1 xij ? mSmi1 xij ? myj for j ? N, xij ? 0
  for i ? M, j ? N, yj ?
  0, 1 for j ? N
• Objective
• min Si?MSj?Ncijxij Sj?Nfjyj

22
Uncapacitated lot-sizing (ULS)

• Description
• n-period horizon for a single product
• ft fixed cost of producing in period t
• pt unit production cost in period t
• ht unit storage cost in period t
• dt demand in period t
• Decide a minimum cost production plan

23
Uncapacitated lot-sizing (ULS)

• Formulation
• Variables
• xt amount produced in period t
• st stock at the end of period t
• yt 1 if production occurs in period t, 0 o.w.
• Constraints
• xt ? Myt for t 1, ..., n (M very large value)
• st-1 xt dt st for t 1, ..., n
• s0 0, st, xt ? 0, yt ? 0, 1 for t 1, ..., n
• Objective
• min Snt1ptxt Snt1htst Snt1ftyt

24
Discrete alternatives

• Disjunctions
• 0 ? x ? u andeither a1x ? b1 or a2x ?
  b2
• Let M ? max aix bi 0 ? x ? u
• aix bi ? M(1 yi) for i 1, 2
• y1 y2 1, yi ? 0, 1 for i 1, 2
• 0 ? x ? u

x2
a1xb1
a2xb2
x1
25
Alternative formulations

• Def. A subset of Rn described by a finite set of
  linear constraints P x ? Rn Ax b is a
  polyhedron.
• Def. A polyhedron P ? Rnp is a formulation for a
  set X ? Zn ? Rp if and only if X P ? (Zn ? Rp).

26
Example

• X (1, 1), (2, 1), (3, 1), (1, 2), (2, 2,), (3,
  2), (2, 3)

P1
3
P2
2
1
0
1
2
3
4
27
Equivalent formulation for UFL

• Formulation
• Variables
• yj 1 if depot j is used, 0 o.w. (fixed cost
  variable)
• xij fraction of demand of client i satisfied
  from depot j
• Constraints
• Snj1 xij 1 for i 1, ..., m
• Smi1 xij ? mSmi1 xij ? myj for j ? N, xij ? 0
  for i ? M, j ? N, yj ?
  0, 1 for j ? N
• Objective
• min Si?M Sj?Ncijxij Sj?Nfjyj

28
Equivalent formulation for UFL

• For fixed j,
• Smi1 xij ? myj, yj ? 0, 1, 0 xij 1 for i ?
  M
• if any xij gt 0, then yj 1, orfor each i, if
  xij gt 0, then yj 1gt 0 xij yj for i ? M,
  yj ? 0, 1
• alternative formulation
• min Smi1 Snj1cijxij Snj1fjyj
• Snj1xij 1 for i ? M
• xij yj for i ? M, j ? N
• xij 0 for i ? M, j ? N, yj ? 0, 1 for j ? N

29
Extended formulations

• Instead of different constraints, use different
  variables
• In ULS, we want to know when the items being
  produced now will be used to satisfy demand
• Variables
• wit amount produced in period i to satisfy
  demand in t
• yt 1 if production occurs in period t, 0 o.w.
• Constraints
• Sti1wit dt for all t
• wit dtyi for all i, t, i t
• wit 0 for all i, t, i t, yt ? 0, 1 for all
  t
• Objective
• min Sni1Snt1ciwit Snt1ftyt

30
Good and ideal formulations

• ideal formulation P if we solve a linear
  program over P, the optimal solution is at an
  (integer) extreme point

P1
3
P2
P3
2
1
0
1
2
3
4
31
Good and ideal formulations

• Def. Given a set X ? Rn, the convex hull of X,
  denoted conv(X), is defined asconv(X) x x
  Sti1?ixi, Sti1 ?i 1, ?i0 for i 1, , t
  over all finite subsets x1, , xt of X
• Prop. conv(X) is a polyhedron.
• Prop. The extreme points of conv(X) all lie in X.
• IP max cx x ? Xgt LP max cx x ?
  conv(X)

32
Good and ideal formulations

• Ideal solution conv(X) satisfies X ? conv(X) ? P
  for all formulations P
• Def. Given a set X ? Rn, and two formulations P1
  and P2 for X, P1 is better formulation than P2
  if P1 ? P2

33
Formulations for UFL

• P1 Si?Mxij myjP2 xij yj for i ? M
• P2 ? P1?
• P2 ? P1?
• need to find a point in P1 that is not in P2
• suppose m kn, each depot serves k clients
• there is a point in P1\ P2 s.t.xij 1 for k(j
  1) 1 i k(j 1) k, 1jn, 0
  o.w.,yj k / m for 1 j n

34
Projection of formulations

• Assume all the variables are integer
• mincx x ? P ? Zn, P ? Rn
• mincx (x, w) ? Q ? (Zn ? Rp), Q ? Rn ? Rp
• Def. Given a polyhedron Q ? (Rn ? Rp), the
  projection of Q onto the subspace Rn, denoted
  projxQ, is defined asprojxQ x ? Rn (x,
  w)?Q for some w?Rp

35
Formulations for ULS

• P1
• st-1 xt dt st for t 1, , n
• xt Myt for t 1, , n
• s0 0, st, xt 0, 0 yt 1 for all t
• P2 projx,s,yQ2, where Q2
• Sti1wit dt for all t
• wit dtyi for all i, t, i t
• xi Sntiwit for all i
• wit 0 for all i, t, i t
• 0 yt 1 for all t

36
Optimality

• Given z maxc(x) x ? X ? Zn
• is x optimal?
• optimality condition as stopping criteria
• lower / upper bound z / z s.t.z1 gt z2 gt gt zs
  z andz1 lt z2 lt lt zt z
• stop when zs zt ?

37
Bounds

• Primal bounds
• every feasible solution x ? X provides a lower
  bound z c(x) z
• Dual bounds
• by relaxation
• replace a difficult max(min) IP by a simpler
  optimization problem whose optimal value is at
  least as large (small) as z
• enlarge the set of feasible solutions
• replace max(min) objective function by a function
  that has the same or larger (smaller) value
  anywhere

38
Bounds

• Def. A problem (RP) zR maxf(x) x ? T ? Rn
  is a relaxation of (IP) z maxc(x) x ? X ?
  Rn ifi) X ? T andii) f(x) c(x) for all x ?
  X
• Prop. If RP is a relaxation of IP, zR z
• Proof. If x is an optimal solution of IP, x ? X
  ? T and z c(x) f(x).Since x ? T, f(x) is
  a lower bound on zR, so z f(x) zR.

39
Linear programming relaxation

• Def. For the integer program maxcx x ? P ? Zn
  with formulation P x ? Rn Ax b, the
  linear programming relaxation is the linear
  program zLP maxcx x ? P
• P ? Zn ? P, no change in objective functiongt
  relaxation!

40
Example

• z max 4x1 x2 7x1 2x2 14
  x2 3 2x1 2x2
  3 x ? Z2
• (2, 1) is a feasible solution gt primal (lower)
  bound z 7
• optimal solution for LP relaxation x (20/7,
  3) with zLP 59/7 gt dual (upper) bound z 59/7
  gt (round to integer) z 8

41
Linear programming relaxation

• Prop. Suppose P1, P2 are two formulations for the
  integer program maxcx x ? X ? Zn with P1 a
  better formulation than P2. If zLPi maxcx x
  ? Pi for i 1, 2 are the values of the
  associated linear programming relaxations, then
  zLP1 zLP2 for all c.
• Prop. i) If a relaxation RP is infeasible, the
  original problem IP is infeasible.ii) Let x be
  an optimal solution of RP. If x ? X and f(x)
  c(x), then x is an optimal solution of IP.

42
Linear programming relaxation

• Proof. i) As RP is infeasible, T ? and thus X
  ?.ii) As x ? X, z c(x) f(x) zR. As z
  zR, c(x) z zR
• max 7x1 4x2 5x3 2x4 3x1 3x2 4x3
  2x4 6 x ? B4
• optimal solution x (1, 1, 0, 0) which is
  integral. So x is the solution for the IP.

43
Combinatorial relaxation

• COP gt combinatorial relaxation
• relax to an easy problem
• TSP
• digraph D (V, A) and cij for (i, j) ? A
• Tours are assignments without subtours!
• zTSP minT?A ?(i, j)?T cij T forms a tour
  ?zASS minT?A ?(i, j)?T cij T forms an
  assignment

subtours
permutations
44
Combinatorial relaxation

• Symmetric Traveling Salesman Problem (STSP)
• graph G (V, E) and ce for e ? E
• Find an undirected tour of minimum weight
• observations
• Every tour consists of 2 edges adjacent to node 1
  and a path through nodes 2, ..., n
• A path is a special case of a tree
• Def. A 1-tree is a subgraph consisting of two
  edges adjacent to node 1, plus the edges of a
  tree on nodes 2, ..., n

1
1
1-tree
tour
45
Combinatorial relaxation

• STSP (contd)
• Every tour is a 1-tree s.t. the tree on nodes 2,
  ..., n forms a path!
• zSTSP minT?E ?e?T ce T forms a tour
  ?z1-tree minT?E ?e?T ce T forms a 1-tree
• Quadratic 0-1 problem
• max ?i, j 1?iltj ?n qijxixj ?nj1 pjxj, x ? 0,
  x ? Bn
• Replace qijxixj with qij lt 0 by 0Let qij
  max(qij, 0)max ?i, j 1?iltj ?n qijxixj ?nj1
  pjxj, x ? 0, x ? Bn

46
Combinatorial relaxation

• Knapsack problem
• X x ? Zn ?nj1 ajxj ? bX x ? Zn
  ?nj1??aj?xj ? ?b?
• Chvátal-Gomory rounding method
• ?nj1 ajxj ? b
• ?nj1 ?aj?xj ( ? ?nj1 ajxj) ? b
• ?nj1??aj?xj ? ?b?

47
Lagrangian relaxation

• IP z max cx Ax ? b, x ? X
• Drop complicating constraints Ax ? b
• Add them into the objective function with
  Lagrange multipliers (dual variables)
• Prop. Let z(u) max cx u(b Ax) x ? X.
  Then z(u) ? z for all u ? 0.
• Proof. x optimal for IPx is feasible in IP,
  so x ? X and Ax ? bu ? 0, so cx ? cx u(b
  Ax) ? z(u)

48
Duality

• Duality provides a standard way to obtain upper
  bounds in linear programming
• value of any feasible solution of dual problem
  provides an upper bound
• relaxation does so only by optimal solution
• Def. The two problems(IP) z max c(x) x ?
  X(D) w min f(u) u ? Uform a (weak)-dual
  pair if c(x) ? f(u) for all x ? X and all u ?
  U.When z w, they form a strong-dual pair.

49
Duality

• Prop. z max cx Ax ? b, x ? Zn and wLP
  min ub uA ? c, u ? Rm form a weak-dual pair.
• Prop. Suppose IP and D are a weak-dual pair.i)
  If D is unbounded, IP is infeasible.ii) If x ?
  X and u ? U satisfy c(x) w(u), then x is
  optimal for IP and u is optimal for D.

50
Matching dual

• G (V, E)
• matching M ? E a set of disjoint edges, i.e.
  set of edges s.t. each node has at most one edge
  in M incident to it
• covering by nodes R ? V a set of nodes s.t.
  every edge has at least one endpoint in R

1
2
3
4
5
8
6
7
51
Matching dual

• Prop. maxM?E M M is a matching andminR?V
  R R is a covering by nodes form a weak-dual
  pair.
• Proof. M (i1, j1), ..., (ik, jk) is a
  matching.2k nodes i1, j1, ..., ik, jk are
  distinct.Any covering by nodes R must contain at
  least one node from each pair is, js.R ? k ?
  M

52
Matching dual

• Using linear programming duality
• Def. The node-edge incidence matrix of G (V, E)
  is an n ( V) by m (E) 0-1 matrix A with aj,
  e 1 if node j is adjacent to edge e, 0 o.w.

1
1
2
3
2
3
6
4
5
?
4
5
8
9
7
10
8
6
11
12
7
53
Matching dual

• matching z max 1x Ax ? 1, x ?
  Zmcovering w min 1y yA ? 1, y ? Zm
• zLP and wLP values of LP relaxationsz ? zLP
  wLP ? w1x ? z ? w ? 1y
• No strong duality between the two problems

z 1 and w 2 (z lt w) xe1 xe2 xe3 ½ is
feasible for the first LP relaxation y1 y2 y3
½ is feasible for the second LP relaxation zLP
wLP 3/2
1
2
3
1
2
3
54
Primal bounds

• Greedy and local search
• greedy heuristic to find an initial feasible
  solution, called the incumbent
• choose at each step the item bringing the best
  reward
• local search heuristic to improve the solution
• define the neighborhood of solutions close to the
  incumbent
• best solution in the neighborhood is found
• if it is better than the incumbent, it replace
  the incumbent
• o.w. the incumbent is locally optimal, terminate

55
Greedy heuristic example

• 0-1 knapsack problem max 12x1 8x2 17x3
  11x4 6x5 2x6 2x7 4x1 3x2 7x3
  5x4 3x5 2x6 3x7 ? 9 x ? B7
• variables are ordered, i.e. cj / aj ? ci1 / aj1
• x1 1 ? 9 4 5
• x2 1 ? 5 3 2
• x3 x4 x5 0 (7, 5, 3 gt 2)
• x6 1 ? 2 2 0
• x7 0 (3 gt 0)
• greedy sol xG (1, 1, 0, 0, 0, 1, 0), zG cxG
  22

56
Greedy heuristic example

• STSP with distance matrix
• order ej s.t. cej ? cej1
• xe1 1 e1 (1, 3), ce1 2
• xe2 1 e2 (4, 6), ce2 3
• xe3 1 e3 (3, 6), ce3 6
• xe4 0 e4 (2, 3), ce4 7
• xe5 0 e5 (1, 4), ce5 8
• xe6 1 e6 (1, 2), ce6 9
• xe7 1 e7 (2, 5), ce7 10
• ?
• xe13 1 e13 (4, 5), ce13 24

57
Local search example

• UFL with m6 clients, n4 depots, costs as below
• define neighborhood and find local optimum
• S0 1, 2 c(S0) (212973)2116 61
• Q(S0) 1c63, 2c66, 1, 2, 3c60, 1, 2,
  4c84
• Q(S1 1, 2, 3) 1, 2, 1, 3, 2, 3c42,
  1, 2, 3, 4
• Q(S2 2, 3) 2, 3c31, 1, 2, 3, 1,
  2, 4
• Q(S3 3c31) ?, 1, 3, 2, 3, 3, 4

58
Local search example

• Graph equipartition problem
• G (V, E) and n V
• Find S ? V s.t. S ?n/2? and ?(S, V \ S) is
  minimized, where ?(S, V \ S) (i, j) ? E i?S,
  j?S
• neighborhood

• S0 1, 2, 3, c(S0) 6
• Q(S0) 1, 2, 46, 1, 2, 55, 1, 2, 64, 1,
  3, 44, 1, 3, 55, 1, 3, 66,
  2, 3, 45, 2, 3, 52, 2, 3,
  65
• Q(S1 2, 3, 52) 2, 3, 1, 2, 3, 4, 2,
  3, 6, 2, 1,
  5, 2, 4, 5, 2, 6, 5,
  1, 3, 5, 4, 3, 5, 6, 3, 5

1
2
6
3
5
4
59
Well-solved problems

• efficient algorithm to solve the problem?
• an algorithm on G (V, E), V n, E m is
  efficient if it requires O(mp) calculations in
  worst case, for some integer p, assuming m ? n
• Def. The Separation Problem associated with COP
  is the problem Given x ? Rn, is x ? conv(X)?
  If not, find an inequality ?x ? ?0 satisfied by
  all points in X, but violated by the point x.

60
Well-solved problems

• i) Efficient Optimization Property For a given
  class of optimization problems (P) max cx x ?X
  ? Rn, there exists an efficient (polynomial)
  algorithm.
• ii) Strong Dual Property For the given problem
  class, there exists a strong dual problem (D) min
  w(u) u ? U allowing us to obtain optimality
  conditions that can be quickly verified x ? X
  is optimal in P iff there is u ? U s.t. cx
  w(u)
• iii) Efficient Separation Property There exists
  an efficient algorithm for the separation problem
  associated with the problem class.
• iv) Explicit Convex Hull Property A compact
  description of the convex hull conv(X) is known,
  which in principle allows us to replace every
  instance by the linear programmax cx x ?
  conv(X)

61
Totally unimodular

• (IP) max cx Ax ? b, x ? Zn(LP) max cx Ax
  ? b, x ? Rn
• optimal sol of LP is integral, the solution is
  optimal for IP
• basic feasible sol x for LP x (xB, xN)
  (B-1b, 0), where B is m?m nonsingular submatrix
  of (A, I)
• Obs. (Sufficient condition) If the optimal basis
  B has det(B) ?1, the LP relaxation solves IP.
• Proof. From Cramers rule, B-1 B/det(B), B
  adjoint matrix of B.The entries of B are all
  integral, so the entries of B-1 are also all
  integral.B-1b is integral for all integral b.

62
Totally unimodular

• Def. A matrix A is totally unimodular (TU) if
  every square submatrix of A has determinant 1, -1
  , 0.
• not TU
  TU
• det 2
• det 2

63
Totally unimodular

• Obs. If A is TU, aij ? 1, -1, 0 for all i, j.
• Prop. A matrix A is TU iffi) the transpose
  matrix AT is TU iffii) the matrix (A, I) is TU.
• Prop. (Sufficient condition) A matrix A is TU
  ifi) aij ? 1, -1, 0 for all i, jii) Each
  column contains at most two nonzero coefficients
  (?mi1aij ? 2)iii) There exists a partition
  (M1, M2) of the set M of rows s.t. each column j
  containing two nonzero coefficients satisfies
  ?i?M1aij ?i?M2aij 0.

64
Totally unimodular

• Proof. Assume A is not TU satisfying all three
  conditions.Let B be the smallest square
  submatrix of A with determinant ? 1, -1, 0.B
  cannot have a column with single nonzero
  entry.(If so, a submatrix of B w/o the row and
  col of the entry will have determinant ? 1, -1,
  0.)So B contains two nonzero entry in each
  column.From iii), ?i?M1Bi ?i?M2Bi 0.So
  det(B) 0, contradiction.

65
Totally unimodular

• aij ? 1, -1, 0
• Each column has exactly two nonzero entries.
• We can partition the set M of rows into (M1, M2)
  s.t. M1 M and M2 ?. If we add all the rows,
  we will get a zero vector.

TU satisfying the proposition (three conditions)