MS 101: Algorithms

1
MS 101 Algorithms

• Instructor
• Neelima Gupta
• ngupta_at_cs.du.ac.in

2
Table of Contents

• Class NP
• Class NPC
• Approximation Algorithms

3
Class NP

• Class of problems for which yes can be verified
  in polynomial time.
• A Verification algorithm takes as an input, a
  problem instance, and a certificate and decides
  whether it is a yes-instance.
• A(x,y) 1 iff, y is a valid certificate.
• input certificate
• instance
• Before we elaborate on this .

4
Optimization problems and Decision problems

• Clique
• Vertex cover
• Back to verification

5
Clique problem

• Given an undirected graph G (V,E), a clique is a
  subset Vof V such that every pair of vertices is
  connected by an edge in E.
• In the graph to the right,
• vertices 1,2,5 form a
• clique since each has
• an edge to all the others.

6
G
Green ovals represent CLIQUE for this graph
7
G
G
8

• Clique
• 1 2
• 3 4
  5

9
Clique

• Optimization version Given an undirected graph
  G, find a largest size Clique in it.
• Decision version Given an undirected graph G,
  does there exist a clique of size at most k?

10
Yes verification

• Given a set of vertices V in G, does V form a
  Clique in G?

11
Vertex Cover problem

• Given an undirected graph G (V,E), a vertex
  cover is a subset Vof V such that every edge in
  E has at least one end point in V.
• In the graph to the right,
• vertices 1,2,5 form a
• clique since each has
• an edge to all the others.

12

• Vertex Cover,
  
• 1 2
• 3 4
  5

13
Vertex Cover

• Optimization version Given an undirected graph
  G, find a smallest size vertex cover in it.
• Decision version Given an undirected graph G,
  does there exist a vertex cover of size at least
  k?

14
Yes verification

• Given a set of vertices V in G, does V form a
  vertex cover in G?

15
Hamiltonian Cycle Problem

• A Hamiltonian cycle of an undirected graph, G, is
  a simple cycle that contains every vertex.
• For instance, consider the graph G

16

• Forming an HC,
  
• 1 2
• 3 4
  5

17

• However, for the graph G, there does not exist
  any Hamiltonian cycle.
• 1) with vertices 1,2,3,4 not an HC
  
• 2) with vertices 1,2,5,6,3 again not an HC
  
• 1 2
• 3 4
  5
• 6

18
Hamiltonian Cycle Problem

• The Hamiltonian-cycle problem given a graph G,
  find a Hamiltonian cycle in it?
• Ham _cycle ltGgt G is hamiltonian.
• Decision version Given an undirected graph G,
  does it contain a Hamiltonian Cycle?

19
Yes verification

• Given a sequence of vertices V in G, does V
  form a Hamiltonian cycle in G?

20
Class NP

• Defined for decision problems.
• The class of decision problems for which there is
  a polynomially bounded nondeterministic
  algorithm.

21
No verification is not easy

• Examples
• Clique
• Vertex Cover
• Hamiltonian

22
Class Co- NP

• Class of problems for which No can be verified
  in polynomial time.
• In other words, it is the set of problems whose
  complement is in NP.

23
Shortest Path

• yes and no both the certificates exist and
  can be verified in polynomial time. Thus SP is in
  the intersection.
• My verification algorithm will compute the
  shortest path in polynomial time and without
  explicitly using the certificate will answer
  yes or no as the case may be.

24
Class P is in the intersection

• Similarly, for every problem in P, both the
  certificates exist and can be verified in
  polynomial time.
• The verification algorithm being the polynomial
  time algorithm to solve the problem itself.
• Hence P is in the intersection.

25
Relation between class P, NP and Co-NP

Co-NP
NP n Co-NP
NP
P
26
NP-Completeness

• Instructor Ms. Neelima Gupta

27
Class NPC

• NP-completeness does not apply directly to
  optimization problems, but to decision problems.
• As we have seen we can obtain a decision version
  of an optimization problem, which will be
  easier or at least, no harder than the
  optimization problem.
• Optimization problem is the harder of the two.

28
Defining NPC

• Let Q be any problem at hand, it belongs to class
  NPC,
• if,
• Q ? NP
• Q is NP hard

29
NP hard

• Q belongs to class NP-hard
• 1. If ? P ? NP, and
• 2. P ?p Q
• In step 2. we are mapping an instance of problem
  P
• to an instance of problem Q.

Poly. Time
IQ
IP
IP Instance of problem P IQ Instance of
problem Q SP Solution of problem P SQ
Solution of problem Q
Poly. Time
SQ (Yes/No)
SP (Yes/No)
30
Transformation Characterstics

• If A(Q) is yes then A(P) is yes
• Vice versa
• It should be done in polynomial time

31
Diagrammatically
NP HARD
NPC
32

• If all problems R ? NP

are reducible to T, then T is NP-Hard
And T ? NP then T is NPC.
T

• Q

If T ?p Q then Q is NP hard.
33
3CNF ?p Clique (Needs to be improved)

• 3CNF problem n variables constituting k clauses
• 3CNF is a satisfiability problem, i.e.
• there exists an assignment expression
  results in T.
• From Cooks Th. The satisfiability problem is
  NPC.
• Hence, 3CNF is NPC.
• If 3CNF reduces to Clique then clique is NP hard.

34

• Let the expression in 3CNF be (x v y v z) (x
  v y v z) (x v y v z)
• Expression ? Graph

x y z
x y z
x y z
35
Clique thus formed
y
x
z
Note- There are many other possible cliques in
previous mapping. This is one of the possible
cliques.
36
Clique ?p Vertex Cover

• Let the instance of Clique ( Ic) be ltG, kgt.
• Reducing it to instance of VC (Ivc) be ltG,
  V-kgt
• where G E(G)Edges b/w vertex pair not
  present in G and V-k is the vertex cover.
• Catch behind this choice Because it works!!!

37
G
Green ovals represent CLIQUE for this graph
38
G
G
39
G
G
Big ovals represent the VC for graph G
40
Independent sets

• Instructor Ms. Neelima Gupta

41
INDEPENDENT SET

• Given an undirected graph, A subset S of V is
  said to be independent if no two nodes in S are
  joined by an edge in G.

42
Independent set of size 2
43
Independent set of size 3
44
IS is in NPC

• Assignment reduce vertex cover to IS.

45
Constrained Optimization Problems

• An objective function is optimized subject to
  certain constraints

46
Subset Sum Problem

• Instructor Ms. Neelima Gupta

47
Subset Sum Problem

• Problem Statement
• Given a finite set S x1, x2, xn of n
  elements with non-negative weights wi and a
  target t, select a subset S of items such that
  the sum of weights of elements in S is at most t
  (constraint) and,
• Optimization version the sum of weights is
  maximized.
• Decision Version sum of weights is at least k,
  for a given k.
• Note without the constraint, problem is not
  interesting (why?)
• as you would just pick all the elements.

48
Subset Sum Problem

• We now prove that Subset Sum Problem is
• NP-Complete.
• Subset Sum is in NP. For an instance
• ltS,tgt, let S be the certificate. Checking
• whether elements of S sum upto t and is at least
  k can be done in polynomial time.

49
Subset Sum Problem

• Subset Sum is NP Hard
• We show this by proving that 3-SAT is
• reducible to Subset Sum in polynomial time.
• Given 3-SAT formula ? over variables x1, x2,
  xn with clauses C1, C2Ck

50
Subset Sum Problem

• Without loss of generality, we make the
• following 2 assumptions
• No clause contains both a variable and its
  negation. WHY?
• (Because such a clause would be trivially
  satisfied.)
• Each variable appears in at least 1 clause. WHY?
• (Because otherwise, it does not matter what
  value is assigned to it.)

51
Subset Sum Problem

• Reduction Process - through example
• Consider the 3-SAT formula ? C1 C2 C3 C4
• where C1 (x1 v x2 v x3)
• C2 (x1 v x2 v x3)
• C3 (x1 v x2 v x3)
• C4 (x1 v x2 v x3)
• A satisfying assignment is ltx10, x20, x31gt

52
Subset Sum Problem

• Construct S and t as follows
• For each xi, add 2 integers vi, vi in S. Both vi
  and vi have 1 corresponding to digit xi.
• If xi appears in Cj, the Cj-digit in vi 1
• If vi appears in Cj, the Cj-digit in vi 1
• All other digits are zero.

53
Subset Sum Problem

• For all Cj, add sj and sj integers in S. Both
  have 0s in all digits other than the one labeled
  by Cj.
• sj has a 1 corresponding to Cj, and sj has a 2
  corresponding to Cj.
• These integers are slack variables, used to get
  clause labeled digit position to add to the
  target value of 4.
• Target t has a 1 in each digit labeled by a
  variable and 4 in each clause-digit.
• k t Thus the instance of SS requires that the
  sum of weights of the selected elements be
  exactly equal (at most and at least) to t.

54
Subset Sum Problem

• Claim All vi and vi in S are unique
• vi (vi) and vj (vj ) will be different in most
  significant positions.
• vi and vi will be different in least significant
  positions. WHY?
• (both cannot belong to the same clause)

55
Subset Sum Problem

• Claim All sj and sj in S are unique
• (for reasons similar to vi and vi)
• Observation The greatest sum of digits in
• any digit position is 6. This occurs in clause-
• digits (vi and vi make a contribution of 3, sj
• and sj make a contribution of 1 and 2
• respectively).
• Conclusion Interpretation is in base 10, so no
• carries would be generated.

56
(x1 v x2 v x3) ? (x1 v x2 v x3) ? (x1 v x2
v x3) ? (x1 v x2 v x3)
C4
C3
C2
C1
X3
X2
X1
 
1
0
0
1
0
0
1
V1
1
0
0
0
0
1
0
V2
0
1
1
1
0
1
0
V2'
1
1
0
0
1
0
0
V3
0
0
1
1
1
0
0
V3'
0
0
0
1
0
0
0
S1
0
0
0
2
0
0
0
S1'
0
0
1
0
0
0
0
S2
0
0
2
0
0
0
0
S2'
0
1
0
0
0
0
0
S3
0
2
0
0
0
0
0
S3'
1
0
0
0
0
0
0
S4
2
0
0
0
0
0
0
S4'
4
4
4
4
1
1
1
t
57
Solution to 3CNF gt Solution to Subset Sum

• Do the following for i 1n
• If xi 1 in the assignment, include vi in S,
  otherwise include vi. In the example,
• x10 gt x11 , v1 is selected
• x20 gt x21 , v2 is selected
• x31 gt x31 , v3 is selected
• Include Sis as per the shortfall.

58

• (x1 v x2 v x3) ? (x1 v x2 v x3) ? (x1 v x2
  v x3) ? (x1 v x2 v x3)
• Satisfying assignment x10, x20, x31
• v1 , v2 and v3 are selected.

C4
C3
C2
C1
X3
X2
X1
 
1
0
0
1
0
0
1
V1
1
0
0
0
0
1
0
V2
0
1
1
1
0
1
0
V2'
1
1
0
0
1
0
0
V3
0
0
1
1
1
0
0
V3'
0
0
0
1
0
0
0
S1
0
0
0
2
0
0
0
S1'
0
0
1
0
0
0
0
S2
0
0
2
0
0
0
0
S2'
0
1
0
0
0
0
0
S3
0
2
0
0
0
0
0
S3'
1
0
0
0
0
0
0
S4
2
0
0
0
0
0
0
S4'
4
4
4
4
1
1
1
t
59
(x1 v x2 v x3) ? (x1 v x2 v x3) ? (x1 v x2
v x3) ? (x1 v x2 v x3) Satisfying assignment
x10, x20, x31
C4
C3
C2
C1
X3
X2
X1
 
1
0
0
1
0
0
1
V1
1
0
0
0
0
1
0
V2
0
1
1
1
0
1
0
V2'
1
1
0
0
1
0
0
V3
0
0
1
1
1
0
0
V3'
0
0
0
1
0
0
0
S1
0
0
0
2
0
0
0
S1'
0
0
1
0
0
0
0
S2
0
0
2
0
0
0
0
S2'
0
1
0
0
0
0
0
S3
0
2
0
0
0
0
0
S3'
1
0
0
0
0
0
0
S4
2
0
0
0
0
0
0
S4'
4
4
4
4
1
1
1
t
60
(x1 v x2 v x3) ? (x1 v x2 v x3) ? (x1 v x2
v x3) ? (x1 v x2 v x3) Satisfying assignment
x10, x20, x31
C4
C3
C2
C1
X3
X2
X1
 
1
0
0
1
0
0
1
V1
1
0
0
0
0
1
0
V2
0
1
1
1
0
1
0
V2'
1
1
0
0
1
0
0
V3
0
0
1
1
1
0
0
V3'
0
0
0
1
0
0
0
S1
0
0
0
2
0
0
0
S1'
0
0
1
0
0
0
0
S2
0
0
2
0
0
0
0
S2'
0
1
0
0
0
0
0
S3
0
2
0
0
0
0
0
S3'
1
0
0
0
0
0
0
S4
2
0
0
0
0
0
0
S4'
4
4
4
4
1
1
1
t
61
(x1 v x2 v x3) ? (x1 v x2 v x3) ? (x1 v x2
v x3) ? (x1 v x2 v x3) Satisfying assignment
x10, x20, x31
C4
C3
C2
C1
X3
X2
X1
 
1
0
0
1
0
0
1
V1
1
0
0
0
0
1
0
V2
0
1
1
1
0
1
0
V2'
1
1
0
0
1
0
0
V3
0
0
1
1
1
0
0
V3'
0
0
0
1
0
0
0
S1
0
0
0
2
0
0
0
S1'
0
0
1
0
0
0
0
S2
0
0
2
0
0
0
0
S2'
0
1
0
0
0
0
0
S3
0
2
0
0
0
0
0
S3'
1
0
0
0
0
0
0
S4
2
0
0
0
0
0
0
S4'
4
4
4
4
1
1
1
t
62
(x1 v x2 v x3) ? (x1 v x2 v x3) ? (x1 v x2
v x3) ? (x1 v x2 v x3) Satisfying assignment
x10, x20, x31
C4
C3
C2
C1
X3
X2
X1
 
1
0
0
1
0
0
1
V1
1
0
0
0
0
1
0
V2
0
1
1
1
0
1
0
V2'
1
1
0
0
1
0
0
V3
0
0
1
1
1
0
0
V3'
0
0
0
1
0
0
0
S1
0
0
0
2
0
0
0
S1'
0
0
1
0
0
0
0
S2
0
0
2
0
0
0
0
S2'
0
1
0
0
0
0
0
S3
0
2
0
0
0
0
0
S3'
1
0
0
0
0
0
0
S4
2
0
0
0
0
0
0
S4'
4
4
4
4
1
1
1
t
63
Solution to SS gt Solution to 3CNF

• For each clause j, maximum that can be picked
  from Sjs is 3. Therefore at least for one i, at
  least one of vi or vi must have been picked in
  S. Set that to 1.
• Claim there is no conflict.
• Claim the assignment is satisfying.
• Proof do it yourself.

64
Proving NP-hardness by generalization
65
0-1 Knapsack Problem

• Instructor Ms. Neelima Gupta

66
The 0-1 Knapsack Problem

• Input
• A set S of n items with weights wi and values vi
• Capacity K
• Output a subset S of items such that
• the sum of weights of items in S is at most K
  (constraint)
• and the sum of values of items in S is maximized

67
0-1 Knapsack is NP-hard

• Its a generalization of subset sum
• For a particular case when vi wi, it becomes SS.

68
Set Cover Problem

• Instructor Ms. Neelima Gupta

69
Set Cover Problem

• Problem Statement
• Given
• A set U of n elements
• A collection S1, S2,., Sm of subsets of U
• A number k
• Optimization version find the smallest
  collection of sets that cover the elements of U.
• Decision Version Does there exist a collection
  of at most k of these sets that cover the
  elements of U.

70
Set Cover Problem is NP hard

• Its a generalization of vertex cover.
• How?
• Do it yourself

71
Traveling Salesman Problem

• Instructor Ms. Neelima Gupta

72
(Metric) Traveling Salesman Problem

• Problem Statement
• Given A complete graph G with non-
• negative edge costs ( that satisfy
• triangle inequality)
• optimization Version find a minimum cost cycle
  visiting
• every vertex exactly once.
• Decision Version Does there exist a TS tour of
  cost ltk

73
TSP is NP hard

• Reduction from Hamiltonian Cycle.
• Hamiltonian Cycle is a particular case of TSP.
• How?
• Include an edge of G in G with cost 0 and an
  edge not in G with cost 1 and set k to 0.

74
Closely related problems to SS and 0-1KS

• Bin Packing Problem
• Partition problem

75
Bin Packing Problem

• Given a set of items S x1xn each with some
  weight wi, pack all the items into a collection
  of finite number of bins each with some capacity
  Bi using
• Optimization version minimum number of bins.
• Decision version at most k bins, for a given k.

76
Partition Problem
77
Up Next

• Approximation Algorithms for some of these
  problems

78
Thank You!