ICS 353: Design and Analysis of Algorithms

1
ICS 353 Design and Analysis of Algorithms
King Fahd University of Petroleum
Minerals Information Computer Science Department

• NP-Complete Problems

2
Reading Assignment

• M. Alsuwaiyel, Introduction to Algorithms Design
  Techniques and Analysis, World Scientific
  Publishing Co., Inc. 1999.
• Chapter 10 Sections 1 4

3
NP-Complete Problems

• Problems in Computer Science are classified into
• Tractable There exists a polynomial time
  algorithm that solves the problem
• O(nk)
• Intractable Unlikely for a polynomial time
  algorithm solution to exist
• NP-Complete Problems

4
Decision Problems vs. Optimization Problems

• Decision Problem Yes/no answer
• Optimization Problem Maximization or
  minimization of a certain quantity
• When studying NP-Completeness, it is easier to
  deal with decision problems than optimization
  problems

5
Element Uniqueness Problem

• Decision Problem Element Uniqueness
• Input A sequence of integers S
• Question Are there two elements in S that are
  equal?
• Optimization Problem Element Count
• Input A sequence of integers S
• Output An element in S of highest frequency
• What is the algorithm to solve this problem? How
  much does it cost?

6
Coloring a Graph

• Decision Problem Coloring
• Input G(V,E) undirected graph and k??, k gt 0.
• Question Is G k-colorable?
• Optimization Problem Chromatic Number
• Input G(V,E) undirected graph
• Output The chromatic number of G,??(G)
• i.e. the minimum number ??(G) of colors needed to
  color a graph in such a way that no two adjacent
  vertices have the same color.

7
Clique

• Definition A clique of size k in G, for some ve
  integer k, is a complete subgraph of G with k
  vertices.
• Decision Problem
• Input
• Question
• Optimization Problem
• Input
• Output

8
Vertex Cover

• Definition A vertex cover of an undirected graph
  G(V,E) is a subset C?? V such that each edge
  e(x,y) ? E is incident to at least one vertex in
  C, i.e. x ? C or y ? C.
• Decision Problem
• Input
• Question
• Optimization Problem
• Input
• Output

9
Independent Set

• Definition Given an undirected graph G(V,E) a
  subset S?? V is called an independent set if for
  each pair of vertices x, y ? S, (x,y) ? E
• Decision Problem
• Input
• Question
• Optimization Problem
• Input
• Output

10
From Decision To Optimization

• For a given problem, assume we were able to find
  a solution to the decision problem in polynomial
  time. Can we find a solution to the optimization
  problem in polynomial time also?

11
Deterministic Algorithms

• Definition Let A be an algorithm to solve
  problem??. A is called deterministic if, when
  presented with an instance of the problem ?, it
  has only one choice in each step throughout its
  execution.
• If we run A again and again, is there a
  possibility that the output may change?
• What type of algorithms did we have so far?

12
The Class P

• Definition The class of decision problems P
  consists of those whose yes/no solution can be
  obtained using a deterministic algorithm that
  runs in polynomial time of steps, i.e. O(nk),
  where k is a non-negative integer and n is the
  input size.

13
Examples

• Sorting Given n integers, are they sorted in
  non-decreasing order?
• Set Disjointness Given two sets of integers, are
  they disjoint?
• Shortest path
• 2-coloring
• Theorem A graph G is 2-colorable if and only if
  G is bipartite

14
Closure Under Complementation

• A class C of problems is closed under
  complementation if for any problem ? ? C the
  complement of ? is also in C.
• Theorem The class P is closed under
  complementation

15
Non-Deterministic Algorithms

• A non-deterministic algorithm A on input x
  consists of two phases
• Guessing An arbitrary string of characters y
  is generated in polynomial time. It may
• Correspond to a solution
• Not correspond to a solution
• Not be in proper format of a solution
• Differ from one run to another
• Verification A deterministic algorithm verifies
• The generated string of characters y is in
  proper format
• Whether y is a solution
• in polynomial time

16
Non-Deterministic Algorithms (Cont.)

• Definition Let A be a nondeterministic algorithm
  for a problem ?. We say that A accepts an
  instance I of ? if and only if on input I, there
  exists a guess that leads to a yes answer.
• Does it mean that if an algorithm A on a given
  input I leads to an answer of no for a certain
  guess, that it does not accept it?
• What is the running time of a non-deterministic
  algorithm?

17
The Class NP

• Definition The class of decision problems NP
  consists of those decision problems for which
  there exists a nondeterministic algorithm that
  runs in polynomial time

18
Example

• Show that the coloring problem belongs to the
  class of NP problems

19
P and NP Problems

• What is the difference between P problems and NP
  Problems?
• We can decide/solve problems in P using
  deterministic algorithms that run in polynomial
  time
• We can check or verify the solution of NP
  problems in polynomial time using a deterministic
  algorithm
• What is the set relationship between the classes
  P and NP?

20
NP-Complete Problems

• Definition Let ?? and ? be two decision
  problems. We say that ?? reduces to ? in
  polynomial time, denoted by ????poly ?, if there
  exists a deterministic algorithm A that behaves
  as follows When A is presented with an instance
  I of problem ??, it transforms it into an
  instance I of problem ?? in polynomial time such
  that the answer to I is yes if and only if the
  answer to I is yes.

21
NP-Hard and NP-Complete

• Definition A decision problem ?? is said to be
  NP-hard if ???? NP, ????poly ?.
• Definition A decision problem ?? is said to be
  NP-complete if
• ???? NP
• ???? NP, ????poly ?.
• What is the difference between an NP-complete
  problem and an NP-hard problem?

22
Conjunctive Normal Forms

• Definition A clause is the disjunction of
  literals, where a literal is a boolean variable
  or its negation
• E.g., x1 ? x2 ? x3 ? x4
• Definition A boolean formula f is said to be in
  conjunctive normal form (CNF) if it is the
  conjunction of clauses.
• E.g., (x1 ? x2) ? (x1 ? x5) ? (x2 ? x3 ? x4 ?
  x6)
• Definition A boolean formula f is said to be
  satisfiable if there is a truth assignment to its
  variables that makes it true.

23
The Satisfiability Problem

• Input A CNF boolean formula f.
• Question Is f satisfiable?
• Theorem Satisfiability is NP-Complete
• Satisfiability is the first problem to be proven
  as NP-Complete
• The proof includes reducing every problem in NP
  to Satisfiability in polynomial time.

24
Transitivity of ??poly

• Theorem Let ?, ?, and ? be three decision
  problems such that ? ?poly ? and ???poly ?.
  Then ???poly ?.
• Proof
• Corollary If ?, ?? NP such that ? ?poly ? and
  ? ? NP-complete, then ? ? NP-complete
• Proof
• How can we prove that ? ? NP-hard?
• How can we prove that ? ? NP-complete?

25
Proving NP-Completeness
SAT
3-CNF-SAT
Subset-Sum
26
Example NP-Complete Problems

• 3-CNF-SAT
• Input Boolean formula f in CNF, such that each
  clause consists of exactly three literals.
• Question Is f satisfiable.
• Hamiltonian Cycle
• Input G (V,E), undirected graph.
• Does G have a cycle that visits each vertex
  exactly once (Hamiltonian Cycle)?
• Traveling Salesman
• Input A set of n cities with their intercity
  distances and an integer k.
• Question Does there exist a tour of length less
  than or equal to k? A tour is a cycle that visits
  each vertex exactly once.

27
Example 1

• Show that the traveling salesman problem is
  NP-complete, assuming that the Hamiltonian cycle
  problem is NP-complete.

28
Example 2

• Prove that the Problem Clique is NP-Complete.
• Proof
• Clique ? NP
• Clique ? NP-Hard
• SAT ?poly Clique

29
Example 3

• Prove that the problem Vertex Cover is
  NP-Complete
• Proof

30
Example 4

• Prove that the problem Independent Set is
  NP-Complete
• Proof

31
Example NP-Complete Problems (Cont.)

• Subset Sum
• 3-Coloring
• 3D-Matching
• Hamiltonian Path
• Partition
• Knapsack
• Bin Packing
• Set Cover
• Multiprocessor Scheduling
• Longest Path