The Theory of NPCompleteness

1
The Theory of NP-Completeness
2
Cooks Theorem (1971)

• Prof. Cook
• Toronto U.
• Receiving Turing Award (1982)
• Discussing difficult problems worst case lower
  bound seems to be in the order of an exponential
  function
• NP-complete (NPC) Problems

3
Finding lower bound by problem transformation

• Problem A reduces to problem B (A?B)
• iff A can be solved by using any algorithm which
  solves B.
• If A?B, B is more difficult (B is at least as
  hard as A)
• Since ?(A)? ?(B) T(tr1) T(tr2), we have?(B)?
  ?(A) (T(tr1) T(tr2))
• We have ?(B)? ?(A) if T(tr1) T(tr2) ? ?(A)

4
The lower bound of the convex hull problem

• sorting ? convex hull
• A B
• an instance of A (x1, x2,, xn)
• ?transformation
• an instance of B ( x1, x12), ( x2, x22),, (
  xn, xn2)
• assume x1 lt x2 lt lt xn

5
The lower bound of the convex hull problem

• If the convex hull problem can be solved, we can
  also solve the sorting problem, but not vice
  versa.
• We have that the convex hull problem is harder
  than the sorting problem.
• The lower bound of sorting problem is ?(n log n),
  so the lower bound of the convex hull problem is
  also ?(n log n).

6
NP
NPC
P
PNP
?
NP Non-deterministic Polynomial
P Polynomial
NPC Non-deterministic Polynomial Complete
7
Nondeterministic algorithms

• A nondeterministic algorithm is an algorithm
  consisting of two phases guessing and checking.
• Furthermore, it is assumed that a
  nondeterministic algorithm always makes a correct
  guessing.

8
Nondeterministic algorithms

• Machines for running nondeterministic algorithms
  do not exist and they would never exist in
  reality. (They can only be made by allowing
  unbounded parallelism in computation.)
• Nondeterministic algorithms are useful only
  because they will help us define a class of
  problems NP problems

9
NP algorithm

• If the checking stage of a nondeterministic
  algorithm is of polynomial time-complexity, then
  this algorithm is called an NP (nondeterministic
  polynomial) algorithm.

10
NP problem

• If a decision problem can be solved by a NP
  algorithm, this problem is called an NP
  (nondeterministic polynomial) problem.
• NP problems (must be decision problems)

11
Decision problems

• The solution is simply Yes or No.
• Optimization problem harder
• Decision problem easier
• E.g. the traveling salesperson problem
• Optimization version
• Find the shortest tour
• Decision version
• Is there a tour whose total length is less than
  or equal to a constant C ?

12
Decision version of sorting

• Given a1, a2,, an and c, is there a permutation
  of ai?s ( a1?, a2? , ,an? ) such
  thata2?a1?a3?a2? an?an-1?lt C ?

13
Decision vs Original Version

• We consider decision version problem D rather
  than the original problem O
• because we are addressing the lower bound of a
  problem
• and D ? O

14
To express Nondeterministic Algorithm

• Choice(S) arbitrarily chooses one of the
  elements in set S
• Failure an unsuccessful completion
• Success a successful completion

15
Nondeterministic searching Algorithm

• input n elements and a target element x
• output success if x is found among the n
  elements failure, otherwise.
• j ? choice(1 n) / guess
• if A(j) x then success / check
• else failure
• A nondeterministic algorithm terminates
  unsuccessfully iff there exist no set of choices
  leading to a success signal.
• The time required for choice(1 n) is O(1).

16
Relationship Between NP and P

• It is known P?NP.
• However, it is not known whether PNP or whether
  P is a proper subset of NP
• It is believed NP is much larger than P
• We cannot find a polynomial-time algorithm for
  many NP problems.
• But, no NP problem is proved to have exponential
  lower bound. (No NP problem has been proved to be
  not in P.)
• So, does P NP? is still an open question!
• Cook tried to answer the question by proposing
  NPC.

17
NP-complete (NPC)

• A problem A is NP-complete (NPC) if A?NP and
  every NP problem reduces to A.

18
SAT is NP-complete

• Every NP problem can be solved by an NP algorithm
• Every NP algorithm can be transformed in
  polynomial time to an SAT problem
• Such that the SAT problem is satisfiable iff the
  answer for the original NP problem is yes
• That is, every NP problem ? SAT
• SAT is NP-complete

19
Cooks theorem (1971)

• NP P iff SAT ? P
• NP P iff the satisfiability (SAT) problem is a
  P problem
• SAT is NP-complete
• It is the first NP-complete problem
• Every NP problem reduces to SAT

20
Proof of NP-Completeness

• To show that A is NP-complete
• (I) Prove that A is an NP problem
• (II) Prove that ? B ? NPC, B ? A
• ? A ? NPC
• Why ?

Transitive property of polynomial-time reduction
21
0/1 Knapsack problem

• Given M (weight limit) and V, is there is a
  solution with value larger than V?
• This is an NPC problem.

22
Traveling salesperson problem

• Given A set of n planar points and a value L
• Find Is there a closed tour which includes all
  points exactly once such that its total length is
  less than L?
• This is an NPC problem.

23
Partition problem

• Given A set of positive integers S
• Find Is there a partition of S1 and S2 such
  that S1?S2?, S1?S2S, ?i?S1i?i?S2 i
• (partition S into S1 and S2 such that element
  sum of S1 is equal to that of S2)
• e.g. S1, 7, 10, 9, 5, 8, 3, 13
• S11, 10, 9, 8
• S27, 5, 3, 13
• This problem is NP-complete.

24
Art gallery problem Given a constant C, is
there a guard placement such that the number of
guards is less than C and every wall is
monitored?This is an NPC problem.
25
Karp

• R. Karp showed several NPC problems, such as
  3-STA, node (vertex) cover, and Hamiltonian
  cycle, etc.
• Karp received Turing Award in 1985

26
NP-Completeness Proof Reduction
All NP problems
SAT
Clique
3-SAT
Vertex Cover
Chromatic Number
Dominating Set
27
NP-Completeness

• NP-complete problems the hardest problems in
  NP
• Interesting property
• If any one NP-complete problem can be solved in
  polynomial time, then every problem in NP can
  also be solved in polynomial time (i.e., PNP)
• Many believe P?NP

28
Importance of NP-Completeness

• NP-complete problems considered intractable
• Important for algorithm designers engineers
• Suppose you have a problem to solve
• Your colleagues have spent a lot of time to solve
  it exactly but in vain
• See whether you can prove that it is NP-complete
• If yes, then spend your time developing an
  approximation (heuristic) algorithm
• Many natural problems can be NP-complete

29
Some concepts

• Up to now, none of the NPC problems can be solved
  by a deterministic polynomial time algorithm in
  the worst case.
• It does not seem to have any polynomial time
  algorithm to solve the NPC problems.
• The lower bound of any NPC problem seems to be in
  the order of an exponential function.
• The theory of NP-completeness always considers
  the worst case.

30
Caution !

• If a problem is NP-complete, its special cases
  may or may not be of exponential time-complexity.
• We consider worst case lower bound in
  NP-complete.

31
Some concepts

• Not all NP problems are difficult. (e.g. the MST
  problem is an NP problem.) (But NPC problem is
  difficult.)
• If A, B ? NPC, then A ? B and B ? A.
• Theory of NP-completeness
• If any NPC problem can be solved in polynomial
  time, then all NP problems can be solved in
  polynomial time. (NP P)

32
Undecidable Problems

• They cannot be solved by guessing and checking.
• They are even more difficult than NP problems.
• E.G. Halting problem Given an arbitrary program
  with an arbitrary input data, will the program
  terminate or not?
• It is not NP
• It is NP-hard (SAT ?Halting problem)

33

• NP the class of decision problem which can be
  solved by a non-deterministic polynomial
  algorithm.
• P the class of problems which can be solved by a
  deterministic polynomial algorithm.
• NP-hard the class of problems to which every NP
  problem reduces. (It is at least as hard as the
  hardest problems in NP.)
• NP-complete the class of problems which are
  NP-hard and belong to NP.

34
The satisfiability (SAT) problem

• Def Given a Boolean formula, determine whether
  this formula is satisfiable or not.
•  
• A literal xi or -xi
• A clause x1 v x2 v -x3 ? ci
• A formula conjunctive normal form
• C1 c2 cm

35
The satisfiability (SAT) problem

• The satisfiability problem
• The logical formula
• x1 v x2 v x3
• - x1
• - x2
• the assignment
• x1 ? F , x2 ? F , x3 ? T
• will make the above formula true
• (-x1, -x2 , x3) represents
• x1 ? F , x2 ? F , x3 ? T

36
The satisfiability problem

• If there is at least one assignment which
  satisfies a formula, then we say that this
  formula is satisfiable otherwise, it is
  unsatisfiable.
• An unsatisfiable formula
• x1 v x2
• x1 v -x2
• -x1 v x2
• -x1 v -x2

37
The satisfiability problem

• Resolution principle
• c1 -x1 v -x2 v x3
• c2 x1 v x4
• ? c3 -x2 v x3 v x4 (resolvent)
• If no new clauses can be deduced
• ? satisfiable
• -x1 v -x2 v x3 (1)
• x1
  (2)
• x2
  (3)
• (1) (2) -x2 v x3 (4)
• (4) (3) x3 (5)
• (1) (3) -x1 v x3 (6)

x1 cannot satisfyc1 and c2 at thesame time, so
itis deleted..
38
The satisfiability problem

• If an empty clause is deduced
• ? unsatisfiable
• - x1 v -x2 v x3 (1)
• x1 v -x2 (2)
• x2 (3)
• - x3 (4)
•   ? deduce
•   (1) (2) -x2 v x3 (5)
• (4) (5) -x2 (6)
• (6) (3) ? (7)

39
Nondeterministic SAT

• Guessing for i 1 to n do
• xi ? choice( true, false )
• if E(x1, x2, ,xn) is true
• Checking then success
• else failure

40
Transforming the NP searching algorithm to the
SAT problem

• Does there exist a number in x(1), x(2), ,
  x(n) , which is equal to 7?
• Assume n 2

41
Transforming searching to SAT

• i1 v i2
• i1 ? i?2
• i2 ? i?1
• x(1)7 i1 ? SUCCESS
• x(2)7 i2 ? SUCCESS
• x(1)?7 i1 ? FAILURE
• x(2)?7 i2 ? FAILURE
• FAILURE ? -SUCCESS
• SUCCESS (Guarantees a successful termination)
• x(1)7 (Input Data)
• x(2)?7

42
Transforming searching to SAT

• CNF (conjunctive normal form)
• i1 v i2 (1)
• i?1 v i?2 (2)
• x(1)?7 v i?1 v SUCCESS (3)
• x(2)?7 v i?2 v SUCCESS (4)
• x(1)7 v i?1 v FAILURE (5)
• x(2)7 v i?2 v FAILURE (6)
• -FAILURE v -SUCCESS (7)
• SUCCESS (8)
• x(1)7 (9)
• x(2)?7 (10)

43
Transforming searching to SAT

• Satisfiable at the following assignment
• i1 satisfying (1)
• i?2 satisfying (2), (4) and (6)
• SUCCESS satisfying (3), (4) and (8)
• -FAILURE satisfying (7)
• x(1)7 satisfying (5) and (9)
• x(2)?7 satisfying (4) and (10)

44
Searching in CNF with inputs

• Searching for 7, but x(1)?7, x(2)?7
• CNF

45
Searching in CNF with inputs

• Apply resolution principle

46
Searching in CNF with inputs

47

• QA