Time Complexity

1
Time Complexity

• Jianhua Yang
• Department of Math and Computer Science
• Bennett College

2
Goals

• The Class P
• The Class NP

3
Whats the problem

• Solvable in principle (decidable)
• Solvable in practice
• time,
• memory
• Other resources

4
Objective

• The objective of this chapter is to present the
  basics of time complexity theory.

5
7.1 The class P

• Polynomial Time
• P problem
• Some examples in P

6
1. Polynomial Time

• Polynomial differences in running time are
  considered to be small
• Exponential differences are considered to be
  large.

7
Example

• Think about n3 and 2n when n100
• n3 would be 1 billion, and 2n a number which is
  not manageable.

8
Significance

• If an algorithm can run with
• Polynomial time (the algorithm is useful)
• Exponential time (the algorithm is useless)

9
Exponential time algorithms

• They typically arise when we solve problems by
  using brute-force-search to search through a
  space of solutions.
• Example (binary tree search problem)

10
Polynomial time algorithms

• It can be simulated with only a polynomial
  increase in running time.
• Such as n42n32n5 O(n4)

11
2. P class problems

• P is the class of languages that are decidable in
  polynomial time on a deterministic single-tape
  Turing machine.
• That is

12
Class P is important

• 1. P is invariant for all models of computation
  that are polynomially equivalent to the
  deterministic single-tape Turing machine
• 2. P roughly corresponds to the class of problems
  that are realistically solvable on a computer.

13
3. Examples of Problems in P

• PATHltG, s, tgt G is a directed graph that has a
  directed path from s to t
• RELPRIMEltx, ygt x and y are relatively prime
• Every context-free language

14
PATH is in Class P

• Proof
• We can design a polynomial time algorithm M for
  PATH as the follows
• M on input ltG, s, tgt where G is a directed graph
  with nodes s and t.
• 1. Place a mark on node s.
• 2. Repeat the following until no additional nodes
  are marked.
• 3. Scan all the edges of G. If an edge (a, b) is
  found going from a marked node a to an unmarked
  node b, mark node b.
• 4. If t is marked, accept. Otherwise, reject.

15
Running time

• The Running time is 11m if there are m nodes.

16
RELPRIME is in Class P

• Proof The Euclidean algorithm, E, is as the
  follows.
• Eon input ltx, ygt, where x and y are natural
  numbers in binary
• 1. Repeat until y0
• Assign x? x mod y
• Exchange x and y
• 2. Output x

17
Proof. (cont.)

• Algorithm R solves RELPRIME, using E as a
  subroutine.
• R On input ltx, ygt, where x and y are natural
  numbers in binary
• 1. Run E on ltx, ygt
• 2. If the result is 1, accept. Otherwise, reject.

18
Running time

• The running time is

19
Context-free language

• Every context-free language is a member of P

20
7.2 The Class NP

• What is NP problem?
• Some examples about NP problem.
• Comparisons between NP and P problems.

21
1. NP problem

• NP is the class of languages that have polynomial
  time verifiers.

22
Polynomial time verifier

• A polynomial time verifier for a language A is a
  algorithm V, where
• Aw V accepts ltw, cgt for a string c. V can
  run in term of length of w with polynomial time.

23
Theorem

• A language is in NP if and only if it is decided
  by some nondeterministic Turing machine with
  polynomial time.

24
Proof

• First, we prove that if a language A is in NP,
  then A is decided by a polynomial time NTM N.
• Suppose V is the polynomial time verifier for A.
• Non input w of length n
• 1. Nondeterministically select string c of length
  nk.
• 2. Run V on input ltw, cgt.
• 3. If V accepts, accept otherwise, reject.

25
Proof (cont.)

• Second, if A is decided by a polynomial time NTM
  N, we can construct a polynomial time verifier V
  as follows
• (skip this part)

26
2. Examples

• HAMPATH
• CLIQUE
• SUBSET-SUM

27
HAMPATH problem

• Definition of Hamiltonian path in a directed
  graph G, it is a directed path that goes through
  each node exactly once.
• HAMPATHltG, s, tgt G is a directed graph with a
  Hamiltonian path from s to t

28
Complexity

• HAMPATH is in NP class

29
Proof (skip this one)

• N1 on input ltG, s, tgt, where G is a directed
  graph with nodes s and t
• 1. Write a list of m numbers, p1, p2, , pm,
  where m is the number of nodes in G. Each number
  in the list is nondeterministically selected to
  be between 1 and m.
• 2. Check for repetitions in the list. If any are
  found, reject
• 3. Check whether sp1, and tpm. If either fail,
  reject.
• 4. For each i between 1 and m-1, check whether
  (pi, pi1) is an edge of G. If any are not,
  reject. Otherwise, all tests have been passed, so
  accept.

30
CLIQUE problem

• A clique in an undirected graph is a subgraph,
  wherein every two nodes are connected by an
  edge.
• A k-clique is a clique that contains k nodes.

31
Example about CLIQUE
32
CLIQUE problem

• CLIQUEltG, kgt G is an undirected graph with a
  k-clique
• CLIQUE is in NP

33
Proof

• Design a NTM N as the following
• N on input ltG, kgt, where G is a graph
• 1. Nondeterministically select a subset c of k
  nodes of G.
• 2. Test whether G contains all edges connecting
  nodes c.
• 3. If yes, accept otherwise, reject.

34
SUBSET-SUM problem
35
Complexity

• SUBSET-SUM is in NP

36
Proof

• We can also prove this theorem by giving a
  nondeterministic polynomial time Turing machine
  for SUBSET-SUM as follows
• N on input ltS, tgt
• 1. Nondeterministically select a subset c of the
  numbers in S.
• 2. Test whether c is a collection of numbers that
  sum to t.
• 3. If the test passes, accept otherwise, reject.

37
3. P versus NP

• 1. P is the class of languages where membership
  can be decided quickly
• 2. NP is the class of languages where membership
  can be verified quickly.

38
Open question

• If NP equals to P is not solved so far.

PNP
?
39
7.3 NP-Completeness

• Stephen Cook and Leonid Levin found
• The complexity of certain problems in NP is
  related to that of the entire class.
• Further more, if a polynomial time algorithm
  exists for any of these problems, all the
  problems in NP would be polynomial time solvable.
• These problems are called NP-complete.

40
Significance of NP-complete

• Theoretically,
• A researcher trying to show that P is unequal to
  NP may focus on NP-complete problem
• Furthermore, a researcher attempting to prove
  that P equals NP only needs to find a polynomial
  time algorithm for an NP-complete problem to
  achieve this goal.

41
Significance of NP-complete

• Practically,
• The phenomenon of NP-completeness may prevent
  wasting time searching for a nonexistent
  polynomial time algorithm to solve a particular
  problem.
• Proving that a problem is NP-complete is strong
  evidence of its nonpolynomiality.

42
1. Satisfiability problem

• Boolean formula
• It is an expression involving Boolean variables
  and operations, such as
• Satisfiable
• A Boolean formula is satisfiable if some
  assignment of 0s and 1s to the variables makes
  the formula evaluate to 1.

43
Satisfiability problem

• This problem is to test whether a Boolean formula
  is satisfiable.
• SATltFgt F is a satisfiable Boolean formula

44
2. Cook-Levin Theorem

• Cook-Levin Theorem SAT? P iff PNP
• This theorem links the complexity of the SAT
  problem to the complexities of all problems in NP

45
3. Polynomial time reducibility

• When problem A is efficiently reducible problem
  B, an efficient solution to B can be used to
  solve A efficiently.

46
Definition of Polynomial time computable function

• A function f??? is a polynomial time
  computable function if a polynomial time Turing
  machine M exists that halts with just f(w) on its
  tape, when started on any input w.

47
Definition of Polynomial time reducible

• A language A is polynomial time reducible to
  language B, written Ap B, if a polynomial time
  computable function f??? exists, where for
  every w,
• w?A ltgt f(w)?B

48
Polynomial time reducible Theorem

• If Ap B, and B?P, then A?P

49
Proof of this Theorem

• Let M be the polynomial time algorithm deciding B
  and f be the polynomial time reduction from A to
  B.
• We design a polynomial time algorithm N deciding
  A as follows
• On input w
• 1. compute f(w)
• Run M on input f(w) and output whatever M outputs.

50
Polynomial time analysis

• Computing f(w) ---Polynomial time
• M decides f(w) --- Polynomial time

51
4. 3SAT problem

• Cnf-formula conjunctive normal form comprises
  several clauses connected with ?s, such as
• 3cnf-formula is the one that each clause has
  three literals.

3SATltFgt F is a satisfiable 3cnf-formula
52
Theorem

• 3SAT is polynomial time reducible to CLIQUE.

53
5. Definition of NP-Completeness

• A language B is NP-complete if it satisfies two
  conditions
• 1. B is in NP, and
• 2. every A in NP is polynomial time reducible to
  B.

54
Theorem about P and NP

• If B is NP-complete, and B?P, then PNP
• Proof? (obvious)

55
Theorem NP-complete

• If B is NP-complete and Bp C, C is in NP, then C
  is NP-complete.
• Proof?

56
The Cook-Levin Theorem

• SAT is NP-complete
• Proof
• 1. SAT is in NP
• 2. Every language in NP is polynomial time
  reducible to SAT.

57
Corollary

• 3SAT is NP-complete

58
Summary

• NP class
• Some examples of NP class
• Comparison between P and NP
• NP-complete