Time Complexity

1
Time Complexity
2

• Measuring Complexity
• Worst-case analysis
• Average-case analysis
• Define M a det. Turing machine that halts on all
  input
• The running time or time complexity of M is the
  function
• f N?N, where f(n) is the maximum number of
  steps that M uses on any input of length n
• If f(n) is the running time of M, we say M runs
  in time f(n) and that M is an f(n) time Turing
  machine.

3

• Big-O and small-o Notation
• Def
• Def
• Def

4

• Complexity relationships among models

5

• The class P
• P is the class of languages that are decidable
  in polynomial time on a deterministic single-tape
  TM
• In other words,

6

7

• The Class NP
• Hamiltonian path in a directed graph G is a
  directed path that goes through each node once
• Note
• It is not hard to find an exponential time
  algorithm for the HAMPATH problem
• The HAMPATH problem can be verified in polynomial
  time

8

• Not all problem can be verified in poly. time.
• Def A verifier for a language A is an algorithm
  V, where
• A V accepts lt ,cgt for some string c
• A polynomial time verifier runs in polynomial
  time in the length of
• A language A is polynomially verifiable if it has
  a polynomial time verifier.
• A verifier uses additional information c to
  verify that a string is member of A
• C certificate or proof of membership in A

12653 123 x 111
verifiable in poly. time
9

• Def NP is the class of languages that have
  polynomial time verifiers
• NP Non-deterministic Polynomial time
• Def
• Thm A language is in NP iff it is decided by
  some non-deterministic polynomial time Turing
  machine
• Pf

10
(No Transcript)
11

• Cor
• Examples of problem in NP
• Def clique is a graph, where every two nodes are
  connected by an edge
• A k-clique is a clique that contains k nodes

A graph of 5-clique
12

• Def
• Thm CLIQUE is in NP
• pf

13

• Def
• Thm SUBSET-SUM is in NP
• Pf

14

• P the class of languages where membership can
  be decided quickly
• NP the class of languages where membership can
  be verified quickly
• BIG Question!! P NP

?
15

• NP-completeness
• The satisfiability problem is to test whether a
  Boolean formula is satisfiable

16

• Thm Cook-Levin Theorem
• Def

17

• Def

A
B
18

• Thm
• Pf

19

• Def
• A language B is NP-complete if it satisfies two
  condition
• 1. B is in NP
• 2. Every A in NP is polynomial time reducible
  to B
• (with only 2, it is called NP-hard)
• Thm
• Pf

20

• Thm
• Pf

21

• Def

22

• Thm 3SAT is polynomial time reducible to CLIQUE
• Pf

23

• Cook-Levin Theorem
• SAT is NP-complete.

Proof
1gt SAT NP
A NTM can guess an assignment to a given formula
and accept if the assignment satisfies .
2gt For any A NP and show that A is polynomial
time reducible to SAT.
Let N be a non-deterministic TM that decides A in
nk time for some constant k.
A tableau for N on w is an nkXnk table whose rows
are the configurations of a branch of the
Computation of N on input w.

q0
w1

start configuration


2nd configuration
cell


window


nkth configuration
A tableau is accepting if any row of the tableau
is accepting configuration.
24
Every accepting tableau for N on w corresponds to
a computation branch of N on w.
Determine whether N accepts w is equivalent to
determine whether an accepting tableau for N
on w exists.
fpolynomial time reduction from A to SAT.
On input w, the reduction produces a formula .
Let Q and be the state set and the tape
alphabet of N.
Let For
and each we have a variable
such variables.

• If , it means celli,j contains an
  s.
• Design so that a satisfying assignment to
  the variable does correspond to an accepting
  tableau
• for N on w.

(1)
s
i
j
25
Ensure that the first row of the table is the
starting configuration of N on w.
(3) Guarantees that an accepting
configuration occurs in the tableau.
(4) This part is more complicated !
guarantee that each row of the table corresponds
to a configuration that legally follows the
preceding Rows configuration according to Ns
rules.
a
a

b
q1
b
a
q2
b
c
a

a
a

q2
b
b
a
a
(c)
(b)
(a)
q1
q1
a
a
a
b
b
a
a
q2
a
a
legal windows
(c)
(b)
(d)
b
b
b
a
b
a
b

a
b
c
b
q2
a
b
a

b
26
Examples of illegal windows
(c)
(b)
(a)
q1
q1
a
a
a
b
a
b
a
q2
q1
b
q2
a
a
a
a
a
ClaimIf the top row of the table is the start
configuration and every window in the table is
legal, each row of the table is a
configuration that legally follows the preceding
one.
a
a


b
q1
b
a
q2
a


b
a
c
a
is a legal window
a2
a1
a3
j
a4
a5
a6
j1
i-1
i1
i
Size of
Total number of variables
Thus the reduction is poly.
27

• Cor
• 3SAT is NP-complete.
• Cor
• CLIQUE is NP-complete.

Proof
3SAT is in NP.
How about
28

• Vertex cover of G
• If G is an undirected graph, a vertex cover of G
  is a subset of the nodes where
• every edge of G touches one of those nodes.

2,3 is a vertex cover.
1
4
1,3 is not a vertex cover.
3
VERTEX-COVERltG,kgtG is an undirected graph that
has a k-node vertex cover.
29

• Thm
• VERTEX-COVER is NP-complete.

Proof
VERTEX-COVER is in NP.
Need to show that is satisfiable iff G has a
vertex cover with k nodes.
Let have m variables and l clauses. Let
km2l.

• One true variable from each variable gadget.
• two node from each clause gadget.

(2) Each of the 3 edges connecting the
variable gadgets with each clause gadget is
covered.
30

• SUBSET-SUM

31

• Thm
• SUBSET-SUM is NP-complete.

Proof
SUBSET-SUM is in NP.
Let be a boolean formula with variables
x1,,xl and clauses c1,,ck.
1 2 3 4 .. l
C1 C2 .. Ck
1 0 0 0 .. 0
1 0 .. 0
1 0 0 0 .. 0
0 0 .. 0
1 0 .. 0
.. 1 .
1 0 0 .. 0
0 1 .. 0
1 0 0 .. 0
1 0 .. 0
1 0 .. 0
.. 0 .
1 0 .. 0
1 1 .. 0
1 0 .. 0
0 0 .. 1
clause cj contains xi
1
0 0 .. 0
1
0 0 .. 0
1 0 .. 0
1 0 .. 0
1 .. 0
1 .. 0
1
1
1 1 1 1 .. 1
3 3 .. 3
32

Suppose is satisfiable.
(1)
Construct a subset S as follows.
If xi is assigned TRUE, select yi
else select zi.
I.e. for each I, we select either yi or zi.
Last k digits add up between 1 and 3.
Select enough of g and h numbers to make each of
the last k digits up to 3.
(2)
Suppose a subset of S sums to t. We construct a
satisfying assignment to .
If the subset contains yi , we assigned xi TRUE
else assign xi FALSE.
This assignment satisfied ! Why??
The table has size
33

• MAX-SAT
• Given a boolean formula ?, and an integer k, is
  there a truth
• assignment that satisfies at least k clauses?
• Thm MAX-SAT is NP-complete
• Pf
• SAT ?p MAX-SAT
• ? c1 ? c2 ? ? cm
• k m ?
• Thm DOUBLE-SAT
• Given a boolean formula ?, are there at least 2
  truth
• assignment for ??
• DOUBLE-SAT is NP-complete

34

• Def
• Hamiltonian path in a directed graph G is a
  directed path that goes through each node once.
• HAMPATH
• Given a directed graph and its 2 vertices s and
  t, is there
• a Hamiltonian path from s to t?
• Thm HAMPATH is NP-complete
• Pf (longlater)
• 3SAT ?p HAMPATH ?
• UHAMPATH
• Given a undirected graph and its 2 vertices s and
  t, is there
• a Hamiltonian path from s to t?

35

• Proof (HAMPATH is NPC)
• The following is a 3cnf-formula with k clauses
• ?(a1?b1?c1) ? (a2?b2?c2) ? ? (ak?bk?ck)
• Each a,b,c is a literal xi or , and x1,,xl
  are l variables of ?.
• xi ci

36
s
c1

• Proof conti.

c2
cl
t
37

• Proof conti.

38

• Proof conti.
• If there is a satisfying assignment, select
  exactly one of the literals in a clause and
  assign it TRUE.
• If there is a truth assignment, then there exists
  an Hamiltonian path from s to t.

39

• Proof conti.
• If the Hamiltonian path is normal, that is, it
  goes through the diamonds in order from the top
  one to the bottom one, we can easily get the
  satisfying assignment.
• Because each clause node appears on the path, by
  observing the diamond at which the detour to it
  is taken, we may determine which of the literals
  in the corresponding clause is TRUE.
• If the path zigzag, we assign corresponding
  variable TRUE
• If the path zagzig, we assign FALSE.

40

• Proof conti.
• We show that a Hamiltonian path must be normal.

c
41

• Thm UHAMPATH is NP-complete
• Pf
• HAMPATH ?p UHAMPATH
• directed G ? undirected G
• u ? V(G)
• vertices of G u ? uin, umid, uout
• s ? sout t ? tin
• edges of G u ? v ? E(G)
• ? uin umid uout vin vmid vout
• if s ? u1 ? u2 ? ? uk ? t is a Hamiltonian
  path in G,
• then sout u1in u1mid u1out u2in u2mid
  u2out
• ukin ukmid ukout tin is an
  undirected Hamiltonian
• path in G

42

• Undirected Hamiltonian Cycle
• Thm
• Undirected Hamiltonian Cycle is NP-comlplete
• Pf UHAMPATH ?p UHAMCYCLE
• Traveling Salesman Problem (TSP)
• Given an integer n?2, an n?n distance matrix of
  some cities,
• and an integer B?0, is there a tour that visits
  every city
• exactly once and returns to the starting city by
  traveling
• within distance B?
• Thm TSP is NP-complete
• Pf UHAMCYCLE ?p TSP

43

• Longest Cycle
• Given a graph and integer k, is there a cycle,
  with no
• repeated nodes, of length at least k?
• Thm
• LONGEST CYCLE is NP-complete
• SUBGRAPH ISOMORPHISM
• Given 2 undirected graphs G and H, is G a
  subgraph of H?
• Thm
• SUBGRAPH ISOMORPHISM is NP-complete

44

• Coping with NP-completeness
• Approximation algorithms
• Let x be an instance of an optimization problem
• Opt(x) the optimum solution of x
• A a poly. time algorithm for x
• ? positive real number
• If A satisfies
• For all x, then we say A is an ?-approximation
  algorithm

45

• Eg.
• The following is a 1-approximation algorithm for
  the vertex cover problem.
• G C2,3
• Algorithm 1. C ?
• 2. while there is an edge u,v in G
• add u, v to C and delete them from G
• Let be the optimum vertex cover
• c , , ,,

2
4
3
1
46

• Polynomial Time Approximation Scheme(PTAS)
• We say that an approximation scheme is PTAS, if
  for any fixed
• ? gt0, the scheme runs in time polynomial in the
  size n of its
• input instance.
• Ratio bound
• Inapproximable problem
• If there is no ?-approximation algorithm for them
  with
• however large ?, unless PNP

47

• Thm TSP is inapproximable unless PNP
• Pf
• Let G be a graph with n nodes.
• UHAMCYCLE ?p TSP
• If G has a Hamiltonian cycle, then the optimum
  cost of a tour is n otherwise if G has no
  Hamiltonian cycle, then the optimum cost of a
  tour gt n(1?)

1
?
23?
1
48

• If TSP had an ?-approximation algorithm A, then
  we would be able to tell whether G has a
  Hamiltonian cycle.
• Run A for TSP
• igt if the returned cost ? n(1?)1 No cycle
• iigt if the returned cost ? n(1?) has a cycle
• Unless PNP, TSP is inapproximable.

49

• Backtracking and BranchBound
• AS0
• while A is not empty do
• choose a subproblem S and delete it from A
• choose a way of branching out of S , say to
  subproblems
• S1, S2,, Sr
• for each subproblem Si in this list do
• if test(Si) returns solution found
• else if test(Si) returns ? then add Si to A
• Return no solution

50

• Eg.

x T
x F
y T
y F
z T
z F
51

• BranchBound algorithm
• AS0 , best_so_far?
• while A is not empty do
• choose a subproblem S and delete it from A
• choose a way of branching out of S , say to
  subproblems
• S1, S2,, Sr
• for each subproblem Si in this list do
• if Si1 then update best_so_far
• else if lowerbound(Si) lt best_so_far then add
  Si to A
• Return best_so_far

52

• Local improvement algorithm
• S initial solution
• while there is a solution S such that
• S is a neighbor of S and cost(S) lt cost(S)
• do SS
• Return S

53

• Simulated annealing
• S initial solution TT0
• repeat
• generate a random solution S such that
• S is a neighbor of S and let ?
  cost(S)?cost(S)
• if ? ? 0 then SS
• else SS with probability e-?/T
• update (T)
• until T0
• Return the best solution seen!
• T temperature