Time Complexity

1
Time Complexity
2

• Measuring Complexity
• Worst-case analysis
• Average-case analysis
• Define M a det. Turing machine that halts on all
  input
• The running time or time complexity of M is the
  function
• f N?N, where f(n) is the maximum number of
  steps that M uses on any input of length n
• If f(n) is the running time of M, we say M runs
  in time f(n) and that M is an f(n) time Turing
  machine.

3

• Big-O and small-o Notation
• Def
• Def
• Def

4

• Complexity relationships among models

5

• The class P
• P is the class of languages that are decidable
  in polynomial time on a deterministic single-tape
  TM
• In other words,

6

7

• The Class NP
• Hamiltonian path in a directed graph G is a
  directed path that goes through each node once
• Note
• It is not hard to find an exponential time
  algorithm for the HAMPATH problem
• The HAMPATH problem can be verified in polynomial
  time

8

• Not all problem can be verified in poly. time.
• Def A verifier for a language A is an algorithm
  V, where
• A V accepts lt ,cgt for some string c
• A polynomial time verifier runs in polynomial
  time in the length of
• A language A is polynomially verifiable if it has
  a polynomial time verifier.
• A verifier uses additional information c to
  verify that a string is member of A
• C certificate or proof of membership in A

12653 123 x 111
verifiable in poly. time
9

• Def NP is the class of languages that have
  polynomial time verifiers
• NP Non-deterministic Polynomial time
• Def
• Thm A language is in NP iff it is decided by
  some non-deterministic polynomial time Turing
  machine
• Pf

10
(No Transcript)
11

• Cor
• Examples of problem in NP
• Def clique is a graph, where every two nodes are
  connected by an edge
• A k-clique is a clique that contains k nodes

A graph of 5-clique
12

• Def
• Thm CLIQUE is in NP
• pf

13

• Def
• Thm SUBSET-SUM is in NP
• Pf

14

• P the class of languages where membership can
  be decided quickly
• NP the class of languages where membership can
  be verified quickly
• BIG Question!! P NP

?
15

• NP-completeness
• The satisfiability problem is to test whether a
  Boolean formula is satisfiable

16

• Thm Cook-Levin Theorem
• Def

17

• Def

A
B
18

• Thm
• Pf

19

• Def
• A language B is NP-complete if it satisfies two
  condition
• 1. B is in NP
• 2. Every A in NP is polynomial time reducible
  to B
• (with only 2, it is called NP-hard)
• Thm
• Pf

20

• Thm
• Pf

21

• Def

22

• Thm 3SAT is polynomial time reducible to CLIQUE
• Pf

23

• Cook-Levin Theorem
• SAT is NP-complete.

Proof
1gt SAT NP
A NTM can guess an assignment to a given formula
and accept if the assignment satisfies .
2gt For any A NP and show that A is polynomial
time reducible to SAT.
Let N be a non-deterministic TM that decides A in
nk time for some constant k.
A tableau for N on w is an nkXnk table whose rows
are the configurations of a branch of the
Computation of N on input w.

q0
w1

start configuration


2nd configuration
cell


window


nkth configuration
A tableau is accepting if any row of the tableau
is accepting configuration.
24
Every accepting tableau for N on w corresponds to
a computation branch of N on w.
Determine whether N accepts w is equivalent to
determine whether an accepting tableau for N
on w exists.
fpolynomial time reduction from A to SAT.
On input w, the reduction produces a formula .
Let Q and be the state set and the tape
alphabet of N.
Let For
and each we have a variable
such variables.

• If , it means celli,j contains an
  s.
• Design so that a satisfying assignment to
  the variable does correspond to an accepting
  tableau
• for N on w.

(1)
s
i
j
25
Ensure that the first row of the table is the
starting configuration of N on w.
(3) Guarantees that an accepting
configuration occurs in the tableau.
(4) This part is more complicated !
guarantee that each row of the table corresponds
to a configuration that legally follows the
preceding Rows configuration according to Ns
rules.
a
a

b
q1
b
a
q2
b
c
a

a
a

q2
b
b
a
a
(c)
(b)
(a)
q1
q1
a
a
a
b
b
a
a
q2
a
a
legal windows
(c)
(b)
(d)
b
b
b
a
b
a
b

a
b
c
b
q2
a
b
a

b
26
Examples of illegal windows
(c)
(b)
(a)
q1
q1
a
a
a
b
a
b
a
q2
q1
b
q2
a
a
a
a
a
ClaimIf the top row of the table is the start
configuration and every window in the table is
legal, each row of the table is a
configuration that legally follows the preceding
one.
a
a


b
q1
b
a
q2
a


b
a
c
a
is a legal window
a2
a1
a3
j
a4
a5
a6
j1
i-1
i1
i
Size of
Total number of variables
Thus the reduction is poly.
27

• Cor
• 3SAT is NP-complete.
• Cor
• CLIQUE is NP-complete.

Proof
3SAT is in NP.
How about
28

• Vertex cover of G
• If G is an undirected graph, a vertex cover of G
  is a subset of the nodes where
• every edge of G touches one of those nodes.

2,3 is a vertex cover.
1
4
1,3 is not a vertex cover.
3
VERTEX-COVERltG,kgtG is an undirected graph that
has a k-node vertex cover.
29

• Thm
• VERTEX-COVER is NP-complete.

Proof
VERTEX-COVER is in NP.
Need to show that is satisfiable iff G has a
vertex cover with k nodes.
Let have m variables and l clauses. Let
km2l.

• One true variable from each variable gadget.
• two node from each clause gadget.

(2) Each of the 3 edges connecting the
variable gadgets with each clause gadget is
covered.
30

• SUBSET-SUM

31

• Thm
• SUBSET-SUM is NP-complete.

Proof
SUBSET-SUM is in NP.
Let be a boolean formula with variables
x1,,xl and clauses c1,,ck.
1 2 3 4 .. l
C1 C2 .. Ck
1 0 0 0 .. 0
1 0 .. 0
1 0 0 0 .. 0
0 0 .. 0
1 0 .. 0
.. 1 .
1 0 0 .. 0
0 1 .. 0
1 0 0 .. 0
1 0 .. 0
1 0 .. 0
.. 0 .
1 0 .. 0
1 1 .. 0
1 0 .. 0
0 0 .. 1
clause cj contains xi
1
0 0 .. 0
1
0 0 .. 0
1 0 .. 0
1 0 .. 0
1 .. 0
1 .. 0
1
1
1 1 1 1 .. 1
3 3 .. 3
32

Suppose is satisfiable.
(1)
Construct a subset S as follows.
If xi is assigned TRUE, select yi
else select zi.
I.e. for each I, we select either yi or zi.
Last k digits add up between 1 and 3.
Select enough of g and h numbers to make each of
the last k digits up to 3.
(2)
Suppose a subset of S sums to t. We construct a
satisfying assignment to .
If the subset contains yi , we assigned xi TRUE
else assign xi FALSE.
This assignment satisfied ! Why??
The table has size
33

• MAX-SAT
• Given a boolean formula ?, and an integer k, is
  there a truth
• assignment that satisfies at least k clauses?
• Thm MAX-SAT is NP-complete
• Pf
• SAT ?p MAX-SAT
• ? c1 ? c2 ? ? cm
• k m ?
• Thm DOUBLE-SAT
• Given a boolean formula ?, are there at least 2
  truth
• assignment for ??
• DOUBLE-SAT is NP-complete

34

• Def
• Hamiltonian path in a directed graph G is a
  directed path that goes through each node once.
• HAMPATH
• Given a directed graph and its 2 vertices s and
  t, is there
• a Hamiltonian path from s to t?
• Thm HAMPATH is NP-complete
• Pf (long)
• 3SAT ?p HAMPATH ?
• UHAMPATH
• Given a undirected graph and its 2 vertices s and
  t, is there
• a Hamiltonian path from s to t?

35

• Thm UHAMPATH is NP-complete
• Pf
• HAMPATH ?p UHAMPATH
• directed G ? undirected G
• u ? V(G)
• vertices of G u ? uin, umid, uout
• s ? sout t ? tin
• edges of G u ? v ? E(G)
• ? uin umid uout vin vmid vout
• if s ? u1 ? u2 ? ? uk ? t is a Hamiltonian
  path in G,
• then sout u1in u1mid u1out u2in u2mid
  u2out
• ukin ukmid ukout tin is an
  undirected Hamiltonian
• path in G

36

• Undirected Hamiltonian Cycle
• Thm
• Undirected Hamiltonian Cycle is NP-comlplete
• Pf UHAMPATH ?p UHAMCYCLE
• Traveling Salesman Problem (TSP)
• Given an integer n?2, an n?n distance matrix of
  some cities,
• and an integer B?0, is there a tour that visits
  every city
• exactly once and returns to the starting city by
  traveling
• within distance B?
• Thm TSP is NP-complete
• Pf UHAMCYCLE ?p TSP

37

• Longest Cycle
• Given a graph and integer k, is there a cycle,
  with no
• repeated nodes, of length at least k?
• Thm
• LONGEST CYCLE is NP-complete
• SUBGRAPH ISOMORPHISM
• Given 2 undirected graphs G and H, is G a
  subgraph of H?
• Thm
• SUBGRAPH ISOMORPHISM is NP-complete

38

• Coping with NP-completeness
• Approximation algorithms
• Let x be an instance of an optimization problem
• Opt(x) the optimum solution of x
• A a poly. time algorithm for x
• ? positive real number
• If A satisfies
• For all x, then we say A is an ?-approximation
  algorithm

39

• Eg.
• The following is a 1-approximation algorithm for
  the vertex cover problem.
• G C2,3
• Algorithm 1. C ?
• 2. while there is an edge u,v in G
• add u, v to C and delete them from G
• Let be the optimum vertex cover
• c , , ,,

2
4
3
1
40

• Polynomial Time Approximation Scheme(PTAS)
• We say that an approximation scheme is PTAS, if
  for any fixed
• ? gt0, the scheme runs in time polynomial in the
  size n of its
• input instance.
• Ratio bound
• Inapproximable problem
• If there is no ?-approximation algorithm for them
  with
• however large ?, unless PNP

41

• Thm TSP is inapproximable unless PNP
• Pf
• Let G be a graph with n nodes.
• UHAMCYCLE ?p TSP
• If G has a Hamiltonian cycle, then the optimum
  cost of a tour is n otherwise if G has no
  Hamiltonian cycle, then the optimum cost of a
  tour gt n(1?)

1
?
23?
1
42

• If TSP had an ?-approximation algorithm A, then
  we would be able to tell whether G has a
  Hamiltonian cycle.
• Run A for TSP
• igt if the returned cost ? n(1?)1 No cycle
• iigt if the returned cost ? n(1?) has a cycle
• Unless PNP, TSP is inapproximable.

43

• Backtracking and BranchBound
• AS0
• while A is not empty do
• choose a subproblem S and delete it from A
• choose a way of branching out of S , say to
  subproblems
• S1, S2,, Sr
• for each subproblem Si in this list do
• if test(Si) returns solution found
• else if test(Si) returns ? then add Si to A
• Return no solution

44

• Eg.

x T
x F
y T
y F
z T
z F
45

• BranchBound algorithm
• AS0 , best_so_far?
• while A is not empty do
• choose a subproblem S and delete it from A
• choose a way of branching out of S , say to
  subproblems
• S1, S2,, Sr
• for each subproblem Si in this list do
• if Si1 then update best_so_far
• else if lowerbound(Si) lt best_so_far then add
  Si to A
• Return best_so_far

46

• Local improvement algorithm
• S initial solution
• while there is a solution S such that
• S is a neighbor of S and cost(S) lt cost(S)
• do SS
• Return S

47

• Simulated annealing
• S initial solution TT0
• repeat
• generate a random solution S such that
• S is a neighbor of S and let ?
  cost(S)?cost(S)
• if ? ? 0 then SS
• else SS with probability e-?/T
• update (T)
• until T0
• Return the best solution seen!
• T temperature