Relationships Between Complexity Classes

1
Relationships Between Complexity Classes

• Preliminaries
• proper complexity functions
• precise Turing Machines
• complements of classes
• The Hierarchy Theorems
• The Time Hierarchy Theorem
• The Space Hierarchy Theorem
• The Gap Theorem (maybe, if time allows)
• Relationships between classes
• Sawitchs Theorem
• Immerman-Szelepscenyi Theorem

2
Complexity functions

• What would be a good complexity function f(n)?
• f(n1) gt f(n)
• f(n) can be computed within time/space it
  allows
• there is a TM Mf that on input x with n x
  writes f(n) in unary on its output tame, and
  works in time O(nf(n)) and space O(f(n))
• called proper complexity function
• We define complexity classes using proper
  complexity functions.
• If non-proper complexity function is used,
  strange things might happen
• see The Gap Theorem There is a function
  recursive function f such that TIME(f(n))TIME(2f(
  n))

3
Complexity functions

• Note that all normal functions log n, logk n,
  n2, n32n2log3 n, n!, vn are proper
• sum, product, exponent of proper functions is a
  proper function
• Exercise Show that vn is a proper complexity
  function.
• Definition A (det or non-det) TM M is precise if
  there exists functions f and g such that for
  every ngt0, for every input x such that x n,
  and for every computation M halts precisely after
  f(n) steps and uses precisely g(n) space on each
  (except first and last, if it is input-ouput TM)
  of its tapes.

4
Precise TM
Theorem Let M be a (det or non-det) TM that
decides language L within time (space) f(n),
where f is a proper function. Then there exists a
precise TM M that decides L in time (space)
O(f(n)). Proof Start by simulating Mf using new
set of tapes. If f(n) is time bound, M then
simulates M, but in each step it advances a
head on the yardstick left by Mf. If M halts,
M continues to move the yardstick head until a
blank is encountered and only then halts, after
precisely f(n) steps of M. If f(n) is space
bound, simulating Mf uses the allotted space and
simulation of M fits inside. The overall
time/space is that of sum of the ones used by Mf
and M and is therefore O(f(n)).
5
Complexity Classes

• TIME(f), SPACE(f)
• NTIME(f), NSPACE(f)
• (N)TIME(nk), (N)SPACE(nk)
• P, NP
• PSPACE, NSPACE
• EXP TIME(2nk)
• L SPACE(log n)
• NP NSPACE(log n)

6
Complements of Classes

• Complement of a language
• x is from coL if an only if x is not in L
• Complement of a decision problem Pr
• answer is yes in coP if an only if the input
  is no input for P
• Example
• Ha Is the graph Hamiltonian?
• coHa The graph does not contain Hamiltonian
  path
• Is it true that LcoHa coLHa?
• consider inputs not encoding graphs

7
Complements of Classes

• Complement of classes
• for a complexity class C, coC coL L is in C
  
• Deterministic classes are closed under
  complement
• if a det. machine decides within specified
  time/space bound, it always halts and we can
  simply reverse the answer
• What about non-deterministic classes?
• we cannot simply reverse the answer (one
  computation does not determine the answer,
  asymmetry between yes and no)
• we will see that NSPACE coNSPACE
  (Immerma-Szelepczenyi theorem)
• NTIME coNTIME is an important open problem

8
The Time Hierarchy Theorem

• We know that TIME(n2) TIME(5n2) for f at least
  linear and a constant c. (From linear speedup
  theorem).
• What about TIME(n) vs TIME(n3)
• maybe there is a clever version of the linear
  speedup theorem that would show TIME(n)
  TIME(n3)?
• We will show that that is not the case
• there is an infinite sequence of sharply
  stronger complexity classes
• intuitively obvious you should be able to do
  more in time n3 that in time n
• but then you should be able to do more in time
  5n2 that in time n2, so dont rely too much on
  intuition

9
The Time Hierarchy Theorem
We want to find a language Hf such that Hf is in
TIME(f(n)2log3(n)) but not in TIME(f(n)). Idea
diagonalization Let f(n) gt n be a proper
complexity function. Define Hf M M(M) yes
after at most f(x) steps. Lemma Hf is in
TIME(f(n)2log3(n))
10
Hf is in TIME(f(n)2log3(n))

• We need to show a TM Uf that decides Hf in time
  f(n)2log3(n).
• use 4 tapes
• tape 1 input
• tape 2 state of M symbols below the head
• tape 3 current configuration of M
• tape 4 yardstick of length f(n)

11
Hf is in TIME(f(n)2log3(n))

• First
• simulate Mf on the input to initialize the
  yardstick
• time O(f(n))
• Second
• copy the input M -gt MM, cost O(n) O(f(n))
• Third
• use UTM to simulate M on MM
• in each step advance the yardstick
• each step costs O(lMk2Mf(n)) O(f(n)log3(n))
• together O(f2(n)log3(n))
• Fourth
• if M says yes while the yardstick has not
  expired, say yes, otherwise say no

12
Hf is not in TIME(f(n))

• Assume that there is a TM MHf deciding Hf in time
  f(n).
• Take the following Df(M)
• if MHf yes then no else yes
• Df(M) also works in time f(n).
• What is the output of Df(Df)?
• if Df(Df) yes then MHf(Df) no (from
  definition of Df)
• if MHf(Df) no then Df(Df) no (from
  definition of Hf)
• similarly when Df(Df) no

13
Time Hierarchy Theorem

• Theorem If f(n)gt n is a proper complexity
  function, then class TIME(f(n)) is strictly
  contained in the class TIME(f2(n)log3(n)).
• In fact, stronger property is true TIME(f(n)) is
  strictly contained in the class
  TIME(f(n)log2(n)).
• Corollary P is a proper subset of EXP.
• P is obviously a subset of EXP (for any k, 2n is
  eventually bigger then nk).
• TIME(2n) contains P, but is a proper subset of
  TIME(22nn3) (by time hierarchy theorem), which is
  in EXP

14
Space Hierarchy Theorem
Theorem If f(n)gt n is a proper complexity
function, then class SPACE(f(n)) is strictly
contained in the class SPACE(f(n)log(n)). Proven
using similar ideas. We know that a space-bound
machine will always stop (there are
Sf(n)kKf(n)k possible configurations) Note
Time and space hierarchies can be proven also for
non-deterministic TMs (but the proofs are much
more complicated).
15
The Gap Theorem

• Theorem There is a recursive function f from N
  to N such that TIMEf(n) TIME(2f(n)).
• we will define f in such a way that no TM
  computing on input n will halt between f(n) and
  2f(n) steps.
• order all TMs M0, M1, M2,
• Define P(i,k) No machine among M0, M1, , Mi on
  any input of length i will halt between k and 2k
  steps.
• note that P(i,k) can be decided by a TM just
  simulate all of them for all inputs for up to 2k
  steps

16
The Gap Theorem

• Let us now define f(i)
• k1 2i, kj 2kj-11
• N(i) of all inputs of length i for the first
  i1 machines
• each input can make P(i, kj) false for at most
  one j, so there is a klltN(i) such that P(i, kl)
  is true
• Define f(i) kl
• Note that f(i) is really unnatural and grows
  crazily fast.
• Consider now L in TIME(2f(n)). L is decided by
  some Mj in time at most 2f(n). For any input x
  with x gtj. Mj does not halt after more then
  f(x) steps (from definition of f(i)), i.e. it
  halts in at most f(x) steps. But then there is
  (a very big, but finite) machine M which on
  inputs of length less then j decides case-by-case
  (in time O(n), and on longer ones does what M
  does (and therefore decides in time O(f(n)).
  Therefore L is in TIME(f(n)).

17
The Reachability Method

• Theorem Let f(n) be a proper complexity
  function. Then
• SPACE(f(n) ? NSPACE(f(n)) and TIME(f(n)) ?
  NTIME(f(n))
• NTIME(f(n)) ? SPACE(f(n))
• NSPACE(f(n)) ? TIME(klognf(n))
• Proof a) trivial, as det. TM is a special case
  of non-det TM
• b) We have seen the proof already generate all
  sequences of non-det choices of length f(n) and
  simulate them one by one

18
The Reachability Method

• Proof of c) NSPACE(f(n)) ? TIME(klognf(n))
• let M be k-string non-det (possibly
  input-output) TM that accepts L in space f(n)
• the configuration of M captures the state,
  position of the input head and state of all
  tapesb
• there are less then (n1)KS(2k-2)f(n)
  configurations
• consider the configuration graph GM(V,E)
• the vertices are configurations
• (u,v) is an edge if u yields v in one step
• determining whether M accepts input word x is
  equivalent to checking whether there is a path
  from the initial configuration to an accepting
  configuration
• for a given graph G of n vertices, there is a
  O(n2) deterministic algorithm for reachability
• applying this algorithm on GM proves the theorem

19
The Reachability Method
Corollary L ? NL ? P ? NP ? PSPACE Nuts We know
that L is a proper subset of PSPACE, and suspect
that each of the inclusions is proper, but cannot
prove that for any of them.
20
The Nondeterministic Space

• We can simulate non-deterministic TM by
  deterministic TM with exponential slowdown. Since
  in time f(n) at most f(n) space can be used, from
  c) of the last theorem we get that NSPACE(f(n)) ?
  SPACE(klognf(n)), i.e. Exponential growth of
  space.
• Can we simulate NSPACE better?
• yes, again using the reachability argument in
  the configuration graph
• Theorem (Savitch) Reachability ?SPACE(log2n).
• i.e. in graph of n vertices we can check in
  O(log2n) space whether there is a path between u
  and v

21
Savitchs Theorem

• Consider the following program started with x
  n
• bool Reachable(u,v,x)
• if (x 1) return (uv) or (u and v are
  neighbours in G)
• else
• for every vertex z in G
• if (Reachable(u,z,x/2) and Reachable(z,v,x/2))
  return true
• return false
• What is the running time?
• What is the space complexity?
• depth of recursion log n, O(log n) space needed
  in one level

22
Savitchs Theorem - Consequences

• Corollary NSPACE(f(n)) ?SPACE(f2(n)).
• Apply the Savitchs theorem on the configuration
  graph of the machine M accepting language L from
  NSPACE(f(n)) - it has cf(n) nodes for some c,
  logarithm of that is O(f(n)) so we can
  deterministically check whether x is in L in
  f2(n) space.
• Corollary PSPACE NPSPACE
• would be nice if that held also for the time
  classes ?

23
Complement of NSPACE

• It is an open problem whether NTIME classes are
  closed under complement (strongly suspected that
  not).
• It has been a long standing open problem whether
  NSPACE classes are closed under complement
  (strongly suspected that not)
• but it has been proven (almost simultaneously by
  Immerman and Szelepscenyi) that indeed they are
• Definition A non-det TM computes function iff
• all successful computations produce the same
  output
• all unsuccessful computations end in state no

24
Exercise nondet TM refresher
Given a graph G, two nodes u and v (given as
integers) and integer k, describe a non-det TM
that answers yes if and only if there is a path
of length at most k from u to v.
25
Immerman-Szelepcsenyi Theorem
Given a graph G and a node s, the number of
nodes reachable from s in G can be computed by a
non-det TM within space log n. Let S(i) be the
sets of nodes reachable from s by paths of length
at most i. S(0)1 for(k1,2,n-1) compute
S(k) from S(k-1)
26
Immerman-Szelepcsenyi Theorem
S(0)1 for (k1,2,n-1) compute S(k) from
S(k-1)
27
Immerman-Szelepcsenyi Theorem
S(0)1 for (k1,2,n-1) l 0 for each node
u 1,2,...,n if u ? S(k) then ll1 S(k)
l
28
Immerman-Szelepcsenyi Theorem
S(0)1 for (k1,2,n-1) l 0 for each node
u 1,2,...,n if check(u,k,S(k-1)) then
ll1 S(k) l
29
Immerman-Szelepcsenyi Theorem
bool check(u, k, S(k-1)) m 0 reply
false for each node v 1,2,,n if v ? S(k-1)
then // the exercise we just
had mm1 if u and v are neighbours in G reply
true if (m S(k-1)) return reply else
return false
30
Immerman-Szelepcsenyi Discussion

• What was the space needed?
• 4 nested loops, in each loop we needed to
  remember constant number of variables of size at
  most n, i.e. O(log n) bits.
• Corollary If f gt log n is a proper complexity
  function, then NSPACE(f(n)) coNSPACE(f(n))
• Run this algorithm on the configuration graph of
  ML for L in NSPACE(f(n)), with s being the
  initial configuration, and up to kn-1. If an
  accepting configuration is encountered, return
  no. For kn-1 all reachable configurations have
  been reached, if none was accepting, then ML does
  not accept the input return yes.