CS 3240: Languages and Computation

1
CS 3240 Languages and Computation

• Mapping Reducibility and Time Complexity

2
TMs and Computation

• TMs can do more than just accept and reject
  strings
• They can perform functions
• Definition A function f? ? ? is a computable
  function if there is some TM M that, on every
  input w, halts with f(w) on the tape

3
Examples

• The copying TM
• Start with w on the tape, halt with ww on the
  tape
• Summation
• Takes input ltm,ngt and returns value of mn
• Finding intersection of two DFAs
• Start with ltA,Bgt on the tape, where A and B are
  DFAs, halt with ltCgt on the tape, where L(C)
  L(A) ? L(B)

4
Mapping Reducibility

• Definition Language A is mapping reducible to
  language B, written A?mB, if there is a
  computable function f? ? ?, where for every w,
• w ? A iff f(w) ? B
• The function f is called the reduction of A to B.

B
A
5
Why m?

• In some sense, if A m B, then A is no more
  powerful than B
• For example, we can map from CFGs to decidable
  languages, but not vice versa
• Example
• We can easily map any CFG C to ACFG
• f(w) ltC,wgt
• w ? C iff ltC,wgt ? ACFG

6
Mapping Reductions Decidability

• Theorem If A ?m B and B is decidable, then A is
  decidable.
• Proof Let M be a decider for B and let f be a
  reduction from A to B.
• Consider the following TM, N
• N On input w
• Compute f(w)
• Run M on f(w) and report Ms output
• Then N decides A

7
Mapping Reductions Undecidability

• Corollary If A ?m B and A is undecidable, then
  B is undecidable.
• We have been using this corollary implicitly
  already

8
Example

• We showed that HALTTM is undecidable by
  contradiction
• Now lets show its undecidable using mapping
  reduction
• Need a function that takes input ltM,wgt
• Halts if M accepts w, and loops if not

9
Mapping from ATM to HALTTM

• F On input x
• If x ? ltM,wgt for some TM M, output x
• Otherwise, construct the following TM
• M On input x
• 1. Run M on x
• 2. If M accepts, accept
• 3. If M rejects, enter a loop
• 2. Output ltM,wgt
• If M accepts w M halts on w, otherwise M loops
  or generates a string not in HALTTM
• I.e., ltM,wgt?ATM iff ltM,wgt?HALTTM

10
HALTTM is Undecidable

• We just showed that ATM m HALTTM
• Since we know ATM is undecidable, we can conclude
  that HALTTM is undecidable

11
Reductions TM-recognizability

• Theorem If A ?m B and B is Turing-recognizable,
  then A is Turing-recognizable.
• Proof (same as decidable proof) Let M be a
  recognizer for B and let f be a reduction from A
  to B.
• Consider the following TM, N
• N On input w
• Compute f(w)
• Run M on f(w) and report Ms output
• Then N recognizes A

12
Applications of Mapping Reductions

• If A ?m B and B is decidable, then A is decidable
• If A ?m B and A is undecidable, then B is
  undecidable
• If A ?m B and B is Turing-recognizable, then A is
  Turing-recognizable
• Equivalently, A ?m B
• If A ?m B and A is not Turing-recognizable, then
  B is not Turing-recognizable

13
Reductions non-TM-recognizability

• Corollary If A ?m B and A is not
  Turing-recognizable, then B is not
  Turing-recognizable.
• Question Which language that have we seen is
  not Turing-recognizable?
• Answer ATM

14
EQTM

• Theorem EQTM is neither Turing-recognizable nor
  co-Turing-recognizable.
• Proof
• Reduce ATM to EQTM, or equivalent reduce ATM to
  EQTM.
• Reduce ATM to EQTM, or equivalent reduce ATM to
  EQTM.

15
Time Complexity
16
Complexity of Algorithms

• Among decidable problems, we still have levels of
  difficulty for algorithms
• We may consider an algorithm more difficult for a
  variety of reasons
• Takes longer to execute
• Requires more memory to execute
• Time complexity Given an algorithm and an input
  string, how long will the algorithm take to
  execute?

17
Time Complexity

• Definition Let M be any deterministic Turing
  machine that halts on all inputs. The running
  time or time complexity of M is the function
  fN?N, where f(n) is the maximum number of steps
  that M uses on any input of length n.

18
Big-O Notation

• Definition Let f and g be two functions from N
  to R (the set of positive real numbers). Then
  f(n)O(g(n)) if positive integers c and n0 exist
  such that for every n n0, f(n) c g(n). In
  this case, we say that g(n) is an upper bound for
  f(n)
• Remark
• Time complexity is most concerned with behavior
  for large n
• For polynomials, we can drop everything except
  nk, where k is the largest exponent

19
Big-O Notation and Logarithms

• With big-O notation, the base of the logarithm is
  not important!
• Recall logbn logxn / logxb
• logbn O(logxn) for every x gt 0
• 5n5 log3n 3n2log2log2n O(n5 log n)

20
Exponential time

• What about f(n) 2O(log n)?
• n 2log_2 n ? nc 2O(log n) ? 2O(log n) nO(1)
• An algorithm takes exponential time if its
  complexity is O(ank), where kgt0
• An algorithm takes polynomial time if its
  complexity is O(nk) for some k gt 0

21
Small-o Notation

• Definition Let f and g be two functions from N
  to R (the set of positive real numbers). Then
  f(n)o(g(n))
• limn??(f(n)/g(n)) 0
• i.e., for any positive real number c, a number
  n0 exists such that for every n ? n0, f(n) lt c
  g(n)

22
Big-O Versus Small-o

• If f(n) o(g(n)) then f(n) O(g(n)), but the
  reverse is not always true
• Big-O is like less than or equal while small-o
  is like strictly less than
• Example
• f(n) O(f(n)) for every function f
• f(n) ? o(f(n)) for every function f
• Inequality of big-O needs to hold for some
  positive constant c, but inequality of small-o
  must hold for every positive constant c

23
Analyzing algorithms

• We examine an algorithm to determine how long it
  will take to halt on an input of length n
• The amount of time to complete is called the
  algorithms complexity class
• Definition Let tN?N be a function. The time
  complexity class, TIME(t(n)), is
• TIME(t(n))L L is a language decided by a
  O(t(n))-time Turing machine

24
Importance of model

• The complexity of our algorithms depends on
  assumptions about our data other model
  assumptions
• The complexity of an algorithm can vary depending
  on the machine we use
• We assume reasonable encoding of all numbers
• Any integer of value v should take O(log v) space

25
Machine-dependent complexity

• Example, let Lw w is a palindrome
• How long will it take us to decide L on a
  standard TM?
• Go back and forth crossing off matching symbols
  at beginning and end
• O(n2)
• How long will it take us to decide L on a 2-tape
  TM?
• Copy string
• Compare symbols reading forward on tape 1 and
  backward on tape 2
• O(n)

26
Complexity relationships

• Theorem Let t(n) be a function, where t(n) ? n.
  Then every t(n) time multitape TM has an
  equivalent O(t2(n)) time single-tape TM
• Proof idea Consider structure of equivalent
  single-tape TM. Analyzing behavior shows each
  step on multi-tape machine takes O(t(n)) on
  single tape machine

27
Equivalent machines
28
Simulating k-Tape Behavior

• Single tape start string is
• w...
• Each move proceeds as follows
• Start at leftmost slot
• Scan right to (k1)st to find symbol at each
  virtual tape head
• Make second pass making updates indicated by
  k-tape transition function
• When a virtual head moves onto a , shift string
  to right

29
Proof of theorem

• Analyzing simulation of k-tape machine
• Each step on single-tape machine has two phases
• Scan tape
• Perform operations
• How long does first phase take?
• Length of string on tape
• Each portion has O(t(n)) length (this occurs if
  tape heads only move right)

30
Proof of theorem (cont.)

• How long does second phase take?
• Perform k steps
• Each step may require a right shift
• Each step takes O(t(n)) time
• Total of k steps is O(t(n)) because k is a
  constant
• Whats the total time?
• O(t(n)) steps each take O(t(n)) time
• Total time is O(t2(n))

31
Determinism vs. non-determinism

• Definition Let P be a non-deterministic Turing
  machine. The running time of P is the function
  fN?N, where f(n) is the maximum number of steps
  that P uses on any branch of its computation in
  any input of length n.

32
Non-Deterministic Tree
33
Complexity Relationship

• Theorem Let t(n) be a function where t(n)?n.
  Then every t(n) time non-deterministic
  single-tape Turing machine has an equivalent
  2O(t(n)) time deterministic single-tape Turing
  machine.

34
Complexity Relationship Proof

• Proof Given a non-deterministic TM, P, running
  in t(n) time, construct a 3-tape deterministic TM
  that simulates P.
• The height of the tree is at most t(n). Assume
  the maximum number of branches in the tree is b.
  Therefore, the number of leaves in the tree is
  O(bt(n)).

35
How Many Nodes in the Tree?

• Claim The total number of nodes is less than
  twice the number of leaves i.e. O(bt(n)).
• (S0kn bk) / bn (1 bn1)/(1-b) / bn
• lt bn1/(1-b) / bn
• - b / (1-b)
• 2

36
Complexity Relationship proof (cont.)

• Deterministic TM does a breadth-first search of
  the non-deterministic TMs tree.
• Total time to search tree is O(t(n)) to travel
  from the root to a leaf O(bt(n)), the number of
  leaves.
• O(t(n)bt(n)) O(2log_2 t(n) 2(log_2 b)t(n))
  O(2O(t(n)))

37
Complexity relationship proof (cont.)

• We constructed a 3-tape TM with running time
  O(2O(t(n)))
• Single-tape TM will take O((2O(t(n)))2)O(22O(t(n)
  ))O(2O(t(n)))