CS4018 Formal Models of Computation weeks 20-23 Computability and Complexity

1
CS4018Formal Models of Computation weeks 20-23
Computability and Complexity

• Kees van Deemter
• (partly based on lecture notes by Dirk Nikodem)

2
Second set of slidesSome uncomputable problems

• The halting problem
• An uncomputable integer function
• Noncomputability proofs by reduction
• Turing Machines
• Posts Correspondence Problem

3
Some uncomputable problems

• Some problems must be uncomputable. (Stronger
  some integer functions must be uncomputable.)
• But our diagonalisation proof was not
  constructive. Can we find examples of
  uncomputable problems/functions?
• First example (also historically first) the
  halting problem. Halt(P,I) means that program P
  halts on input I.
• This is a Boolean problem, so we will also speak
  of decidability instead of computability

4
Some appetisers (B.Russell)

• Suppose a barber, John, cuts everyones hair who
  does not cut his own hair (and noone elses).
• Does John cut his own hair?

5
Some appetisers (B.Russell)

• Suppose a barber, John, cuts everyones hair who
  does not cut his own hair (and noone elses).
• Does John cut his own hair?
• Suppose he does. Then hes the type of person
  whose hair he doesnt cut.
• Suppose he does not. Then hes the type of person
  whose hair he does cut.
• Paradox!

6
Original home of Russells paradox
(old-fashioned) set theory

• Assumption 1 Give me a property, then Ill give
  you a set.
• Assumption 2 Sometimes, a set is a member of
  itself. (E.g., the set of all sets is a set.)
  Thats a property!
• Let S def x x ? x
• Is S ? S?
• If yes (i.e., S ? S) then S ? S, so noIf no
  (i.e., S ? S) then S ? S, so yes

7
Halt(P,I) P terminates on input I

• Weve seen examples of programs whose termination
  is unknown
• It does not follow that no algorithm exists
• Whats your intuition? Could there exist an
  algorithm for Halt(P,I)?

8
Halt(P,I) P terminates on input I

• Suppose P halts on I, then it halts in finite
  time. Consequently, well discover by letting P
  run on I.
• We say that this makes Halt(P,I) partially
  decidable.

9
Halt(P,I) P terminates on input I

• Some problems are neither wholly nor partially
  decidable. (In fact, theres a hierarchy of
  decidability, e.g., Harel Ch. 8)
• Example of such a highly undecidable problem the
  totality problem, AlwaysHalts(P), which asks
  whether P halts on every input.
• Now lets have a look at Halt(P,I)
• Observe that programs may be applied to other
  programs. (E.g., a compiler)

10
Proof of undecidability of Halt

• Suppose Halt(P,I) was decidable
• Then Halt(P,P) would also be decidable
• Supposing all this, lets cook up a paradoxical
  algorithm

11
Proof of undecidability of Halt

• Suppose the programming language L contains a
  construction called loop, which loops forever.
  (E.g., loop while True do)
• Suppose L also contains a stop.
• Then heres a paradoxical program S
• S(P) If Halt(P,P) then loop else stop

12
Proof (ctd.)

• S(P) If Halt(P,P) then loop else stop
• Q Does S(S) halt? S(S) is
• If Halt(S,S) then loop else stop
• Suppose S(S) halts. Then Halt(S,S)True, so loop
  is called, so S(S) loops.
• Suppose S(S) does not halt. Then
  Halt(S,S)False, so stop is called, so S(S)
  halts.

13
Proof (ctd.)

• Weve derived a contradiction from the
• assumption that Halt(P,I) can be
• programmed.
• S(P) If Halt(P,P) then loop else stop
• Compare Russell
• loop gt not halt
• stop gt halt

14
Proof (ctd.)

• How satisfying is this proof?
• We now have an actual example of an undecidable
  problem.
• Yet, the proof of its undecidability is rather
  indirect
• A diagonalisation perspective
• consider a table matching programs P with
• Inputs I, asking whether (P,I) halts

15

• P1 P2 P3 P4 P5
• P1 y n y n
  n
• P2 n n n y
  y
• P3 n y y n
  y
• P4 n y y n
  y
• P5 y n y y
  n
• ... ...
• Diagonal records whether P(P) halts.
• Recall S(P) says If Halt(P,P) then loop else
  stop
• S is the negation of the diagonal
• S(P)Halt iff P(P) does not halt
• S n y n y
  y
• S differs from every program in the list! It
  differs
• from P1 on argument P1,
• from P2 on argument P2, etc.

16
An uncomputable integer function

• Many non-computability results are inferred from
  others. This is called reduction (the third proof
  method after enumeration and diagonalisation).
• Many noncomputability results are inferred from
  the noncomputability of Halting
• Before explaining proofs by reduction, lets
  perform diagonalisation once again

17
Defining an integer function

• Recall Integer function F N?N
• Recall most of these functions must be
  uncomputable (proven by enumeration)
• Yet, examples are difficult to find
• Enumerate strings, for example in lexicographical
  order
• wn the n-th string
• F will hinge on the length of a program, so we
  define
• Hn the set of programs of length n

18
Defining an integer function

• wn the n-th string in the lexicographical
  ordering
• Hn the set of programs of length n
• ----------------------------------------------
  ------------------
• P(n) the number of the string that results
  when P is applied to wn
• Were interested in programs that are applied to
  their own length Zn P(n) P ? Hn
• (Observe for some n, Zn may be empty.)
• Un MAX(Zn), or 0 if Zn is empty
• F(n) U(n)1 (i.e., 1 more than the maximum)

19

• Zn P(n) P ? Hn
• Un MAX(Zn), or 0 if Zn is empty
• F(n) U(n)1 (i.e., 1 more than the maximum)
• --------------------------------------------------
  ------------------------
• Observe Once the programming language is
  specified
• and the ordering of strings, then F is a
  well-defined,
• total, integer function.
• Yet, no program can calculate this function.
• Suppose program P did, while P ? Hj. .
• How would P(j) F(j) compare with U(j)?
• U(j) is the largest element of Zj P(j) P ?
  Hj .
• But P(j)U(n)1, so it purports to be even
  larger.
• Since P(j) ? Zj P(j) P ? Hj as well,
• this is a contradiction.

20
Our last proof method reduction

• Problem 1 How do you make tea, starting from an
  empty kettle? Answer
• fill the kettle
• put it on cooker,
• let it boil,
• pour water into teapot,
• add tea,
• wait 2 minutes. DONE

21
Our last proof method reduction

• Problem 2 How do you make tea, starting from a
  kettle thats partly filled?

22
Our last proof method reduction

• Problem 2 How do you make tea, starting from a
  kettle thats partly filled?
• Empty the kettle
• Then do as in solution to Problem 1.
• Thats the spirit of reduction!
• Reduction is important in computability,
• and in complexity as well.

23
Reduction (informally)

• Suppose we wonder whether problem P1 is
  computable
• It sometimes suffices to find a mapping between
  P1 and another problem P2, whose computablity is
  known
• One direction P2 is computable. Mapping should
  transform every P1-problem into a P2-problem.
  Solution of P2 implies solution in P1.
• Main direction P2 is not computable. Mapping
  should transform every P2-problem into a
  P1-problem. Non-existence of some P2 solutions
  implies non-existence of some P1-solutions.
  Hence, P1 must also be noncomputable.

24
Reduction (informally)

• Terminology
• Hypothetical algorithm for P1 is called an
  oracle.
• P2 is being reduced to P1.(Easy to get
  confused!)
• Also P1 reduced from P2.
• (Deduce contradiction from the assumption
• that an oracle exists)

25
Reduction (more formally)

• Target problem P1 used as oracle for uncomputable
  P2 (e.g., P2Halting)
• P1 and P2 can have different domains(i.e.,
  different possible inputs) both have Boolean
  output.
• Find function f Dom(P2)?Dom(P1), such that
  P2(x) is true iff P1(f(x)) is true
• f itself has to be computable

26
Reduction (more formally)

• P2(x) is true iff P1(f(x)) is true
• Note the asymmetry
• f is a function but not necessarily one-to-one
  (i.e., may not be a bijection)
• Hence, P1 can be used as oracle for P2, but not
  conversely (unless f is one-to-one)

27

• Many reductions are reductions from the Halting
  problem. (Halting is reduced to the new problem
  hypothetical new algorithm is oracle)
• Halting is (hypothetically) reduced to the target
  problem any solution to the target problem would
  imply a solution to Halting.
• Some simple examples. (Caveat most reductions
  are much less expected than these.)

28
AlwaysHalt(P)

• Also know as the Totality Problem, or the
  Uniform Halting Problem
• AlwaysHalt(P) is true iff ?I(Halt(P,I))
• Intuitively, AlwaysHalt(P) is related to
  Halt(P), and it is harder. Therefore, it is
  natural to reduce it from Halt(P).
• Reduction is based on the equivalence
• Halt(P,I) iff AlwaysHalt(f(P))

29
AlwaysHalt(P)

• Reduction is based on the equivalence
• Halt(P,I) iff AlwaysHalt(f(P))
• f works by incorporating the input I into the
  program P instead of reading it from outside,
  it is represented in the programf(P) (See next
  slide for an example)
• f(P) does not take any input, hence
  AlwaysHalt(f(P)) just means that f(P) terminates.

30

• Recall Algorithm B (input)
• var xinteger
• (1) while xltgt1 do if x is even
• then
  do xx/2
• else
  do x3x1
• (2) stop
• To determine if Halt(B,29), check whether
  AlwaysHalt(f(B))
• Algorithm f(B)
• x29
• (1) while xltgt1 do if x is even
• then do xx/2
• else do x3x1
• (2) stop

31
Summary of argument

1. Suppose Alwayshalt was computable
2. Consider arbitrary program P, input I
3. From (1) Alwayshalt(f(P)) would be comp.
4. f(P) halts for every input means simplythat
   f(P) halts
5. Outcome of Alwayshalt(f(P)) would tell you
   whether Halt(P,I)(N.B. I is made part of f(P))
6. This cant be true, so (1) must be false.

32
Another reduction from HaltingProgram
verification sketch

• Definition Correct(Program,Problem) iff
  Program correctly solves Problem.
• If Correct(Program,Problem) was computable then
  many computer scientists would be hunting for a
  solution! Unfortunately it is not.
• Intuition To determine whether
  Correct(Program,Problem), you also have to
  termine when Program terminates. Hence if
  Correct(Program,Problem) was decidable then
  Halting would be decidable.

33
Reduction is transitive

• If Halting is reduced to P1 andP1 is reduced to
  P2, then this is an indirect reduction of
  Halting to P2(thereby proving the
  uncomputability of P1 and P2).
• Thats how many results are proven reductions to
  reductions to () reductions

34
Turing Machines

• Historically the first mathematical model of
  computation
• Still the most widely known model
• Computability can be studied by focussing on
  Turing machines completely
• We will only take a brief look

35
TMs as simple models of computation (following
Harel 1987)

• Advantage a simple model makes it easy to prove
  things
• Drawback to express an algorithm in TMs is a lot
  of work
• Simplifying data strings of symbols on tape
• Simplifying basic operations transform one
  symbol into another
• Simplifying control moving 1 place along
  tape,left or right, depending on state and data.
  (State can encode memory!)

36
Turing machines (informal)

1. A head moving over a tape, readingand writing
   symbols
2. The head can move left as well as right
3. The head can correct the tape, replacing one
   symbol with another
4. Actions determined by the state of the machine,
   and by the symbols encountered ( input)
5. Initial state success states

37
(No Transcript)
38
Some observations

• Start, Success
• Halting states (4,5)
• Role of B
• Symmetry (states 1 and 2)
• Which strings end in a success state?
• Give it input string abab

39

• a. (0) BBababBB j. (3) BBxxxxBB
• b. (1) BBxbabBB k. (3) BBxxxxBB
• c. (3) BBxxabBB l. (3) BBxxxxBB
• d. (3) BBxxabBB m. (0) BBxxxxBB
• e. (0) BBxxabBB n. (0) BBxxxxBB
• f. (0) BBxxabBB o. (0) BBxxxxBB
• g. (0) BBxxabBB p. (0) BBxxxxBB
• h. (1) BBxxxbBB q. (0) BBxxxxBB
• i. (3) BBxxxxBB r. (4) Success!

40
Better names for states 0-5?
41
States as control/memory

• 4success
• 5failure
• 1 a found gt b found
• 2 b found gt a found
• 0 and 3 a found b found
• 0 looking for more (towards right edge)
• 3 moving back to the left edge

42
Turing machines (formal)TM ltQ,?,?,?,q0,B,Fgt

• Q set of states
• ? set of allowable tape symbols
• B ? ? is blank (or boundary)
• ? ? ?-B is finite input alphabet
• ? Q x ? ? Q x ? x ,-
• q0 ? Q is the initial state
• F is the (nonempty) set of success states
• (Note ? may be a partial function)

43
Express example TM in these terms

• Function ?
• Please specify as pairs ((state,symbol),(state,sym
  bol,direction))
• A representative sample of pairs will do

44
Example TM in formal terms

• Q 0,1,2,3,4,5
• ? a,b,x,B
• ? a,b
• ? ((0,a),(1,x,)), ((1,a)(1,a,)),
• q0 0
• F 4
• (failure state 5 does not require explicit
  marking)

45
Does this formal apparatus remind you of anything?
46
Finite State Automata

• (Also called Deterministic Finite Automata)

47
Turing machines

• TMs are formal automata, like Finite State
  Automata and pushdown store automata
• TMs are the strongest automata that tend to be
  studied. Generating power equals that of
  Type-0rewriting systems ??? ? ??? ,
  stronger than regular grammars, context free
  grammars, and even context-sensitive grammars
• Hopcroft Ullman 1979, Introduction to Automata
  Theory, Languages and Computation See also
  course on formal languages

48
FSA ltQ,?,?,q0,Fgt

• Q set of states
• ? finite input alphabet
• ? Q x ? ? Q is a transition function
• q0 ? Q is the initial state
• F is the (nonempty) set of success states
• Example

49

• abna (ngt0)

50

• FSA corresponds to a tape being read
• from left to right after a transition, move
• to the symbol to the right on the tape

51
Turing Machines

• The head can move left as well as right (as in
  two-way finite automata, seeHopcroft Ullman
  1979)
• The head can correct the tape, replacing one
  symbol with another
• Therefore, ? Q x ? ? Q (in FSAs) becomes ?
  Q x ? ? Q x ? x ,- (in TMs)

52
Play with TMs. E.g.,

• Build TM that recognises exactly all palindromes
  in the alphabeth a,b.(See Harel 87 for a
  solution)

53

• Fiddling with TMs does not extend or diminish
  their generating power. E.g.,
• Multiple tapes (more)
• Nondeterminism (more)
• Multidimensionality (more)
• More than one head (more)
• At most three states (less)
• (E.g., Hopcroft Ullman 79, chapter
  7)

54
TM a very robust notion

• Cf. the speed of light given that nothing can go
  faster than light, there must be something
  important about it.

55
Another noncomputability resultPosts
Correspondence Problem (PCP)

• Another uncomputable problem
• Proven by reduction from Halting
• This reduction is less obvious
• Many other problems have been proven undecidable
  by reduction from PCP
• General statement of the problem (example in
  lecture notes)

56
Viewed as a puzzle

• Can pieces be joined in such a way that
• top row symbols bottom row symbols ?

57

• In this case yes (but its hard in general)
• oxx xx xx o xxx xxx

58
Posts Correspondence Problem (PCP)

• Given n pairs of finite strings p1 lts11,s12gt
  ,, pn ltsn1,sn2gt
• n size of problem
• Notation First(pi) si1, Second(pi) si2
• Does there exist a sequence of pairs (possibly
  with iterations) ltpi1,,pikgt, such
  thatFirst(pi1) First(pik)Second(pi1)
  Second(pik)

59
Expressing example in this way

• Strings from alphabeth o,x
• Size of problem2 p1lts11,s12gt, p2lts21,s22gt
  ltoxx,ogt, ltxx,xxxgt
• Solution ltp1,p2,p2gt, sinceoxxxxxxoxxxxxxox
  xxxxx
• Does it matter that repetitions are allowed?

60
Looking ahead to complexity

• If repetitions were not allowed then the problem
  would be finite and therefore decidable (for all
  sizes n).
• n pairs can be ordered in maximally n! ways,
  e.g., 3 pairs a,b,c can be ordered abc,
  acb bac, bca, cab, cba
• Worst case, there are n! orders to check

61
Posts Correspondence Problem (PCP)

• p1 lts11,s12gt ,, pn ltsn1,sn2gt ,
• Size of problem n
• Decidability
• Size2 is decidable
• Size7 is undecidable
• Sizes in between are unknown

62
Lessons

• Some reductions are quite unexpected
• Whether a problem is computable can be difficult
  to predict

63
Where this comes from, there is much more

• E.g., Rices Theorem
• Only syntactic properties of programs
• are computable
• (Read Harel, one of the two books)

64
Remember high uncomputability

• AlwaysHalt
• Little brain teaser Suppose we fix the program P
  and the input I. Does there exist an algorithm
  for determining whether Halt(P,I)?