CS4018 Formal Models of Computation weeks 2023 Computability and Complexity

1
CS4018Formal Models of Computation weeks 20-23
Computability and Complexity

• Kees van Deemter
• (partly based on lecture notes by Dirk Nikodem)

2
Second set of slidesSome uncomputable problems

• The halting problem
• An uncomputable integer function
• Noncomputability proofs by reduction
• Turing Machines
• Posts Correspondence Problem

3
Some uncomputable problems

• Some problems must be uncomputable. (Stronger
  some integer functions must be uncomputable.)
• But our diagonalisation proof was not
  constructive. Can we find examples of
  uncomputable problems/functions?
• First example (also historically first) the
  halting problem. Halt(P,I) means that program P
  halts on input I.
• This is a Boolean problem, so we will also speak
  of decidability instead of computability

4
Some appetisers (B.Russell)

• Suppose a barber, John, cuts everyones hair who
  does not cut his own hair (and noone elses).
• Does John cut his own hair?

5
Some appetisers (B.Russell)

• Suppose a barber, John, cuts everyones hair who
  does not cut his own hair (and noone elses).
• Does John cut his own hair?
• Suppose he does. Then hes the type of person
  whose hair he doesnt cut.
• Suppose he does not. Then hes the type of person
  whose hair he does cut.
• Paradox!

6
Original home of Russells paradox
(old-fashioned) set theory

• Assumption 1 Give me a property, then Ill give
  you a set.
• Assumption 2 Sometimes, a set is a member of
  itself. (E.g., the set of all sets is a set.)
  Thats a property!
• Let S def x x ? x
• Is S ? S?
• If yes (i.e., S ? S) then S ? S, so noIf no
  (i.e., S ? S) then S ? S, so yes

7
Halt(P,I) P terminates on input I

• Weve seen examples of programs whose termination
  is unknown
• It does not follow that no algorithm exists
• Whats your intuition? Could there exist an
  algorithm for Halt(P,I)?

8
Halt(P,I) P terminates on input I

• Suppose P halts on I, then it halts in finite
  time. Consequently, well discover by letting P
  run on I.
• We say that this makes Halt(P,I) partially
  decidable.

9
Halt(P,I) P terminates on input I

• Some problems are neither wholly nor partially
  decidable. (In fact, theres a hierarchy of
  decidability, e.g., Harel Ch. 8)
• Example of such a highly undecidable problem the
  totality problem, AlwaysHalts(P), which asks
  whether P halts on every input.
• Now lets have a look at Halt(P,I)
• Observe that programs may be applied to other
  programs. (E.g., a compiler)

10
Proof of undecidability of Halt

• Suppose Halt(P,I) was decidable
• Then Halt(P,P) would also be decidable
• Supposing all this, lets cook up a paradoxical
  algorithm

11
Proof of undecidability of Halt

• Suppose the programming language L contains a
  construction called loop, which loops forever.
  (E.g., loop while True do)
• Suppose L also contains a stop.
• Then heres a paradoxical program S
• S(P) If Halt(P,P) then loop else stop

12
Proof (ctd.)

• S(P) If Halt(P,P) then loop else stop
• Q Does S(S) halt? S(S) is
• If Halt(S,S) then loop else stop
• Suppose S(S) halts. Then Halt(S,S)True, so loop
  is called, so S(S) loops.
• Suppose S(S) does not halt. Then
  Halt(S,S)False, so stop is called, so S(S)
  halts.

13
Proof (ctd.)

• Weve derived a contradiction from the
• assumption that Halt(P,I) can be
• programmed.
• S(P) If Halt(P,P) then loop else stop
• Compare Russell
• loop gt not halt
• stop gt halt

14
Proof (ctd.)

• How satisfying is this proof?
• We now have an actual example of an undecidable
  problem.
• Yet, the proof of its undecidability is rather
  indirect
• A diagonalisation perspective
• consider a table matching programs P with
• Inputs I, asking whether (P,I) halts

15

• P1 P2 P3 P4 P5
• P1 y n y n
  n
• P2 n n n y
  y
• P3 n y y n
  y
• P4 n y y n
  y
• P5 y n y y
  n
• ... ...
• Diagonal records whether P(P) halts.
• Recall S(P) says If Halt(P,P) then loop else
  stop
• S is the negation of the diagonal
• S(P)Halt iff P(P) does not halt
• S n y n y
  y
• S differs from every program in the list! It
  differs
• from P1 on argument P1,
• from P2 on argument P2, etc.

16
An uncomputable integer function

• Many non-computability results are inferred from
  others. This is called reduction (the third proof
  method after enumeration and diagonalisation).
• Many noncomputability results are inferred from
  the noncomputability of Halting
• Before explaining proofs by reduction, lets
  perform diagonalisation once again

17
Defining an integer function

• Recall Integer function F N?N
• Recall most of these functions must be
  uncomputable (proven by enumeration)
• Yet, examples are difficult to find
• Enumerate strings, for example in lexicographical
  order
• wn the n-th string
• F will hinge on the length of a program, so we
  define
• Hn the set of programs of length n

18
Defining an integer function

• wn the n-th string in the lexicographical
  ordering
• Hn the set of programs of length n
• ----------------------------------------------
  ------------------
• P(n) the number of the string that results
  when P is applied to wn
• Were interested in programs that are applied to
  their own length Zn P(n) P ? Hn
• (Observe for some n, Zn may be empty.)
• Un MAX(Zn), or 0 if Zn is empty
• F(n) U(n)1 (i.e., 1 more than the maximum)

19

• Zn P(n) P ? Hn
• Un MAX(Zn), or 0 if Zn is empty
• F(n) U(n)1 (i.e., 1 more than the maximum)
• --------------------------------------------------
  ------------------------
• Observe Once the programming language is
  specified
• and the ordering of strings, then F is a
  well-defined,
• total, integer function.
• Yet, no program can calculate this function.
• Suppose program P did, while P ? Hj. .
• How would P(j) F(j) compare with U(j)?
• U(j) is the largest element of Zj P(j) P ?
  Hj .
• But P(j)U(n)1, so it purports to be even
  larger.
• Since P(j) ? Zj P(j) P ? Hj as well,
• this is a contradiction.

20
Our last proof method reduction

• Problem 1 How do you make tea, starting from an
  empty kettle? Answer
• fill the kettle
• put it on cooker,
• let it boil,
• pour water into teapot,
• add tea,
• wait 2 minutes. DONE

21
Our last proof method reduction

• Problem 2 How do you make tea, starting from a
  kettle thats partly filled?

22
Our last proof method reduction

• Problem 2 How do you make tea, starting from a
  kettle thats partly filled?
• Empty the kettle
• Then do as in solution to Problem 1.
• Thats the spirit of reduction!
• Reduction is important in computability,
• and in complexity as well.

23
Reduction (informally)

• Suppose we wonder whether problem P1 is
  computable
• It sometimes suffices to find a mapping between
  P1 and another problem P2, whose computablity is
  known
• One direction P2 is computable. Mapping should
  transform every P1-problem into a P2-problem.
  Solution of P2 implies solution in P1.
• Main direction P2 is not computable. Mapping
  should transform every P2-problem into a
  P1-problem. Non-existence of some P2 solutions
  implies non-existence of some P1-solutions.
  Hence, P1 must also be noncomputable.

24
Reduction (informally)

• Terminology
• Hypothetical algorithm for P1 is called an
  oracle.
• P2 is being reduced to P1.(Easy to get
  confused!)
• Also P1 reduced from P2.
• (Deduce contradiction from the assumption
• that an oracle exists)

25
Reduction (more formally)

• Target problem P1 used as oracle for uncomputable
  P2 (e.g., P2Halting)
• P1 and P2 can have different domains(i.e.,
  different possible inputs) both have Boolean
  output.
• Find function f Dom(P2)?Dom(P1), such that
  P2(x) is true iff P1(f(x)) is true
• f itself has to be computable

26
Reduction (more formally)

• P2(x) is true iff P1(f(x)) is true
• Note the asymmetry
• f is a function but not necessarily one-to-one
  (i.e., may not be a bijection)
• Hence, P1 can be used as oracle for P2, but not
  conversely (unless f is one-to-one)

27

• Many reductions are reductions from the Halting
  problem. (Halting is reduced to the new problem
  hypothetical new algorithm is oracle)
• Halting is (hypothetically) reduced to the target
  problem any solution to the target problem would
  imply a solution to Halting.
• Some simple examples. (Caveat most reductions
  are much less expected than these.)

28
AlwaysHalt(P)

• Also know as the Totality Problem, or the
  Uniform Halting Problem
• AlwaysHalt(P) is true iff ?I(Halt(P,I))
• Intuitively, AlwaysHalt(P) is related to
  Halt(P), and it is harder. Therefore, it is
  natural to reduce it from Halt(P).
• Reduction is based on the equivalence
• Halt(P,I) iff AlwaysHalt(f(P))

29
AlwaysHalt(P)

• Reduction is based on the equivalence
• Halt(P,I) iff AlwaysHalt(f(P))
• f works by incorporating the input I into the
  program P instead of reading it from outside,
  it is represented in the programf(P) (See next
  slide for an example)
• f(P) does not take any input, hence
  AlwaysHalt(f(P)) just means that f(P) terminates.

30

• Recall Algorithm B (input)
• var xinteger
• (1) while xltgt1 do if x is even
• then
  do xx/2
• else
  do x3x1
• (2) stop
• To determine if Halt(B,29), check whether
  AlwaysHalt(f(B))
• Algorithm f(B)
• x29
• (1) while xltgt1 do if x is even
• then do xx/2
• else do x3x1
• (2) stop

31
Summary of argument

• Suppose Alwayshalt was computable
• Consider arbitrary program P, input I
• From (1) Alwayshalt(f(P)) would be comp.
• f(P) halts for every input means simplythat
  f(P) halts
• Outcome of Alwayshalt(f(P)) would tell you
  whether Halt(P,I)(N.B. I is made part of f(P))
• This cant be true, so (1) must be false.

32
Another reduction from HaltingProgram
verification sketch

• Definition Correct(Program,Problem) iff
  Program correctly solves Problem.
• If Correct(Program,Problem) was computable then
  many computer scientists would be hunting for a
  solution! Unfortunately it is not.
• Intuition To determine whether
  Correct(Program,Problem), you also have to
  termine when Program terminates. Hence if
  Correct(Program,Problem) was decidable then
  Halting would be decidable.

33
Reduction is transitive

• If Halting is reduced to P1 andP1 is reduced to
  P2, then this is an indirect reduction of
  Halting to P2(thereby proving the
  uncomputability of P1 and P2).
• Thats how many results are proven reductions to
  reductions to () reductions

34
Turing Machines

• Historically the first mathematical model of
  computation
• Still the most widely known model
• Computability can be studied by focussing on
  Turing machines completely
• We will only take a brief look

35
TMs as simple models of computation (following
Harel 1987)

• Advantage a simple model makes it easy to prove
  things
• Drawback to express an algorithm in TMs is a lot
  of work
• Simplifying data strings of symbols on tape
• Simplifying basic operations transform one
  symbol into another
• Simplifying control moving 1 place along
  tape,left or right, depending on state and data.
  (State can encode memory!)

36
Turing machines (informal)

• A head moving over a tape, readingand writing
  symbols
• The head can move left as well as right
• The head can correct the tape, replacing one
  symbol with another
• Actions determined by the state of the machine,
  and by the symbols encountered ( input)
• Initial state success states

37
(No Transcript)
38
Some observations

• Start, Success
• Halting states (4,5)
• Role of B
• Symmetry (states 1 and 2)
• Which strings end in a success state?
• Give it input string abab

39

• a. (0) BBababBB j. (3) BBxxxxBB
• b. (1) BBxbabBB k. (3) BBxxxxBB
• c. (3) BBxxabBB l. (3) BBxxxxBB
• d. (3) BBxxabBB m. (0) BBxxxxBB
• e. (0) BBxxabBB n. (0) BBxxxxBB
• f. (0) BBxxabBB o. (0) BBxxxxBB
• g. (0) BBxxabBB p. (0) BBxxxxBB
• h. (1) BBxxxbBB q. (0) BBxxxxBB
• i. (3) BBxxxxBB r. (4) Success!

40
Better names for states 0-5?
41
States as control/memory

• 4success
• 5failure
• 1 a found gt b found
• 2 b found gt a found
• 0 and 3 a found b found
• 0 looking for more (towards right edge)
• 3 moving back to the left edge

42
Turing machines (formal)TM ltQ,?,?,?,q0,B,Fgt

• Q set of states
• ? set of allowable tape symbols
• B ? ? is blank (or boundary)
• ? ? ?-B is finite input alphabet
• ? Q x ? ? Q x ? x ,-
• q0 ? Q is the initial state
• F is the (nonempty) set of success states
• (Note ? may be a partial function)

43
Express example TM in these terms

• Function ?
• Please specify as pairs ((state,symbol),(state,sym
  bol,direction))
• A representative sample of pairs will do

44
Example TM in formal terms

• Q 0,1,2,3,4,5
• ? a,b,x,B
• ? a,b
• ? ((0,a),(1,x,)), ((1,a)(1,a,)),
• q0 0
• F 4
• (failure state 5 does not require explicit
  marking)

45
Does this formal apparatus remind you of anything?
46
Finite State Automata

• (Also called Deterministic Finite Automata)

47
Turing machines

• TMs are formal automata, like Finite State
  Automata and pushdown store automata
• TMs are the strongest automata that tend to be
  studied. Generating power equals that of
  Type-0rewriting systems ??? ? ??? ,
  stronger than regular grammars, context free
  grammars, and even context-sensitive grammars
• Hopcroft Ullman 1979, Introduction to Automata
  Theory, Languages and Computation See also
  course on formal languages

48
FSA ltQ,?,?,q0,Fgt

• Q set of states
• ? finite input alphabet
• ? Q x ? ? Q is a transition function
• q0 ? Q is the initial state
• F is the (nonempty) set of success states
• Example

49

• abna (ngt0)

50

• FSA corresponds to a tape being read
• from left to right after a transition, move
• to the symbol to the right on the tape

51
Turing Machines

• The head can move left as well as right (as in
  two-way finite automata, seeHopcroft Ullman
  1979)
• The head can correct the tape, replacing one
  symbol with another
• Therefore, ? Q x ? ? Q (in FSAs) becomes ?
  Q x ? ? Q x ? x ,- (in TMs)

52
Play with TMs. E.g.,

• Build TM that recognises exactly all palindromes
  in the alphabeth a,b.(See Harel 87 for a
  solution)

53

• Fiddling with TMs does not extend or diminish
  their generating power. E.g.,
• Multiple tapes (more)
• Nondeterminism (more)
• Multidimensionality (more)
• More than one head (more)
• At most three states (less)
• (E.g., Hopcroft Ullman 79, chapter
  7)

54
TM a very robust notion

• Cf. the speed of light given that nothing can go
  faster than light, there must be something
  important about it.

55
Another noncomputability resultPosts
Correspondence Problem (PCP)

• Another uncomputable problem
• Proven by reduction from Halting
• This reduction is less obvious
• Many other problems have been proven undecidable
  by reduction from PCP
• General statement of the problem (example in
  lecture notes)

56
Viewed as a puzzle

• Can pieces be joined in such a way that
• top row symbols bottom row symbols ?

57

• In this case yes (but its hard in general)
• oxx xx xx o xxx xxx

58
Posts Correspondence Problem (PCP)

• Given n pairs of finite strings p1 lts11,s12gt
  ,, pn ltsn1,sn2gt
• n size of problem
• Notation First(pi) si1, Second(pi) si2
• Does there exist a sequence of pairs (possibly
  with iterations) ltpi1,,pikgt, such
  thatFirst(pi1) First(pik)Second(pi1)
  Second(pik)

59
Expressing example in this way

• Strings from alphabeth o,x
• Size of problem2 p1lts11,s12gt, p2lts21,s22gt
  ltoxx,ogt, ltxx,xxxgt
• Solution ltp1,p2,p2gt, sinceoxxxxxxoxxxxxxox
  xxxxx
• Does it matter that repetitions are allowed?

60
Looking ahead to complexity

• If repetitions were not allowed then the problem
  would be finite and therefore decidable (for all
  sizes n).
• n pairs can be ordered in maximally n! ways,
  e.g., 3 pairs a,b,c can be ordered abc,
  acb bac, bca, cab, cba
• Worst case, there are n! orders to check

61
Posts Correspondence Problem (PCP)

• p1 lts11,s12gt ,, pn ltsn1,sn2gt ,
• Size of problem n
• Decidability
• Size2 is decidable
• Size7 is undecidable
• Sizes in between are unknown

62
Lessons

• Some reductions are quite unexpected
• Whether a problem is computable can be difficult
  to predict

63
Where this comes from, there is much more

• E.g., Rices Theorem
• Only syntactic properties of programs
• are computable
• (Read Harel, one of the two books)

64
Remember high uncomputability

• AlwaysHalt
• Little brain teaser Suppose we fix the program P
  and the input I. Does there exist an algorithm
  for determining whether Halt(P,I)?