Turing Machines

1
Turing Machines
Alan Turing (1912-1954) mathematician and
logician
2
Models of computing

• DFA and NFA - Regular languages
• Pushdown automata - Context-free
• Bounded Turing Ms - Context sensitive
• Turing machines - Unrestricted
• Each level of languages in the Chomsky hierarchy
  has a computing model associated with it. As the
  languages grow more complex, the computing models
  become more powerful.

3
Phrase structure grammars

• A phrase structure, or unrestricted grammar is a
  grammar whose productions are of the form
• ? ? ?
• where ? and ? are any strings over the alphabet
  of terminals and non-terminals, so basically any
  kind of production is allowed, even shortening
  rules.
• Phrase structure grammars contain within them all
  the other kinds of grammars and are potentially
  the most complex. Thus to recognise them, we
  need a powerful computational model.

4
Turing machines

• Turing machines (TMs) were introduced by Alan
  Turing in 1936
• They are more powerful than both finite automata
  or pushdown automata. In fact, they are as
  powerful as any computer we have ever built.
• The main improvement from PDAs is that they have
  infinite accessible memory (in the form of a
  tape) which can be read and written to.

5
Basic design of Turing machine
Infinite tape
Tape head
Control
Usual finite state control
The basic improvement from the previous automata
is the infinite tape, which can be read and
written to. The input data begins on the tape,
so no separate input is needed (e.g. DFA, PDA).
6
Comparing computing models

• Finite automata finite state control input
  string
• no memory
• Pushdown automata finite state control input
  string
• stack memory
• Turing machine finite state control
• input on tape
• infinite tape memory
• A TM is equivalent to a PDA with two stacks

7
Basic operation of a TM

• The TM is based on a person working on a long
  strip of paper (infinite in principle) divided up
  into cells, each of which contains a symbol from
  an alphabet.
• The person uses a pencil with an eraser.
  Starting at some cell, the person reads the
  symbol in the cell and decides either to leave it
  alone or to erase it and write a new symbol in
  its place.
• The person can then perform the same action on
  one of the adjacent cells. The computation
  continues in this manner, moving from one cell to
  the next along the paper in either direction.

8

TM instructions (transition functions)

• Each Turing machine instruction contains the
  following five parts
• The current machine state.
• A tape symbol read from the current tape cell.
• A tape symbol to write into the current tape
  cell.
• The next machine state.
• A direction for the tape head to move.
• Two inputs, three outputs T(i, a) (b, j, R)

9
Moving around the tape

• At each step there are three possible actions for
  the tape head.
• "move left one cell,"
• "stay at the current cell," and
• "move right one cell," respectively.  
• We'll represent these by the letters
• L, S, and R

10
Instruction shorthand

• We can represent a full TM instruction as
• ?i, a, b, L, j?
• This means
• If the current state of the machine is i,
• and if the symbol in the current tape cell is a,
  
• then write b into the current tape cell, move
  left one cell,
• and go to state j.

11
The instruction in graphical form

• This means the same as above
• If the current state of the machine is i, and if
  the symbol in the current tape cell is a, then
  write b into the current tape cell, move left one
  cell, and go to state j.

12
Putting the input on the tape

• An input string is represented on the tape by
  placing the letters of the string in adjacent
  tape cells. All other cells of the tape contain
  the blank symbol, which we'll denote by ? or ?.
• The tape head is usually put at the leftmost cell
  of the input string (unless specified otherwise).

13
Starting and stopping

• As before, there is one start state which must be
  specified.
• However in this case, there is usually only one
  halt state, which we denote by "Halt."

14
The Machine stops

• A Turing machine stops (halts) when it either
• 1. enters the Halt state or
• 2. when it enters a state for which there is no
  valid move.
• For example, if a Turing machine enters state i
  and reads a in the current cell, but there is no
  instruction of the form ?i,a,.?, then the
  machine stops in state i.

15
Instantaneous description

• To describe the TM at any given time we need to
  know three things
• 1) What is on the tape?
• 2) Where is the tape head?
• 3) What state is the control in?
• Well represent this information as follows
• State i ? a a b a b ?
• Here, the ? symbol represents an empty cell
  (repeated indefinitely to left and right) and the
  position of the tape head is underlined.

16
Turing machine problems

• In the finite automata and pushdown automata
  problems we concentrated on accepting and
  rejecting strings of a grammar
• Turing machines can do this as well, but most of
  the types of problems will be doing simple tasks,
  generally performing transformations on the
  string on a tape. For example, mathematical
  operations, etc.

17
Simple Turing machine

• EXAMPLE Adding 2 to a Natural Number
• Lets represent natural numbers in unary form.
  (e.g. 3 111)
• We will represent 0 by the empty string ?.
• Plan
• Move to the left of the first 1.
• Change that empty cell to a 1.
• Move left and repeat.
• Halt.

18
There are just three instructions

• Rule 1 ?0, 1, 1, L, 0?

Move left to blank cell.
?0, ?, 1, L, 1?
Rule 2
Add 1 and move left.
Rule 3
?1, ?, 1, S, Halt?
Add 1 and halt.
19
Graphically
Instantaneous description State 0 ? 1 1
1 ? Begin in state 0 State 0 ? 1 1 1
? Rule 1 State 1 ? 1 1 1 1 ? Rule 2 Halt
? 1 1 1 1 1 ? Rule 3
20
Example

• Adding 1 to a Binary Natural Number
• Binary numbers
• 1 1
• 2 10
• 3 11
• 4 100
• 5 101
• 6 110

21
The algorithm is

• 1. Move to the right end of string
• 2. Repeat
• If current cell contains 1, write 0 and move
  left
• until current cell contains 0 or ?
• 3. Write a 1
• 4. Move to left end of string and halt.
• WHY IS THIS THE ALGORITHM?

22
The TM instructions

• ?0, 0, 0, R, O? Scan right.
• ?0, 1, 1, R, 0? Scan right.
• ?0, ?, ?, L, 1? Found right end of string.
• ?1, 1, 0, L, 1? Write 0 and move left with
  carry bit.
• ?1, 0, 1, L, 2? Write 1, done adding.
• ?1, ?, 1, S, Halt? Write 1, done and in
  proper position.

23
The TM instructions (continued)

• Otherwise move to left most character in the
  string
• ?2, 0, 0, L, 2? Scan left.
• ?2, 1, 1, L, 2? Scan left.
• ?2, ?, ?, R, Halt? Reached left end of
  string, and halt.

24
Instantaneous description 3 1 4

• State 0 ?? 1 1 ?
• State 0 ?? 1 1 ?
• State 0 ?? 1 1 ?
• State 1 ?? 1 1 ?
• State 1 ?? 1 0 ?
• State 1 ?? 0 0 ?
• Halt ? 1 0 0 ?

25
Example equality test

• Check to see if two unary numbers, separated by a
  sign, are equal.
• For example, 3 and 2 would be written initially
  on the tape as ? 1 1 1 1 1 ?
• If the numbers are equal, end with a ? in the
  current cell, otherwise end with 1.
• Basic plan
• Repeatedly cross off the leftmost and rightmost
  1s until none remain.

26
Machine instructions

• ?0, 1, ?, R, 1? Cancel leftmost 1.
• ?0, ?, ?, R, 4? Left number is 0.
• ?0, , , R, 4? Finished with left number.
• ?1, 1, 1, R, 1? Scan right.
• ?1, , , R, 1? Scan right.
• ?1, ?, ?, L, 2? Found right end.
• ?2, 1, ?, L, 3? Cancel rightmost 1.
• ?2, , 1, S, Halt? Not equal, first gt second.

27
Machine instructions

• ?3, 1, 1, L, 3? Scan left.
• ?3, , , L, 3? Scan left.
• ?3, ?, ?, R, 0? Finished with left number.
• ?4, 1, 1, S, Halt? Not equal, first lt second.
• ?4, , , R, 4? Scan right.
• ?4, ?, ?, S, Halt? Numbers are equal.

28
Sample ID (skipping a few steps)

• State 0 ? 1 1 1 1 ? Starting point 2 2?
• State 1 ? 1 1 1 ? Cancel 1st 1.
• State 2 ? 1 1 1 ? Scan to right end.
• State 3 ? 1 1 ? Cancel last 1.
• State 0 ? 1 1 ? Scan to left end.
• State 1 ? 1 ? Cancel 1st 1.
• State 2 ? 1 ? Scan to right end.
• State 3 ? ? Cancel last 1.
• State 0 ? ? Scan to left end.
• State 4 ? ? Left number empty.
• Halt ? ? Right number also
  empty, so equal!

29
Non-deterministic Turing machines

• We can have non-deterministic Turing machines as
  well, just as in the other automata weve
  considered.
• Deterministic Turing machines are just as
  powerful as non-deterministic ones, and so they
  accept the same languages, those of the phrase
  structure grammar.
• In addition, there are many other similar
  machines which can be seen to be equivalent to
  the simple Turing machine.

30
Turing machine variations
Two state pushdown automaton
Input
Control

• The right half of the tape is kept on one stack,
  and the left half on the other. As we move
  along, we pop characters off one and push them
  onto the other.

31
Turing machine variations
Semi-infinite tape Turing machine
Tape infinite in only one direction
Tape head
Control

• We can emulate the standard TM by splitting the
  cells into two groups (every other one), with one
  group representing the left half of infinite tape
  and the other representing the right half.

32
Turing machine variations
Multi-track Turing machine
Tape head reads many tracks at the same time
Tape head
Control

This is emulated in standard TM just by grouping
the cells together, so that each tape read
includes multiple characters
33
Turing machine variations
Multi-tape Turing machine
Many tapes with tape heads moving independently
Tape 1
Tape 2
Control

• We can emulate this with a multi-track machine
  with twice as many tracks as tapes. One track
  for each tape and another to remember position of
  the tape head on each tape.

34
Turing machine variations
Multi-head Turing machine
One tape with many tape heads moving independently
Control

• As with the multiple tape machine, use a
  multi-track machine, where one track remembers
  the tape contents and the other tracks keep track
  of each tape head position.

35
Turing machine variations

• Off-line Turing machine a two tape TM where one
  tape is a read-only version of the input
• Multi-dimensional tape where the tape may
  extend into two or more dimensions
• Also, we can usually trade the number of tape
  symbols for the number of states, for example
• Two symbols but many states
• Two states but many symbols

36
More complex applications

• Understanding the variations is useful, because
  many jobs are easier to do the with more
  complicated machines.
• For example, multiplying two numbers to get a
  third would be difficult to do with a simple
  Turing machine, but is fairly straight forward
  with a three tape machine.
• Plan
• If either number is zero, the product is zero
• Otherwise, use the first number as a counter to
  repeatedly add the second number to itself

37
Multiplying two unary numbers
38
Machine instructions

• Are either of the numbers zero?
• ?0, ?, ?, ?,?, ?, ?,S, S, S, Halt? Both
• ?0, ?, 1, ?,?, 1, ?,S, S, S, Halt? 1st 0
• ?0, 1, ?, ?,1, ?, ?,S, S, S, Halt? 2nd 0
• ?0, 1, 1, ?,1, 1, ?,S, S, S, 1? Neither
• Add the number from the 2nd tape to 3rd tape
• ?1, 1, 1, ?,1, 1, 1,S, R, R, 1? Copy
• ?1, 1, ?, ?,1, ?, ?,S, L, S, 2? Done

39
Machine instructions (continued)

• Move head to beginning of 2nd tape,
• and move head of 1st tape once to the right
• ?2, 1, 1, ?,1, 1, ?,S, L, S, 2? go left
• ?2, 1, ?, ?,1, ?, ?,R, R, S, 3? both right
• Check to see if 1st tape is exhausted
• ?3, 1, 1, ?,1, 1, 1,S, S, S, 1? Repeat
• ?3, ?, 1, ?,?, 1, ?,S, S, L, Halt? Job
  done!

40
Linear bounded automaton

• All the previous machines we considered were
  equivalent in computing power, however if we
  restrict the amount of tape, then the machine
  becomes less powerful
• A linear bounded automaton (LBA) is a Turing
  machine where you cannot use any more tape than
  the size of the initial input.
• LBAs recognize exactly the family of
  context-sensitive languages! This is a subset of
  the phrase structure grammars which Turing
  machines recognize.

41
Why do we care about Turing Machines?

• Universality!

Universality a universal computing machine can
do any computation that can be done by any other
machine.
42
Programmable Turing machines

• Up until now, we constructed specific Turing
  machines to do specific things.
• For example
• Adding 2 to a natural number
• Adding 1 to a binary number
• But it is possible to do more

43
The Universal Turing Machine

• We can build a Turing Machine that can do the job
  of any other Turing Machine. That is, it can tell
  you what the output from the second TM would be
  for any input!
• This is known as a universal Turing machine
  (UTM).
• A UTM acts as a completely general-purpose
  computer, and basically it stores a program and
  data on the tape and then executes the program.

44
A three tape Turing machine

• We can imaging the UTN as a three tape TM, where
  we have a separate tape to keep track of
• 1) the program
• 2) the state
• 3) the input
• Is there anything that a Turing machine cannot
  do?

45
The Church-Turing thesis

• Anything which is intuitively computable can be
  computed by a Turing machine.
• What do we mean by intuitively computable ?
• It means that we can describe a procedure which
  will solve the problem.
• Its a thesis, or a conjecture, not a theorem!

46
Computational Equivalence

• No one has ever invented a more powerful
  computing model than a Turing machine!
• A number have been invented which are equivalent,
  in the sense that they solve the same class of
  problems (?-calculus, etc.)
• However, we believe that there isnt a more
  powerful computing model!

47
Decidability

• While we believe that Turing machines can compute
  anything which is computable in principle, they
  cannot do everything!
• One example is the halting problem, which has to
  do with whether a Turing machine will stop for a
  given input or will keep computing forever.
• For example, we cannot make a Turing machine that
  will be able to decide whether another Turing
  will halt on any given input.
• These issues will be discussed in detail next
  term in THCSB!