Sample Models of Computation

1
Chapter 7

• Sample Models of Computation

2
Introduction

• Everything we have been discussing is about
  algorithms the concepts, the correctness, the
  implementation, etc.
• We now discuss the other side of the story. We
  will show that there are some problems that no
  computer can solve.
• Models are very important to help our
  understanding of various objects and their
  behaviors. A model must capture the essence, and
  the important properties of the real thing, while
  suppressing the unnecessary details.
• A model could be either physical, like a model
  car, or mathematical (we can use an equation to
  model the driving distance of a vehicle).

3
Models of Computation

• There are problems that do not have any
  algorithmic solution
• (not the same as saying that there are problems
  for which no algorithmic solution has yet been
  found, but for which such a solution may exist,
  if we were sufficiently clever to discover it)
• April 1984, TIME magazine
• Put the right kind of software into a
  computer, and it will do whatever you want it to.
  There may be limits on what you can do with the
  machines themselves, but there are no limits on
  what you can do with software.

4
Models of Computation cont,

• The algorithmic problems we wish to discuss are
  such that no amount of money, time or brains will
  suffice to yield solutions.
• We still require and/or can give
• termination for each legal input even allowing
  any amount of time (algorithm can take as long as
  it wishes on each input, but it must eventually
  stop and produce desired output)
• algorithm can be given any amount of memory it
  asks for

5
Model of Computing Agent

• Central Features
• a computing agent must be able to follow the
  instructions in an algorithm
• notwithstanding the format of the instructions, a
  computing agent must be able to read them, decode
  them, ask for and use pertinent data
• a computing agent must be able to act according
  to instructions
• a computing agent carries out an algorithm
  reacting both to input and to the present
  state
• a computing agent is expected to produce output
  the outcome of an algorithm must be an observable
  result

6
Finite State Machine

• Simple processes can be modelled by a Finite
  State Machine
• Components

A state, a stage in the processing
A transition between two states. Controlled by
the a particular event, or input character
The starting state
The ending state
7
Finite State Machine

• Simple FSM for a turnstyle
• Gate is opened by token (no need to worry about
  exact amounts/change)
• Passing through the gate locks it for the next
  person
• Note that there is no start state and no stop
  (halt) state for this FSM

8
Finite State Machine

• A simple FSM for a vending machine.
• Machine accepts nickels, dimes and quarters only
• Exact amount (35c in this case) must be tendered

9
Defining Syntax

• Syntax rules for constructing, e.g., programs
  in a particular language
• Most programming language syntax can be defined
  as a Context Free Grammer (CFG)
• Various ways of representing syntax
• Easiest, most common is BNF (Backus-Naur Form)
• A very useful skill for any computer technologist
  to acquire

10
BNF The Basics

• A CFG definition consists of
• Set of Terminal Symbols elementary symbols
  permitted in the language, keywords, constants,
  etc.
• Set of Nonterminals linguistic constructs
• A set of Productions
• Left side consisting of a single nonterminal
• Right side consisting of a sequence of zero or
  more terminals or nonterminals
• A single Starting Nonterminal representing the
  main construct of the language

11
BNF The Basics

• Notation
• Non-terminals written inside angle-brackets lt and
  gt
• Exclusive or written as
• Production rules written as
• LHS RHS
• Empty string written as ltemptygt
• Example A School Week
• ltWeekgt ltDaygt ltWeekgt ltemptygt
• ltDaygt ltSessiongt ltDaygt
• ltSessiongt ltClassroomgt ltLabgt ltFreegt

12
BNF The Basics

• Extended BNF
• Non-terminals written as words, no lt or gt
• Exclusive or written as
• Production rules written as
• LHS RHS
• ( ) used for grouping
• ... a sequence of 0 or more occurrences of
  the construct
• ... an optional construct
• The School Week again
• Week Day
• Day Session
• Session Class Lab Free

13
Expressions

• Arithmetic expressions e.g. 2 3 5 occur
  in virtually all programming languages
• A BNF description of expressions
• ltexprgt ltexprgt lttermgt ltexprgt -
  lttermgt lttermgt
• lttermgt lttermgt ltfactgt lttermgt / ltfactgt
  ltfactgt
• ltfactgt ltnumbergt ( ltexprgt )
• The BNF captures the precedence of multiplication
  and division over addition and subtraction

14
Parse Trees

• A Parse Tree is created by a compiler using the
  syntax definition
• It can be used to generate code or to evaluate
  expressions
• 2 3 5
• ltexprgt
• / \
• / \
• ltexprgt lttermgt
• / \
• lttermgt lttermgt ltfactgt
• ltfactgt ltfactgt ltnumbergt
• ltnumbergt ltnumbergt
• 2 3 5

15
Turing Machine

• The Turing machine is the primary model for a
  computing agent.
• It consists of
• a tape extending infinitely in both directions
  that is divided into cells - each cell being able
  to hold one symbol
• a read/write head that can read and write the
  contents on one cell. The read/write head can
  move one cell to the left or one cell to the
  right.
• a set of internal states

16
Turing Machine

• It should be pointed out that a Turing
  machine is a "finite state" device
• Has a finite number of cells on the tape that are
  not blank,
• it can only read and write a finite set of
  symbols on the tape
• it can only assume one of a finite number of
  internal states.

17
Turing Machine Instructions

• Usually denoted by a 5-tuple
• (current state, read symbol, next state, write
  symbol, direction of move)
• e.g. (1, 0, 1, 2, R)
• NOTE Output and next state depend on current
  input and the present state
• Some versions allow either a move or a write
  symbol, but there is no loss of generality either
  way.

18
Turning Machine

• Is a TM an appropriate model for a computing
  agent?
• Can accept input
• Can store information in and retrieve it from
  memory
• Can take actions according to algorithm
  instructions action taken depends on present
  state and input item being processed
• Can produce output

19
Other considerations

• What if there is more than one instruction that
  applies to the current configuration of (current
  state, current symbol)?
• What if there is no instruction that applies to
  the current configuration of (current state,
  current symbol)?
• What is the initial configuration?
• By convention, a turing machine
• starts at leftmost non-blank character on the
  tape, in state 1
• halts in state 2 (put another way, state 2 is the
  halt state)
• cannot move left from start without crashing

20
Algorithm vs. TM

• An algorithm is a collection of instructions for
  a computing agent to follow.
• If a Turing machine is a good model of a
  computing agent, then instructions for a Turing
  machine can be a model for an algorithm.
• Properties of an Algorithm
• be a well-ordered collection
• consist of unambiguous and effectively computable
  operations
• halt in a finite amount of time
• produce a result / output

21
An Example

• Assume we have the following instructions (H
  stands for the halt state)
• (1, _, 1, _, R)
• (1, 1, 3, 0, R)
• (1, 0, 3, 1, R)
• (3, 0, 3, 1, R)
• (3, 1, 3, 0, R)
• (3, _, H, _, L)
• And the following tape (input) to begin with

22
Example cont.

• We will have the following execution

23
Example continued

• The previous TM accepts any binary string as an
  input and produces the complement of the string
  as the output, when it halts.
• This qualifies as a model for computing agents.
• TM is actually more powerful than a real computer
  in the sense that there is no limit to its memory
  size.

24
A FSM Diagram

• Is a visual representation of a TM algorithm
• Consists of
• Circles indicating states
• Arrows representing the transitions
• From every state there must have at least 2
  labels for each edges one for the input and one
  for the output
• An optional direction, may be applied to each edge

25
Example Unary Addition

• Alphabet 0, 1 (regular grammar)
• Construct the state transition diagram for a
  unary addition machine.
• Two numbers are represented as strings of 1s with
  (any number of) blanks in between and an
  afterwards (this is to make the machine simpler,
  it is not necessary)
• input 1 1 1 1 1 b b b 1 1 1
• output 1 1 1 1 1 1 1 1

26
Unary Addition Machine
27
Example Binary Incrementing

• Alphabet 0, 1 (regular grammar)
• Construct the state transition diagram for a
  binary incrementing (adds 1) machine.
• Number is represented as strings of 1s with a
  space marking the end
• input 1 1 1 1 1 0 0 0 output 1 1 1 1 1 0 0
  1
• input 1 1 1 1 1 0 0 1 output 1 1 1 1 1 0 1
  0
• The key to working out this TM is the realisation
  that the number ends with either a series (1 or
  more) or zeros or a series of 1s.
• Ends with 0 simply replace the 0 with a 1
• Ends with 1 work left, replacing 1 with 0 until
  a 1 is found, replace that 1 with a 0

28
Binary Incrementer FSM
29
TM instructions

• A Turing Machine does not always halt
• Basically any input on the tape gives one of
  three situations
• The TM accepts the input (halts)
• The TM rejects the input (stops in a non-halt
  state)
• The TM does not stop, it loops
• The unary add machine we just saw will loop if we
  omit the from the input

30
Church-Turing Thesis

• If there is an algorithm to do a symbol
  manipulation task, then there is a Turing machine
  to do that task.
• This implies that Turing machines define the
  limits of computability
• Thus if we can find a symbol manipulation problem
  for which we can prove that no Turing machine
  exists to solve it, then no algorithm exists, and
  thus the problem is uncomputable.

31
The Halting Problem

• Given any collection of Turing machine
  instructions together with any initial tape
  contents, whether that Turing machine will never
  halt if started on that tape
• This is an uncomputable problem no Turing
  machine exists to solve this problem

32
Example Algorithm A

• while x ! 1
• x x 2
• legal input set of positive integers (1,2,)
• Algorithm A will halt for ODD inputs, and will
  run forever for EVEN numbers
• Trivial to decide!

33
Example Algorithm B

• while x ! 1
• if x 2 0
• x x/2
• else
• x 3x1

• Algorithm B has been tested on many positive
  integers and has always terminated.
• No one has been able to prove that it will
  always terminate.

Try x 7 sequence of x is 7,22,11,34,17,52,26,13
,40,20,10,5,16,8,4,2,1
34
Unsolvability of the Halting Problem

• Assume that P is a TM that takes as input the
  encoding of T of a TM T and an input t for that
  TM and that solves the halting problem. In
  particular, P always halts and its outputs are

35
Unsolvability of the Halting Problem

• Use P to build a TM Q that takes as input the
  encoding T of a TM T and input t for that TM and
  that halts exactly when P outputs 0. The result
  of executing Q is

36
Unsolvability of the Halting Problem

• Finally, use Q to build a TM S that takes as
  input the encoding T of a TM T, makes a copy of
  T and feeds (T,T) to Q. The result of running
  S is

37
Unsolvability of the Halting Problem

• What happens when S is given S as input?

This table embodies a contradiction. We must
conclude that P does not exist and that the
Halting Problem is unsolvable.
38
Other Unsolvable Problems

• Given a TM T, determine whether it ever halts on
  any input.
• Given two TM, T1 and T2, determine whether they
  solve he same problem.
• Given a formal logical statement about algebra,
  determine whether it is a theorem or not.
• A variety of puzzles that involve tiling the
  plane.

39
Sphere of algorithmic problems
Problems admitting no algorithms at all
Problems admitting reasonable (polynomial-time)
algorithms
Problems admitting no reasonable algorithm
40
What are the consequences?

• A computer scientists, we must recognize that
  there are many things we cannot do.
• No program in any language can be written to
  decide whether any given program always stops
  eventually no matter what the input
• No program can be written to decide whether any
  two programs are equivalent (i.e. will produce
  the same output for all inputs)
• No program can be written to decide whether any
  given program on any given input will ever
  produce some specific output
• (Last case equivalent to stating that it is
  impossible to write a general tester program.)
• Finally, note that it may be the generality of
  such tasks which is making life difficult

41
Research on undecidable Problems

• the pessimistic part of algorithm design and
  analysis
• important to remember that when trying to solve
  an algorithmic problem there is always the chance
  that it might not be solvable at all, or it might
  not admit any practically acceptable solution
• IN principle intractable and tractable problems
  appear to have a solution
• IN practice only tractable problems appear to
  have a solution

42
NOTE

• Only in a limited number of cases can we
  construct an equivalent FSM for a given TM.
• We can only if the TM
• Does not rewrite characters on the tape
• Moves in only one direction