Universal Computation

1
Universal Computation
2
Universal Computation
Universal Computation One form of (abstract)
computing which can emulate any other computing
method.
Universal Computation is not obvious If it
should turn out that the basic logics of a
machine designed for the numerical solution of
differential equations coincide with the logics
of a machine intended to make bills for a
department store, I would regard this as the most
amazing coincidence I have even
encountered. -Howard Aiken, Harvard University
(1956)
3
Universal computation
The universal computing machine It is possible to
invent a single machine which can be used to
compute any computable sequence. -Alan Turing
(1936)
There will be positively no internal alterations
to be made even if we wish suddenly to switch
from calculating the energy levels of the neon
atom to the enumeration of groups of order
720. -Alan Turing (1945)
Let us now return to the analogy of the
theoretical computing machines It can be shown
that a single special machine of that type can be
made to do the work of all. It could in fact be
made to work as a model of any other machine. The
special machine may be called the universal
machine. -Alan Turing (1947)
We do not need to have an infinity of different
machines doing different jobs... the engineering
problem of producing various machines for various
jobs is replaced by the office work of
programming the universal machine to do these
jobs. -Alan Turing (1948)
4
A note on practicality

• When he discovered universal computation in
  1936,Turing was not
• solving differential equations, or
• doing department-store billing
• His goal solve a basic question in logic of
  mathematics
• This forced an approach that was
• as general as possible
• as simple as possible
• and permitted an approach that was
• conceptual rather than physical
• in fact completely impractical
• Nevertheless. . .

5
Background Foundations of Mathematics

• Frege Russell (1879, 1913)
• invention of formal logic for mathematical
  reasoning
• new logical foundations for mathematics
• Hilbert (1928)
• is mathematics logically complete?
• is mathematics logically consistent?
• is mathematics logically decidable?
• Gödel (1931)
• mathematical logic is incomplete
• consistency of such a system cant be proved
  within itself
• Turing (1936)
• mathematical logic is undecidable
• there is no procedure for determining whether a
  proposition is provable

6
Turings Universal Machine

• Infinite tape
• one-dimensional sequence of squares
• each square may bear a symbol or be blank
• there are a finite number of discrete symbols
• Finite control
• machine has finitely many control states(what
  Turing calls conditions or m-configurations)
• machine sees or scans one square of the tape at
  any step
• a given control state specifies what to do for
  each possible scanned symbol
• possible actions
• write a symbol in the scanned square or erase the
  square
• move the tape one square left or one square right
• or halt

7
Function of the Universal Machine

• The tape provides
• input information (before the machine starts)
• output information (after the machines stops if
  it does)
• a scratch pad for working storage
• The set of control states is the program
• Each control state is a tuple, i.e.
• control-state
• scanned-symbol
• written-symbol
• tape-motion
• next-state
• order of tuples doesnt affect the program

8
Example Code

• old old new new
• state char state char direction
• 1 _ 1 _ gt
• 1 1 1 1 gt
• 1 - 1 - gt
• 1 2 _ lt
• 2 1 3 lt
• 2 - H _ lt
• 3 1 3 1 lt
• 3 - 4 - lt
• 4 _ 4 _ lt
• 4 1 1 _ gt

9
How it works

• TM java applet by Suzanne Britton

10
Universal Machine Variations

• Is the tape infinite in both directions or just
  rightwards?
• One tape or multiple tapes?
• One or many control units?
• Can the machine both write a symbol move the
  tape in one step?
• Replace the tape with a two-dimensional grid?
• and so on...
• None of this matters
• any Universal Machine can simulate any other
  design...

11
Why is the T(uring) M(achine) Universal?

• We can write a TM program that runs any TM
  program!Given as input a symbolic description of
  a TM,combined with a specified input, it
  simulates the operation of the specified machine
  and produces the output that the specified
  machine would have produced for the specified
  input.
• We can do the same thing for arbitrary variant
  designs of automatic machines each can be
  effectively simulated by the Universal Machine
• Therefore, the Universal Machine can compute
  anything that can be computed!

But dont forget the mathematicians always have
some fine print
12
How Turing used his machine

• Some TMs with some inputs eventually halt,but
  some TMs with some inputs will never halt
• The halting set of a TM the set of inputs for
  which it eventually halts
• Define a set D of natural numbers that is not the
  halting set of any TM
• Method
• express TM programs as numbers
• crudely too few programs vs. too many sets of
  numbers
• Problem find an algorithm to determine whether a
  given natural number belongs to set D
• if such an algorithm existed, we could turn it
  into a TM whose halting set was D
• this is a contradiction
• therefore no such algorithm exists
• Thus there are logical propositions that are not
  decidable

13
A historical noteComputational logic at war
(Debate in Wittgensteins Foundations of
mathematics course, Cambridge University,
spring 1939)
14
Turing v. Wittgenstein
Wittgenstein . . . The question is Why are
people afraid of contradictions? . . . Turing
says, Because something may go wrong with
the application. But nothing need go wrong. And
if something does go wrong if the bridge
breaks down then your mistake was of the kind
of using a wrong natural law. Turing You cannot
be confident of applying your calculus until you
know that there is no hidden contradiction in
it. Wittgenstein There seems to me to be an
enormous mistake there. . . Suppose I convince
Rhees of the paradox of the Liar, and he
says, I lie, therefore I do not lie, therefore
I lie and I do not lie, therefore we have a
contradiction, therefore 2x2269. Well, we
should not call this multiplication, that is all
. . . Turing Although you do not know that the
bridge will fall if there are no contradictions,
yet it is almost certain that if there are
contradictions it will go wrong
somewhere. Wittgenstein But nothing has ever
gone wrong that way yet . . .
15
Some types of problems

• Ill-defined problemse.g. what is truth?
• Undecidable problems
• e.g. will an arbitrary Turing Machine program
  ever halt?
• Decidable problems (classified by complexity)
• polynomial
• exponential
• non-deterministic polynomial

16
Polynomial vs. exponential complexity (one
algorithm step 1 microsecond)
(Figure 1.2 from Garvey Johnson 1979)
17
Non-deterministic algorithms

• Non-determinism in TMs and similar automata
• one next state... or more?
• guessing and backtracking
• determinization eliminating non-determinism
• Complexity of non-deterministic algorithms
• interesting class of problems
• guessing the answer seems hard (exponential or
  worse)
• checking a guess is easy (polynomial)
• called non-deterministic polynomial problems
• abbreviated NP
• open question P NP?
• i.e. is guessing really, truly hard for these
  problems?

18
On beyond Turing...

• Plenty of pre-Turing specialized computer designs
• Samples from the post-Turing bestiary
• Genetic algorithms / evolutionary computing
• Social insects, markets etc.
• DNA computing / biomolecular computing
• Neural nets / connectionism / PDP
• Quantum computing
• Themes
• Parallelism / interconnection
• Gradient vs. discrete values
• Chance

19
Collective Intelligence in Social Insects
Self-organization was originally introduced in
the context of physics and chemistry to describe
how microscopic processes give rise to
macroscopic structures in out-of-equilibrium
systems. Recent research that extends this
concept to ethology, suggests that it provides a
concise description of a wide rage of collective
phenomena in animals, especially in social
insects. This description does not rely on
individual complexity to account for complex
spatiotemporal features which emerge at the
colony level, but rather assumes that
interactions among simple individuals can produce
highly structured collective behaviors. E.
Bonabeau et al., Self-Organization in Social
Insects, 1997
20
Collective decision-making among ants
Percentage of Iridomyrex Humulis workers passing
each (equal) arm of bridge per 3-minute period
21
Finding the shortest path
(from Dorigo et al. 1997)
22
More complex emergent structure termite mounds
23
How Termites Work
Bruinsma (1979) positive feedback mechanisms,
involving responses to a short-lived pheromone in
deposited soil pellets, a long-lived pheromone
along travel paths, and a general tendency to
orient pellet deposition to spatial
heterogeneities these lead to the construction
of pillars and roofed lamellae around the
queen. Deneubourg (1977) a simple model with
parameters for the random walk of the termites
and the diffusion and attractivity of the pellet
pheronome, producing a regular array of
pillars. Bonabeau et al. (1997) air convection,
pheromone trails along walkways, and pheromones
emitted by the queen "under certain conditions,
pillars are transformed into walls or galleries
or chambers", with different outcomes depending
not on changes in behavioral dispositions but on
environmental changes caused by previous
building. Thus "nest complexity can result from
the unfolding of a morphogenetic process that
progressively generates a diversity of
history-dependent structures." Similar to
models of embryological morphogenesis.
24
Computers and Brains

• Computers
• fast circuits (109 state changes per second)
• small number of interconnected processors (1-100)
• digital flow of information
• determinate, exactly repeatable behavior
• designed and replicated exactly
• small flaw or damage is catastrophic
• mostly programmed
• good at arithmetic, logic bad at pattern
  recognition, abduction
• Brains
• slow circuits (103 state changes per second)
• large number of interconnected processors (1010)
• quasi analog flow of information
• stochastic behavior
• grow organically with lots of case-to-case
  variation
• small flaw or damage usually irrelevant
• some programming (by evolution), some learning
  (from individual experience)
• bad at arithmetic, logic good at pattern
  recognition, abduction

25
Artificial Neural Networks

• Many interconnected neurons
• Each takes inputs from others
• And computes and output
• Each learns based on errors made by its
  neighbors
• Gives Parallel Distributed Processing (PDP)
• The appeal of neural net models a family of
  mathematical models that share many of the
  differential properties of brains vs. computers

26
Some questions to ponder

• Can a Turing Machine simulate a neural net?
• If so, what is the relative size (tape length
  vs. number of units and connections)?
• If not, what can a neural net do that a Turing
  Machine cant?
• Can a neural net simulate a Turing Machine?
• If so, how?
• If not, whats missing?
• Do these questions matter? Why? Why not?