Lecture 18. Unsolvability

1
Lecture 18. Unsolvability

Kurt Gödel

• Before the 1930s, mathematics was not like
  today.
• Then people believed that everything true must
  be provable. (More formally, in a powerful
  enough mathematical system, a true statement
  should be provable as a theorem.)
• Or put in todays and computational language
  given a computational problem, we should always
  be able to find a solution to it, in finite
  amount time, using an expressive programming
  language say Java.
• All these had changed because of one man Kurt
  Gödel, who has made immense impact upon
  scientific and philosophical thinking in the 20th
  century, and today.

2
You are not that far away from the great man
Kurt Gödel and Albert Einstein
postdoc
Anil Nerode
Me
You
3
Solvability and Decidability

• Up to now we have been discussing problems that
  are solvable on a computer, and for each problem
  we have been trying to find efficient algorithms.
  Here "solvable" means that there exists a finite,
  deterministic program given access to unlimited
  resources in terms of time and space, that will,
  after a finite time, halt and output the correct
  solution for the problem.
• If, further, the computational problem has a
  "yes/no" answer, we call it a decision problem
  (If such a problem is solvable, we call them
  decidable. Decision problems are particularly
  easy to study and analyze. Usually it is possible
  to take a computational problem and turn it into
  a related decision problem, and vice versa. One
  way of doing this is, for example, to add a new
  parameter i and say "Is the i'th bit of the
  answer 1 or 0?" By repeatedly asking decision
  problems of this sort, we could solve a
  computational problem that has a string or number
  solution, instead of just true/false.

4
Is every computational problem solvable?

• In 1900, David Hilbert gave a famous address at
  the International Congress of Mathematicians in
  which he listed 25 important problems for the
  next century. His 10th problem concerned the
  solution of Diophantine equations. He asked
  given a multivariate polynomial equation with
  integer coefficients, such as
• 3x2y 17 xz - 5 xy3 - 4 0.
• Can you write a computer program (process,
  in Hilberts original words) to find solution?
• You might think you can just do exhaustive search
  since we do not care about time complexity.
  However, then if no solution exists, your program
  will run forever.

5
Russells paradox

• Frege and Russell Gottlob Frege was a
  philosopher of mathematics who wanted to
  axiomatize mathematics (1848-1925) he wanted to
  write down a system of axioms from which all
  mathematical truths would follow. Among other
  things, he was responsible for formalizing the
  ideas of the existential and universal
  quantifiers that we still consider essential
  today. He formulated his ideas in a book called
  Grundgesetze der Arithmetik (Basic Laws of
  Arithmetic), but just before the 1903 edition was
  to go to press, he got a very surprising and
  disturbing letter from Bertrand Russell. Russell
  wrote to Frege to show that some sentences which
  could be constructed according to Frege's rules
  did not result in a consistent mathematical
  object. Today this is known as Russell's paradox.
  
• To illustrate the idea of Russell's paradox,
  consider another similar paradox mentioned by
  Russell (but invented by an unknown acquaintance
  of him). There is a certain town with a barber.
  This barber cuts the hair of every person in the
  town who does not cut their own hair. Who cuts
  the barber's hair?
• If the barber cuts his own hair, then according
  to the rule, he doesn't cut his own hair. And if
  he doesn't cut his own hair, then by the rule, he
  does. So we get a contradiction. The only
  reasonable conclusion is that there cannot be
  such a barber.

6
Another paradox

• Shortest string that cannot be described in less
  than thirteen English words.
• Does this string exist?
• If it does, then we have just described it above
  in 12 English words, contradiction.
• We will use this to prove that we cannot find
  such a string.

7
Lets explain

• Let's think about sets. Sets contain members but
  could we have a set that contained itself as a
  member? A priori, nothing rules this out we
  could, for example let
• S the set of all mathematical concepts
• Since S is itself a mathematical concept, S is a
  member of S.
• Another example is
• S the set of all ideas expressible in
  less than 12 words
• Since S is expressible in less than 12 words, S
  is a member of S. Frege's axiomatization of
  mathematics did not rule out expressions such as
  S S ? S . But even worse, it did not rule out
  expressions such as T S S ? S . Such an
  expression would denote a valid mathematical
  object, according to Frege. But as Russell wrote
  him, this introduces a genuine paradox that is
  impossible to resolve within Frege's system. The
  problem comes when we try to decide if T ? T. If
  T ? T, then T ? T. And if T ? T, then T ? T. This
  contradiction so disturbed Frege that he proposed
  a change to his system that would have created a
  mathematical universe with only one object.

8
Randomness (Solomonoff, Kolmogorov, Chatin), 30
years later in the 1960s

• Remember we used incompressibility to analyze
  average case complexity of algorithms.
• We defined
• C(x) length of the shortest description
  (program) of x,
• and this is invariant w.r.t. description
  language / Turing machine.
• If C(x) x, i.e. x is not compressible, then
  we say x is random.
• It turns out that this randomness
• Provides another foundation of mathematics (for
  probability theory)
• provides methods for us to analyze algorithms
• Provides a concrete statement for Godels theorem
  (Godels construction was Theorem This theorem
  is not provable).

9
Godels Theorem in a simple form
Theorem. The statement x is random is not
provable. Proof (G. Chatin). Let F be an
axiomatic theory. C(F) C. If the theorem is
false and statement x is random is provable in
F, then we can enumerate all proofs in F to find
a proof of x is random and x gtgt C, output
(first) such x. Then C(x) lt C O(1) But the proof
for x is random implies that C(x) x gtgt C.
Contradiction. QED
10
Undecidability

• Corollary. L x x is random is not
  decidable.
• Proof. If one has a program that, with input x,
  outputs yes iff x is random, then this
  provides a mathematical proof (treat the
  programming language as a mathematical system,
  inference rules) for x being random,
  contradiction.
  QED
• Oh, by the way, Hilberts 10th problem was also
  proved to be undecidable.

11
Turing, in 1936Everyone who taps a keyboard is
working on an incarnation of a Turing machine
Time Magazine

• In 1928, David Hilbert posed three questions
• Is mathematics complete
• Is mathematics consistent
• Is mathematics decidable
• Kurt Gödel answered no to the first two questions
  in his famous 1931 paper.
• But for the third, it was actually unclear what
  do we mean by decidable by what?
• To make the idea of a program more rigorous,
  Turing developed the notion of "Turing machine",
  an abstract computational model that, he argued,
  could do anything that a human computer could do.
  He also invented universal Turing machine.

12
Turings uncomputability proof

• Theorem. Halting problem is not decidable.
• Proof. We prove by contradiction assume such a
  program H exists
• H(P,x) 1 if P(x) halts, o.w. H(P,x) 0.
• Construct H s.t. H(P) halts iff H(P,P) 0.
• Thus, will H(H) halt?
• If it does, then H(H,H) 1, hence H(H) does
  not halt. Contradiction.
• If it does not, then H(H,H) 0, hence H(H)
  halts. Contradiction.