Unsolvability and Infeasibility

1
Unsolvability and Infeasibility

2
Computability (Solvable)

• A problem is computable if it is possible to
  write a computer program to solve it.
• Can all problems be computed?
• This question concerned mathematicians even
  before digital computers were developed. They
  looked for an algorithm (a finite set of
  instructions to carry out a task).

3
Turing

• Alan Turing developed the concept of a computing
  machine in the 1930s
• A Turing machine, as his model became known,
  consists of a control unit with a read/write head
  that can read and write symbols on an infinite
  tape.

4
Church-Turing Thesis

• Any function that can be computed can be computed
  by a simple Turing Machine. The Turing Machine
  is as powerful as any algorithm.
• Cannot prove thesis

5
Is there an unsolvable problem?

• YES
• A proof that there is a problem for which there
  is no algorithm is not
• I cant come up with an algorithm. Therefore
  there is no algorithm that solves the problem.
• I cant develop a program . Therefore there is
  no algorithm that solves the problem.
• We need some background

6
Paradox

• There is a small town with only one barber.
• The barber shaves only those people who do not
  shave themselves.
• All people are shaved.
• Who shaves the barber? If
• Assuming that such a barber exists, leads to a
  contradiction. Thus he cant exist.

7
Loop

• A loop is a set of instructions that is executed
  repeatedly.
• x0
• 10 times
• add 1 to x
• What will be the value of x when the loop ends?

8
Loop

• x5
• while x gt 0
• subtract 1 from x
• What will be the value of x when the loop ends?
• How many times will the loop be executed?

9
Loop

• x-5
• while x ? 0
• add 1 to x
• What will be the value of x when the loop ends?
• How many times will the loop be executed?

10
Loop

• x0
• while x ? 0
• add 1 to x
• What will be the value of x when the loop ends?
• How many times will the loop be executed?

11
Loop

• x5
• while x ? 0
• add 1 to x
• What will be the value of x when the loop ends?
• How many times will the loop be executed?

12
Infinite loop

• Executes forever

13
Halt?

• Given the initial value of x, we can predict
  whether this loop will end or halt.
• Given a clock, can you predict whether it will
  halt? How long will you have to watch it?
• Given an arbitrary program, can we predict
  whether it will end.?
• Can we write a program to do this?

14
The Halting problem

• Given a program and an input to the program,
  determine if the program will eventually stop
  with this input
• Running the program is not a solution. Suppose we
  tried to run the program corresponding to the
  infinite loop to see if it ends. We would get
  tired of waiting for the answer, and stop the
  program. Thus we would still not have a
  prediction.

15
The Halting problem

• Theorem The Halting Problem is not Computable.
• The Halting Problem is important because it
  proves the existence of an uncomputable/unsolvable
  problem.

16
Proof of Theorem

• See the diagram
• Recall that both programs and data are stored in
  binary. There is no difference in the
  representation.
• Assume there is an algorithm A that solves the
  Halting Problem.
• Write a new program N as follows

17
N

• Given a binary representation of program P
• Use algorithm A to determine if P halts on input
  P
• If A says HALTS, N goes into an infinite loop.
• If A says LOOPS, then N says HALTS.

18
N

• Imagine giving N to itself as data.
• If A says that N HALTS, then N LOOPS.
• If A says that N LOOPS, then N HALTS.
• We have a paradox. A cannot exist. Our only
  assumption was that there is an algorithm A that
  solves the halting problem. It was wrong.
• The Halting Problem is not computable. It is
  unsolvable/uncomputable.

19
Complexity

• linear time proportional to the size of the data
• Recall
• Finding the maximum value in a list of n elements
  by looking at each element in turn is linear.
• Sequential search is linear

20
Complexity

• logarithmic time --algorithms that successively
  cut the amount of data to be processed in half at
  each step are logarithmic
• Finding max by comparing pairs
• Finding a value in a list of n sorted elements
  using the binary search algorithm

21
Complexity

• exponential time
• Processing all subsets of a set
• Trying all moves in chess
• factorial time
• Processing all permutations
• Traveling Salesperson Problem

22
Feasible

• A problem is feasible if it can be solved in a
  reasonable amount of time.
• Must consider all algorithms. If at least one is
  able to do it, the problem is feasible.
• A function can be computable, but yet infeasible.

23
Growth
24
Sort a set of n elements (I)

• The selection sort algorithm
• find the largest of the n elements.
• find the largest of the remaining n-1 elements.
• find the largest of the remaining n-2 elements.
  
• the smallest element is remaining.
• Time n n-1 ... 2 1 n(n 1)/2 units
  quadratic

25
Sort a set of n elements (II)

• Exhaustive listing and search
• List all permutations (orderings) of the data
• Pick the one that is sorted.
• Time n(n-1)(n-2) ... (3)(2)(1) n! units
  factorial. Not very efficient!

26
Feasible

• Sorting is feasible because there exists a
  reasonable algorithm. quadratic
• Linear Search
• Binary Search. logarithmic
• Finding maximum. both linear and logarithmic.

27
Infeasible

• Playing chess by considering every possible move
  is not. exponential
• Considering every possible seating arrangement
  for a large group is not. factorial
• Checking all possible subsets of a set is not .
  exponential

28
Feasibility

• Logarithmic and linear algorithms are feasible
• Exponential and factorial algorithms are
  infeasible.
• An approximate solution may be adequate for an
  infeasible problem.