Algorithms and Data Structures

1

• Algorithms and Data Structures
• Limits of Computation
• Revisions
• Dr. Zumao Weng

2
Inefficient Problem Solutions

• For example we might sort an array of N integers
  by successively generating permutations of the N
  values and then testing whether the sorted
  sequence has been created.
• 7 2 3 25 100
• 2 25 3 7 100
• 100 2 3 7 25
• ........
• This is a very inefficient way of sorting but we
  might be lucky and generate the correct
  permutation after one try!
• Complexity
• For n values there are n! (n?(n-1)
  ?(n-2)...1) permutations.
• The lower bound for sort algorithms is nlog2n.
• No sort algorithm can do better than this
  complexity.

3
Limits of Problem Solution

• Problem solutions mean that
• reasons for measuring and comparing algorithm
  complexity.
• asymptotic analysis is a means of assessing and
  comparing algorithm behaviour.

4
Issues

• Are there problems that cannot be solved?
• This question asks whether there are problems
  that have no algorithmic solution.
• Are there algorithms that are not feasible to
  use?
• This question asks whether there are good
  algorithms that operate so slowly that they are
  not useful problem solvers.

5
Towers of Hanoi

• There are many computing problems that
  superficially seem simple and yet are not
  feasible to solve.
• Towers of Hanoi Problem
• A puzzle that consists of three poles and n rings
  that initial start on the left-most pole.
• Each ring is of a different size.
• The problem is to move the rings from the
  leftmost pole to the rightmost pole without
  placing a larger ring on top of a smaller one.

6
Towers of Hanoi
void TOH (int N, Pole start, Pole goal, Pole
temp) if (N ltgt 0) TOH(n-1,
start, temp, goal) move(start,
goal) TOH(n-1, temp, goal, start)

7
Complexity

• The algorithm requires 2n moves.
• It is not possible for a computer to run in less
  that 2n since each move must be printed.

8
Hard Problems

• A problem might be hard because
• it is difficult to understand or
• it is difficult to find an algorithm to solve it.
• A hard problem is one whose complexity is such
  that the algorithm is expensive to run.
• e.g. Towers of Hanoi for 30 discs.
• ToH is an exponential complexity algorithm. 2n
• Algorithms can be n2 or n3 or n4
• These are examples of polynomial running times.

9
ToH as a Hard Problem

• ToH is an exponential complexity algorithm. 2n.
• Consider a computer on which we execute ToH in a
  fixed period.
• If we buy a new (whizz-bang XX) computer that is
  twice as fast as the current computer then we
  will be able to solve ToH with only one new disc!
• Formally, a hard problem (often called
  intractable) is
• one whose complexity is exponential time.
• Cn for a constant C gt 1
• For an exponential time algorithm, a constant
  factor increase in execution speed will result in
  a fixed addition in the problem size.

10
NP Problems

• Imagine a magical computer that guesses the
  correct solution to problems from all the
  possible solutions.
• For example, in sorting N numbers this magical
  computer would guess the sorted permutation of
  the N numbers.
• The magical computer might be realised by a super
  parallel computer that simultaneously checks all
  possible solutions.
• The magical computer might be able to solve
  problems faster than a conventional one.

11
Checking Solutions

• Consider a problem where a guess takes polynomial
  time to verify that it is a correct solution.
• e.g. verify that a sequence is sorted can be
  performed in polynomial time.
• The parallel implementation of the magical
  computer could check a solution in polynomial
  time or more generally check all possible
  solutions simultaneously in polynomial time.
• If an exponential time problem can be verified in
  polynomial time then the magical computer can
  solve an exponential time algorithm in polynomial
  time.
• Conversely, if one cannot obtain a solution in
  polynomial time by guessing then one cannot do it
  in polynomial time in any other way.

12
Non-deterministic Solutions

• The idea of guessing the right answer (or
  checking all possible solutions in parallel to
  determine which is correct) is called
  non-determinism.
• An algorithm that works in this manner is called
  a non-deterministic algorithm.
• Any problem with an algorithm that runs on a
  non-deterministic machine in polynomial time is
  called an NP one.
• non-deterministic solution checked in polynomial
  time.

13
NP Problems

• A problem solution performed non-deterministically
  and checked in polynomial time
• Not all problems requiring exponential time on a
  regular computer are in NP.
• Towers of Hanoi is not in NP since it requires 2n
  moves for n discs.
• NP class
• non-NP class

14
Travelling salesman

• The travelling salesman problem is one that
  attempts to find the shortest possible route
  visiting all nodes in a graph.
• The travelling salesman problem is an exponential
  algorithm that is in NP

15
NP-complete problems

• A problem is NP-complete if
• 1. if the problem is in NP and
• 2. any other problem in NP can be reduced to the
  problem.
• If a polynomial algorithm can be found for an NP
  problem then a polynomial can be found for all NP
  problems.
• k-clique problem and travelling salesman
• connected subgraph of G with k vertices.

16
(No Transcript)
17
Classes of problem solution

• Clearly, all problems in class P are solvable by
  a non-deterministic machine
• by just ignoring the non-deterministic
  capability.
• Some problems in NP are NP-complete.
• The biggest unanswered question in computer
  science is
• P NP

18
Problems that cannot be solved

• All programmers (unfortunately) write programs
  that sometimes go in to infinite loops.
• It would be good if a compiler could detect an
  infinite loop and flag an error.
• Unfortunately, it is not possible to devise an
  algorithm that will deduce whether a program will
  terminate
• generally referred to as the Halting Problem.

19
An Infinite Loop.........perhaps

• while (n gt 1)
• if (ODD(N))
• n 3 n 1
• else
• n n / 2

• The program computes the Collatz sequence.
• No proof exists that this fragment terminates for
  all values of n.
• It is hoped that one day this fragment will be
  completely analysed.

20
Halting problem cannot be solved...

• Let's assume that there is a method halt that
  returns true if a program terminates for some
  input and false otherwise.

boolean halt(String prog, String input)
if (...prog does halt for input...)
return true else return false
Unfortunately we cannot write such a method
because it is possible to establish a
contradiction about this method.....
21
Halting

• boolean I_Halt(String myprog, String myinput)
• if (halt(myprog, myinput))
• return true
• else
• return false

• Before a program executes it can first ask
  whether it will terminate

22
The Contradiction
boolean I_Cannot_Halt(String myprog, String
myinput) if (I_Halt(myprog, myinput))
while (true) // infinite loop

• if I_Halt is true, i.e. the program terminates,
  then the method goes into an infinite loop.
• If it does not halt then I_Halt returns false and
  the program terminates.
• This is an obvious contradiction which can only
  be resolved by asserting that it cannot be
  possible to write function I Halt and by
  implication function Halt.
• An thus the halting problem is not solvable or
  decidable.

23
Why is this important?

• It is not generally possible automatically to
  prove that a program is mathematically correct.
• although programs have been proved automatically
  correct.
• It is not generally possible to prove that a
  program computes a particular function.
• Although particular cases can be deduced.
• It is not generally possible to prove that a
  particular line of code in a program will ever be
  executed.
• Although particular cases might be deduced.

24
Why is this important?

• It is not generally possible to prove that a
  program does not contain a virus.
• Although particular viruses might be searched
  for!
• The standard way of proving that a program cannot
  be solved is to reduce the new problem to an
  instance of the halting problem.

25

• Revisions

26
Examination Format

• 6 Questions
• You choose 4 questions to answer from the 6
  questions
• Each question is worth 25 marks. (100 4 x 25)
• Each Question has an approximate break down of
• 8 marks Background and definitions
  (bookwork)
• 9 marks Application of familiar material
• examples from notes, examples from laboratory
  classes
• 8 marks Unseen application.
• examples which are not part of the notes.
• Definitions are intended to set the scene in
  order that you can answer the other parts of the
  question.

27
Module Areas

• Java Review.
• Java knowledge will be examined in each question.
• Background
• ADTs, Algorithm theory and algorithm analysis
• Sort Algorithms
• Linear ADTs
• Linear abstraction forms
• Hash Table ADTs
• Hierarchical ADTs
• binary tree ADT
• B-tree ADT
• Graph ADTs