Analysis of Algorithms

1
Analysis of Algorithms

• Introductory Example
• Principles of Analysis
• Big-Oh notation
• Time complexity of iterative algorithms
• Time complexity of recursive algorithms
• References
• GT 4.2 , or
• LD Chapter 1, 2.1-2.2, or
• CL Chapter 1-3 , or

2
Example problem

• Checking results of an exam
• Input
• A set of students (name, personal no) who passed
  the exam
• Name of a student
• Output yes or no
• To implement this on a computer we
• Characterize the data and the needed operations
  on data ? ADT (e.g ADT Set)
• Choose a representation of the data in the
  computer? ? datastructure (e.g. table/array)
• Implement the needed operations algorithm
  (e.g. Table search)
• Analyse the efficiency of the implementation

3
How to measure time/space

• Describe algorithms in pseudocode
  (or in a high-level
  programming language)
• Analyse number of basic operations as function of
  the size of input data
• Use simple data memory model
• Analyse number of needed memory cells

4
Example of pseudocode

• function TableSearch ( tableltintegergt T1..n,
  key k )boolean
• for i from 1 to n do
• if T i k then return true
• if T i gt k then return false
• return false
• ...scope defined by indentation!

5
Principles of Algorithm Analysis

• An algorithm should work for (input) data of any
  size.(Example TableSearch input size is the
  size of the table.)
• Show the resource (time/memory) used as an
  increasing function of input size.
• Focus on the worst case performance.
• Ignore constant factors
• analysis should be machine-independent
• more powerful computers introduce speed-up by
  constant factors.
• Study scalability / asymptotic behaviour for
  large problem sizes
• ignore lower-order terms, focus on dominating
  terms.

6
Order notation

• Compares growth rate of functions
• f ? O(g)
• f grows at most as fast as g
• apart of constant factors
• Refer to simple well known functions
• f(n) grows aproximately as n2

7
Types of growth comparison

• f , g growing functions from natural numbers to
  positive real numbers
• f is (in) O(g) iff there exist c gt 0, n0 ? 1 such
  that f(n) ? c g(n) for all n ? n0Intuition
  Apart from constant factors, f grows at most as
  quickly as g
• f is (in) W(g) iff there exist c gt 0, n0 ? 1 such
  that f(n) ? c g(n) for all n ? n0Intuition
  Apart from constant factors, f grows at least as
  quickly as g
• W() is the converse of O, i.e. f is in W(g) iff
  g is in O(f)??
• f is (in) Q(g) iff f(n) ? O(g(n)) and g(n) ?
  O(f(n))Intuition Apart from constant factors,
  f grows exactly as quickly as g

8
Types of growth comparison
W(g) , Q(g), O(g) ..??
9
comparison with simple math function

• 1.84 1019 µsec 2.14 108 days 5845
  centuries

10
To check growth rate?
11
Estimating execution time for iterative programs
12
Example of algebraic analysis
13
Example Dependent Nested Loops
14
Analysis of Recursive Program (1)
15
Analysis of Recursive Programs...
16
Towers of Hanoi....

• As stated earlier
• Formulate an equation T(1)... T(2)...
• Unroll a few times, get hypothesis for T(n)...
• Prove the hypothesis!

17
Average case analysis

• Reconsider TableSearch() sequential search
  through a table
• Input argument one of the table elements,
• assume it is chosen with equal probability for
  all elements.
• Expected search time

Time to find the element when it was in the nth
place