Leaps and Bounds: Finer Points of Analysis

1
Leaps and BoundsFiner Points of Analysis

• Growth of Functions, Tightness of Bounds, Proving
  Upper and Lower Bounds
• Helpful Reading CLR 2.1-2.2, 9.1

2
Growth of Functions

• Often, we will derive algorithms and identify
  their worst-case average time complexity by
  inspection of the code or the master method
• Since constants are stripped, two algorithms that
  are Q(f(n)) basically have a constant ratio
  performance difference that is independent of n
  -- go out and measure it
• How do we compare O(f(n)) and O(g(n))?

3
Watching Functions Grow

• The relative asymptotic behavior of two functions
  f(n) and g(n) is given by
• Evaluate the limit
• zero g(n) grows faster than f(n), f(n) O(g(n))
• infinity f(n) grows faster than g(n), f(n)
  W(g(n))
• constant similar growth rates, f(n) Q(g(n))
• Sometimes we need LHopitals rule to help
  evaluate the limit (from calculus)

4
What the bounds really say

• The asymptotic notations really describe sets of
  functions - many different functions are O(n2),
  for example (n, n log n, n2, 5, etc.)
• f(n) Î O(g(n)) if there exist positive constants
  c and n0 such that 0 lt f(n) lt cg(n) for all n gt
  n0
• As long as at some point, g(n) grows faster than
  f(n) and never loses ground, f(n) O(g(n))
• f(n) Î W(g(n)) if there exist positive constants
  c and n0 such that 0 lt cg(n) lt f(n) for all n gt n0

5
Proving Bounds for a Problem

• Easy to show upper bound
• Give an algorithm as proof!
• Harder to show a lower bound
• An algorithm by itself doesnt do the trick
  unless you can argue it is the fastest algorithm!
• Example is comparison-based sorting
• I have shown a O(n2) upper bound (bubblesort)
• Is Q(n log n) (mergesort) the best we can do?

6
Decision Trees

• We will use a decision tree to show every
  possible program output for sorting
• In this tree, left true, right false
• In a decision tree, every possible outcome
  appears as a leaf of the tree

7
Lower Bounds from Trees

• For a problem with n outcomes, there must be n
  leaves
• The minimum height of a decision tree with n
  leaves is logb n (b is branch factor)
• Thus, every path through the algorithm must have
  at least length logb n
• We can use this fact to show a lower bound for
  any problem where the operations can be modeled
  as a decision tree

8
Lower Bound for Sorting

• Since every possible permutation of the n
  elements appears, n! leaves nPr(n, n)
• b 2 for sorting (either a lt b or not)
• Thus, the tree height is at least lg (n!)
• We will use Stirlings approximation, which says
  n! gt (n/e)n
• Thus, tree height is at least lg ((n/e)n) n(lg
  n - lg e) n lg n - n Q(n lg n)
• The lower bound is thus W(n lg n)