BM545: Analysis of Algorithms

1
BM545 Analysis of Algorithms

• Murat KÜÇÜKBAYRAK
• mkucukbayrak_at_dho.edu.tr
• Deniz Harp Okulu Sis.Isl.Prog.Subayi
• Lecture 8 Lower Bound Theory, Comparison
  trees, Sorting

2
1.Lower Bound Theory
3
Upper Bound

• That means
• We know we can at least do it within a certain
  amount of time,
• But there might be something better out there.

4
Lower Bound Analysis

• To find out how good we actually can do, we can
  seek LOWER BOUNDS for the algorithms, by proving
  that some given task will take at least so many
  steps to do.
• This would be the best possible amount of time
  we would have to spend doing the problem, given
  arbitrary input.

5
Lower Bound Analysis

• When computer scientists perform their proofs and
  analyses and so on and find that their UPPER
  BOUND is the in the same order as the LOWER BOUND
  for a particular algorithm,
• (that is, the lower bound MEETS the upper bound,)
  then they know they have found an optimal
  algorithm, for something. More or less.

6
Lower Bound Analysis

• We've seen how to calculate an upper bound for a
  running time,
• But how do we compute a LOWER BOUND?
• Well, there are many ways, but most involve a
  lot of math. We won't discuss these so much, but
  we will see example proofs that relates to stuff
  we've seen

7
Lower Bound Analysis

• A Lower Bound states a fact about all possible
  algorithms for solving a problem.

• So Lower Bound proofs are often hard to obtain.

• However for many problems it is possible to
  easily observe that a Lower Bound identical to n
  exists, where n is the number of inputs (or
  possibly outputs) to the problem.

8
An example

• Theorem Any algorithm that sorts by exchanging
  adjacent elements requires O(N2) time on average.
• Proof The average number of inversions is
  initialy N(N-1)/4 O(N2). Each swap removes only
  one inversion, so O(N2) swaps required.

9
1.2 Comparison Trees

• A Comparison Tree is an abstraction used to prove
  Lower Bounds.
• A Comparison Tree is a binary tree.

10
1.2.1 Ordered Searching

• compare x and Ai

• x lt Ai

• x Ai

• x gt Ai

11
Ordered Searching
xA1
xA2
failure
failure
xAn
failure
failure
Comparison Tree modelling Linear Searching
algorithm.
12
Ordered Searching
...
Comparison Tree modelling Binary Search.
13
1.2.2 A Lower Bound For Sorting by Comparisons

• Is k1 lt k2 ?

14
Example with n3 Elements a,b and c
A2ltA1 (b lt a ?)
A3ltA2 (c lt a ?)
A3ltA2 (c lt b ?)
A2ltA1 (c lt b ?)
A2ltA1 (c lt a ?)
Comparison Tree for insertion sort with n3.
There can be 3! possible outcomes.
15
1.2.3 The size of Comparison Trees

• n elements

• n!n.(n-1).(n-2) ... (2).(1)

• a1,a2, ... ,an

16
The size of Comparison Trees

• L input list

• k number of leaves

• p number of leaves

• 1 root
• 2 nodes at level 1
• 4 nodes at level 2
• 2i nodes at level i

17
The size of Comparison Trees

• The Largest number of leaves of a tree with no
  nodes at levels higher than p is 2p.

18
The size of Comparison Trees

• Put another way, a binary tree with k leaves must
  have a path of length at least logk. If we let
  kn!, then we know that any sorting algorithm
  that uses comparisons to determine the sorted
  order must take O(log(n!)) time in the worst case.

19
The size of Comparison Trees

• But how fast does log(n!) grow?

20
The size of Comparison Trees

• A close approximation to n! is (n/e)n
• where e2.7183...is the natural logarithms.
  Since
• log((n/e)n)nlogn-nloge
• we see log(n!) is of order of nlogn.

21
The size of Comparison Trees

• We can get a precise lower bound by noting that
  n(n-2)...(2)(1) is the product of at least n/2
  factors that are at least n/2.

22
The size of Comparison Trees

• Thus n! (n/2)(n/2). Hence
• log(n!) (n/2)log(n/2)
• (n/2)logn - (n/2)

23
RESULT

• Thus sorting by comparisons requires
• O (n log n)
• time in the worst case.

24
References

• Alfred A.,John H.,Jeffrey U.,1987, Data
  Structures and Algorithms, p.282.
• Mark W.,1997,Data Structures and Algorithm
  Analysis in C, p.221.
• Horowitz,Sahni,Rajasekaran, 1998, Computer
  Algorithms, p.457.