CSE 502N Fundamentals of Computer Science

1
(No Transcript)
2
CSE 502NFundamentals of Computer Science

• Fall 2004
• Lecture 11
• Lower Bounds for Sorting
• Counting Sort Radix Sort
• (CLRS 8)

3
Lower bound for comparison sorting

• Comparison sorting algorithms determine the
  sorted order based only on comparisons between
  the input elements
• Given input sequence a1,a2,,an, we perform
  comparisons of the form ai aj in order to
  determine their relative order
• None of the algorithms we studied perform better
  thanO(n lg n)
• Insertion sort, T(n) O(n2)
• Merge sort and heapsort, T(n) O(n lg n)
• Quicksort, ET(n) O(n lg n)
• Is W(n lg n) a lower bound for all comparison
  sorts? What about all sorting algorithms?

4
Decision-tree model

• Comparison sorts can be viewed abstractly as
  decision trees
• An execution of the algorithm corresponds to
  tracing a path from the root to a leaf
• A comparison ai aj is made at each internal
  node
• Each leaf corresponds to an ordering of the
  elements
• Any sorting algorithm must be able to produce all
  n! possible orderings
• Decision tree must contain n! leaves reachable
  from the root of the tree

a1a2

gt
a2a3
a1a3

gt

gt
a2a3
a1a3
a1,a2,a3
a2,a1,a3

gt

gt
a1,a3,a2
a3,a1,a2
a2,a3,a1
a3,a2,a1
5
Proving a minimum tree height

• Worst-case number of comparisons equals the
  height of the decision tree
• A lower bound for comparison sorting is equal to
  the minimum height of the tree
• Minimum tree height for a decision tree
  containing n! reachable leaves
• Consider a decision tree of height h with l
  reachable leaves
• Since the decision tree corresponding to a
  comparison sorting algorithm must contain n!
  leaves, then n! l
• Since a binary tree of height h contains no more
  than 2h leaves, then n! l 2h
• Taking the logarithm h ³ lg(n!)
• Recall that lg(n!) Q(n lg n)
• Thus, h W(n lg n)
• Therefore, heapsort and merge sort are
  asymptotically optimal comparison sorts

6
Counting Sort

• Assume each of the n input elements is an integer
  in the range 0 to k
• Basic idea for each input element x, determine
  the number of input elements less than x
• Use this information to place x in the proper
  position in the sorted order
• Line 11 handles the case of non-distinct elements

7
Running time of counting sort

• T(n) Q(k) Q(n) Q(k) Q(n) Q(k n)
• If k O(n), then T(n) Q(n)
• Beats the W(n lg n) lower bound for comparison
  sorting
• Never performs a comparison between input
  elements!

8
Stability of counting sort

• Input elements with the same value appear in the
  output array in the same order as they do in the
  input array
• Important property when satellite data is
  associated with each element
• Also useful for constructing other sorting
  algorithms

9
Radix Sort

• Algorithm used by card-sorting machines
• Sorts a set of d-digit numbers by sorting on the
  i-th digit where 0 i d
• d iterations of a stable Q(n) sorting algorithm
• T(n) Q(d(k n))
• When d is constant and k O(n), then T(n) Q(n)