Lower Bounds for Comparison Sorts

1
Lower Bounds for Comparison Sorts

• Assume
• All elements are distinct.
• All comparisons are of form ai aj
• Can view any comparison sort in terms of a
  decision tree.

2
Decision Tree for Insertion Sort(n 3)

12


gt


23

13

gt



gt



lt1, 2, 3gt

lt2, 1, 3gt

13

23

gt


gt




lt1, 3, 2gt

lt3, 2, 1gt

lt3, 1, 2gt

lt2, 3, 1gt

3
Lower Bound for Worst Case

• Theorem
• Any comparison sort algorithm requires O(n lg n)
  comparisons in the worst case.

4
Lower Bound for Worst Case (continued)

• Proof
• Let h be height of tree and l be number of
  leaves. Tree must have at least n! leaves since
  each permutation of input must be a leaf. Binary
  tree of height h has at most 2h leaves. Thus
• n! l 2h
• By taking logarithms
• h lg(n!)
• O(n lg n)

5
Lower Bound for Worst Case (continued)

• Corollary
• Heapsort and merge sort are asymptotically
  optimal comparison sorts.
• Proof
• The O(n lg n) upper bounds on the running times
  for heapsort and merge sort match the O(n lg n)
  worst-case lower bound from the theorem.

6
Counting Sort

• Assumes each of the n input elements is an
  integer in the range 0 to k, for some integer k
• For each element x, determine the number of
  values ??x.
• Requires three arrays
• An input array A1..n
• An array B1..n for the sorted output
• An array C0..k for counting the number of times
  each element occurs (temporary working storage)

7
Counting Sort (continued)

• COUNTING-SORT (A, B, k)
• for i ? 0 to k
• do Ci ? 0
• for j ? 1 to lengthA
• do CAj ? CAj 1
• // Ci now contains the number of elements equal
  to i.
• for i ? 1 to k
• do Ci ? Ci Ci-1
• //Ci now contains the number of elements less
  than or equal to i.
• for j ? lengthA downto 1
• do BCAj ? Aj
• CAj ? CAj - 1

8
Complexity of Counting Sort

• Lines 1-2 T(k)
• Lines 3-4 T(n)
• Lines 6-7 T(k)
• Lines 9-11 T(n)
• Total T(nk)

9
Counting Sort

• Beats the lower bound of ?(nlgn) because it is
  not a comparison sort
• Makes assumptions about the input data
• Is a stable sort numbers with the same value
  appear in the output array in the same order as
  they do in the input array

10
Radix Sort

• RADIX-SORT (A, d)
• for i ? 1 to d
• do use a stable sort to sort array A on
  digit i

11
Bucket Sort

• BUCKET-SORT(A)
• n ? lengthA
• for i ? 1 to n
• do insert Ai into list B
• for i ? 0 to n-1
• do sort list Bi with insertion sort
• concatenate lists B0, B1, , Bn-1 together
  in order