CS2420: Lecture 14

1
CS2420 Lecture 14

• Vladimir Kulyukin
• Computer Science Department
• Utah State University

2
Outline

• Sorting Algorithms (Chapter 7)

3
Binary Tree Heights and Leaves

• For any binary tree with L leaves and height H

4
A Decision Tree

• Each internal node represents a decision.
• Each leaf represents an outcome.

5
Decision Trees for Comparison Sorting Algorithms

• Given an array A of N elements.
• Each outcome is a leaf.
• Each outcome is a permutation of N elements. We
  will call is P(A).
• There are N! possible outcomes.
• Each decision node is a comparison of two
  elements.

6
Lower Bound for Comparison Sorting
length(A) N P(A)s are permuations of A.
A
Log(N!)

P(A)
P(A)
P(A)
P(A)
P(A)



N!
7
Decision Trees for Comparison Sorting Algorithms
8
Stirling Approximation
9
Stirling Approximation
10
Lower Bound for Comparision Sorting
11
What Does the Lower Bound Tell Us?

• The lower bound tells us that, in general, at
  least NlogN comparisons (decisions) must be made
  to sort an array of N elements if we use a
  comparison-based sorting algorithm.

12
Comparison Sorts So Far

• InsertionSort O(N2)
• SelectionSort O(N2)
• BubbleSort O(N2)
• MergeSort O(NlogN)
• QuickSort O(N2) worst,
• O(NlogN) - average

13
Can We Beat NLOGN?

• Suppose that we have N positive numbers.
• Suppose that we know that all of them are smaller
  that some number M.
• Do we need an asymptotically optimal sort or can
  we do better?

14
Can We Do Better Than NLOGN?

• We can create an array countM1 of M elements.
• Each element in B is initialized to 0.
• What is counti?
• counti is a non-negative integer that records
  how many times number i occurs in the input.

15
Bucket Sort

• Given an array A of N positive integers less than
  M
• Create and initialize countM
• Go over the array A once and update the counts in
  countM
• Create an array B of size N and fill it up with
  the information from countM.

16
Bucket Sort Example

• All numbers are less than 10.
• Suppose that our input is A 1, 3, 4, 3, 5, 5,
  9, 7, 7, 6, 8, 8, 8.
• count10 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
• We go through A and count the occurrences of each
  number in A and record them in count.
• count10 0, 1, 0, 2, 1, 2, 1, 2, 3, 1.
• The sum of occurrences in the final count must be
  equal to the number of elements in A.

17
Bucket Sort Example

• We then allocate an array BN and use countM
  to fill it up. Here is how
• count1 1, so we one 1 into B. So B 1.
• count2 0, so we do not add any 2s into B.
• count3 2, so we put two 3s into B. B 1,
  3, 3.
• count4 1, so we put one 4 into B. So B 1,
  3, 3, 4.
• count5 2, so we add two 5s into B. So B
  1, 3, 3, 4, 5, 5.

18
Bucket Sort Example

• count6 1, so we add one 6 into B. So B 1,
  3, 3, 4, 5, 5, 6.
• count7 2, so we add two 7s into B. B 1,
  3, 3, 4, 5, 5, 6, 7, 7.
• count8 3, so we add three 8s into B. B 1,
  3, 3, 4, 5, 5, 6, 7, 7, 8, 8, 8.
• count9 1, so we add one 9 into B. B 1, 3,
  3, 4, 5, 5, 6, 7, 7, 8, 8, 8, 9.
• B is the sorted version of A.

19
Bucket Sort Asymptotic Analysis

• We need O(M) to allocate and initialize countM.
• We need to go over A once and fill in the
  frequencies in countM. This takes O(N).
• We need to allocate B and fill it up. This takes
  O(N).
• Total time O(MN). This is linear, not NLogN.

20
Bucket Sort Asymptotic Analysis

• Does Bucket Sort violate the theorem and if we
  use a comparison sort we need at least NlogN
  comparisons to sort an array of N elements?
• No! Because Bucket Sort is NOT a comparison sort.
  It makes no comparisons whatsoever. It just
  counts frequencies.
• Frequency counting is made possible, only because
  there is a piece of additional information all
  numbers are less than M.