COMP171 Tutorial 5

1
COMP171 Tutorial 5

• Sorting Algorithms Wrap-Up

2
Agenda

• Review of Lower Bound of Sorting and Radix Sort
• Exercises related to sorting algorithms

3
Review of Lower Bound of Sorting

• Theorem all comparison sorts are ?(n log n)
• Corollary Heap-sort and Merge-sort are
  asymptotically optimal comparison sorts
• Proof?

4
Review of Lower Bound of Sorting

• Decision trees provide an abstraction of
  comparison sorts
• A decision tree represents the comparisons made
  by a comparison sort. Everything else ignored

5
Review of Lower Bound of Sorting

• Decision Tree Example

6
Review of Lower Bound of Sorting

• Decision trees provide an abstraction of
  comparison sorts
• A decision tree represents the comparisons made
  by a comparison sort. Everything else ignored
• What do the internal nodes represent?
• What do the leaves represent?
• How many leaves must there be? n!
• What do a sorting procedure correspond to?
• What is the asymptotic height of any decision
  tree for sorting n elements?
• Answer ?(n log n)

7
Review of Simple Counting Sort

• Also a simple case of bucket sort
• For standard counting sort, refer to
  Introduction to algorithms, 2nd Ed., Ch.6.

8
Review of Simple Counting Sort

• Why dont we always use counting sort?
• Because it depends on range k of elements
• Could we use counting sort to sort 32 bit
  integers? Why or why not?
• Answer no, k too large (232 4,294,967,296)

9
Review of Radix Sort

• Sort from the least significant bit to the most
  significant bit using counting sort.

10
Review of Radix Sort

• In general, radix sort based on counting sort is
• Fast
• Asymptotically fast (i.e., O(n))
• Simple to code
• A good choice
• Can radix sort be used on floating-point numbers?
• Generally NO
• For specific representations YES
• First base then mantissa
• Not applicable to all floating point
  representations!

11
Review of Lower Bound of Sorting

• Why comparison sorts?
• Complexity is worse than linear ?
• So many to choose from ?
• Answer
• In place sorting storage issues
• Elements that do not have KEYs
• E.g. Sorting fruits
• E.g. Sorting students by height when no exact
  height is available

12
Sorting Algorithms

• Demo
• Animation of sorting algorithms
• http//math.hws.edu/TMCM/java/xSortLab/
• http//www.cs.ubc.ca/harrison/Java/sorting-demo.
  html
• http//www.cs.uwaterloo.ca/bwbecker/sortingDemo/
• http//www2.hawaii.edu/copley/665/HSApplet.html(
  heap sort)
• http//pages.stern.nyu.edu/panos/java/Quicksort/
  (quick sort)

13
Sorting Algorithms Comparison
14
Sorting Algorithms Comparison
15
Sorting Algorithms Comparison

• Sorting a Linked-List?
• Simple Sorts(Selection, Insertion, Bubble)
• YES ( Singled / Doubled Linked-List )
• Quick Sort
• YES (Doubled / Singled with a modified PARTITION
  )
• Heap Sort
• NO
• Merge Sort
• YES ( Singled / Doubled Linked-List )
• Radix Sort
• YES ( Need an extra array )

16
Timing Comparisons

• O(n2) Sorting Algorithms

17
Timing Comparisons

• O(n log n) Sorting Algorithms

18
Complexity Comparison

• Comparing the functions 10n2 and 30 n log(n) for
  small values of n.

19
Timing Comparisons

• Sorting contest
• http//www.iti.fh-flensburg.de/lang/algorithmen/s
  ortieren/sortcontest/sortcontest.htm

20
How to sort

• 1. Distinct Integers in Reverse Order
• Radix Sort is best, if space is not a factor.
• Insertion Sort O(n2) also worst case
• Selection Sort always O(n2)
• Bubble Sort O(n2) also worst case
• Quicksort
• Simple Quicksort O(n2) worst case
• Median-of-3 pivot picking, O(n log n)
• Heapsort always O(n log n)
• Mergesort always O(n log n)
• Radix Sort O(nk) O(n).

21
How to sort

• 2. Distinct Real Numbers in Random Order
• Quicksort is best. Heapsort is good. Mergesort is
  also good if space is not a factor.
• Insertion Sort O(n2)
• Selection Sort always O(n2)
• Bubble Sort O(n2)
• Quicksort O(n log n) in average case (instable)
• Heapsort always O(n log n) (instable)
• Mergesort always O(n log n) (stable)
• Radix Sort not appropriate for real numbers.

22
How to sort

• 3. Distinct Integers with One Element Out of
  Place
• Insertion Sort is best.
• If the element is later than its proper place
  then Bubble Sort (to the smallest end) is also
  good.
• Otherwise, Radix Sort.

23
How to sort

• 3. Distinct Integers with One Element Out of
  Place
• Insertion Sort O(n)
• Selection Sort always O(n2)
• Bubble Sort
• Later O(n).
• Earlier O(n2).
• Quicksort
• Simple Quicksort O(n2) close to the worst case
• Median-of-3 pivot picking, O(n log n)
• Heapsort always O(n log n)
• Mergesort always O(n log n)
• Radix Sort O(kn) O(n).

24
How to sort

• 4. Distinct Real Numbers, Almost Sorted
• Insertion Sort is best, Bubble Sort almost as
  good
• Insertion Sort Almost O(n).
• Selection Sort always O(n2)
• Bubble Sort Almost O(n).
• Quicksort depending on data, somewhere between
  O(n2) and O(n log n)
• Heapsort always O(n log n)
• Mergesort always O(n log n)
• Radix Sort Not appropriate for real numbers.

25
Why so many sorting algorithms?

• Different sorting algorithms for different
  purposes
• In-place and stable sorting
• Dont just rely on asymptotic analysis think on
  a case-by-case basis!
• STL sort in ltalgorithmgt package
• Implemented as Introsort (a combination of
  Quick-sort, Heap-sort and insertion sort)
• Less than 16 elements insertion sort
• If recursion of quick-sort goes too deep, use
  heap sort

26
Sorting Application Example

• Problem
• Find 100 distinct elements in an 1 million sized
  unsorted array

27
Sorting Application Example

• Approach 1
• Direct search (sequential)
• worst case
• 100 1,000,000
• 100 million comparisons.

28
Sorting Application Example

• Approach 2
• First, sort the array using Heapsort or
  Quicksort
• O(n log n) 19.9 million operations
• Then, do 100 binary searches
• O(100 log n) 1993 operations.
• Total is about 20 million operations.
• O(n log n 100 log n) O(n log n).

29
Sorting Application Example

• Approach 3
• First, sort the elements to be searched
• 100 log 100 664 operations (negligible)
• Then, for each of the n elements, perform a
  binary search on the 100 sorted elements to see
  if it matches
• O(n log 100) 1,000,000 x 6.64
• 6.64 million operations.
• O(100 log 100 n log 100) O(c1 n.c2) O(n).

30
Q A?