Sorting

1
Sorting

• Problem Given a sequence of elements, find a
  permutation such that the resulting sequence is
  sorted in some order.
• We have already seen
• Insertion sort
• average/worst-case running time O(n2)
• best-case running time O(n)
• good for partially sorted or small arrays
• Heap sort
• average/worst-case running time O(nlgn)

2
MergeSort

• A divide conquer algorithm
• Main idea
• break the list in two
• sort each half recursively
• base case a single-element array is already
  sorted
• merge the two halves

3
Sorting MergeSort

• void mergesort (int array , int low, int high)
  
• int mid
• if (low lt high)
• mid(lowhigh)/2
• mergesort(array, low, mid)
• mergesort(array, mid1, high)
• merge(array, low, mid, mid1, high)

4
Sorting MergeSort

• Running time
• Time to mergesort n items
• twice the time to mergesort n/2 items
• the time to merge a total of n items
• T(n) 2T(n/2) n ...O(nlgn)

5
Quicksort

• A divide conquer algorithm
• Basic idea
• Pick an element (called pivot)
• Partition the array in two subsequences those
  smaller than the pivot and those larger or equal
• Sort each subsequence recursively

6
Quicksort

• Partitioning an array
• Goal move all elements smaller than or equal to
  the pivot to the left of the array and all
  elements larger than or equal to the pivot to the
  right of the array. In the end, the right part of
  the array will contain elements that are larger
  than or equal to those in the left part.

7
Quicksort

• Running time depends on selection of pivot
• Best case (array partitioned in half every time)
• time to quicksort n elements
• time to partition array
• twice the time to quicksort n/2 elements
• T(n) 2T(n/2) n ... O(nlgn)

8
Quicksort

• Running time depends on selection of pivot
• Average case
• Still O(nlgn) but with a slightly larger constant
  
• Worst case
• Partitioning n elements results in two
  subsequences of lengths 1 and n-1
• T(n) T(n-1) n ... O(n2)

9
Quicksort

• Selecting the pivot
• The first element
• The middle element
• The median of three elements
• Random element
• IdeaRandomized quicksort
• Before sorting the array, randomly permute the
  elements to make the running time independent of
  the input ordering
• The permutation takes some time, but in practice
  randomized quicksort is almost always O(nlgn)

10
Related problem Selection

• Problem
• Given an unsorted sequence of elements find the
  k-th smallest
• Idea
• Use a procedure similar to quicksort but sort a
  subsequence only if you need to.

11
Related problem Selection

• Algorithm Quickselect(A, low, high, k)
• select the pivot
• partition the array in two subsequences around
  the pivot. Let i be the index between the two
  pieces.
• if i k, return pivot
• if i gt k return Quickselect(A, low, i-1, k)
• if i lt k return Quickselect(A, i1, high, k)

12
Sorting Lower bounds

• Comparison sort A family of algorithms that use
  comparisons to determine the sorted order of a
  collection
• Examples Mergesort, quicksort, insertion sort
• Decision tree a tree that represents the
  comparisons performed by a sorting algorithm on
  input of given size

13
Sorting Lower bounds

• decision tree for 3 elements

a ? b
?
?
b ? c
a ? c
?
?
?
?
b ? c
(b , a , c )
a ? c
(a , b , c)
?
?
?
?
(a , c , b )
(c , a , b )
(c , b a )
(b , c , a )
14
Sorting Lower bounds

• Each internal node contains a comparison
• Each leaf contains a permutation
• Algorithm execution a path from the root to a
  leaf
• Worst-case number of comparisons height of
  decision tree
• Idea If we find a lower bound on the height of
  the decision tree, well have a lower bound on
  the running time of any comparison algorithm

15
Sorting Lower bounds

• A decision tree that sorts n elements has height
  at least nlgn
• This means that algorithms such as heapsort and
  mergesort whose running times are O(nlgn) are
  asymptotically optimal.
• However, in some cases, if we have additional
  information about the input, we may be able to
  perform a sorting in linear time.

16
Extra! Extra! Counting sort

• Restriction each of the n elements is an integer
  in the range 1 through k
• Idea
• For each element x, count the number of elements
  less than x.
• Then, we will be able to place x directly into
  its correct position in the sorted array.
• The algorithm uses two temporary arrays
• array B of size n to hold the sorted output
• array C of size k for temporary storage

17
Extra! Extra! Counting sort
// array of size n, input in range 1..k for i1
to k Ci 0 for i1 to n CAi CAi
1 for i2 to k Ci Ci Ci-1 for in
downto 1 BCAi Ai CAi CAi - 1

• Running time O(nk)

18
Stable sorting algorithms

• A sorting algorithm is stable if it does not
  change the relative order of items with the same
  value.
• This is important when satellite data is carried
  around with the element being sorted.
• Example ATM transactions where the key is the
  account number. Initially the transactions are
  ordered by time, but if we sort them by account
  number, the relative order of the transactions
  must stay the same.
• Counting sort is stable (but would it still be so
  if the last loop was increasing instead of
  decreasing?)
• Quicksort is not stable

19
Space requirements

• Mergesort needs additional space of ?(n) for the
  temporary array.
• Counting sort needs additional space of ?(nk)
  for the temporary arrays
• All other algorithms that we saw sort the arrays
  in-place and need only constant additional space
  for temporary variables.