Selection sort, reviewed

1
Lecture 25

• Selection sort, reviewed
• Insertion sort, reviewed
• Merge sort
• Running time of merge sort, 2 ways to look at it
• Quicksort
• Course evaluations

2
Selection sort

• for k 1 to n-1
• find kth smallest item
• swap it with the one in the kth position
• number of comparisons
• n-1 for k1
• n-2 for k2
• 1 for kn-1
• total n(n-1)/2

3
Insertion sort

• for k2 to n
• move kth item into sorted order with respect to
  the first k-1 items which already sorted
• involves moving sorted items over can do this at
  the same time as finding where kth item has to go
• number of comparisons worst case
• 1 for k2
• 2 for k3
• n-1 for kn
• total n(n-1)/2
• on average, maybe n(n-1)/4, but involves more
  moving of data than selection sort

4
Bubble sort

• Pass through data, swapping neighbors that are
  out of order
• After first pass, largest element has sunk to
  bottom
• Repeat until no more swaps are needed,
  considering one less pair each time
• Cost n-1 comparisons for first pass, n-2 for 2nd
  pass, etc.
• May need n-1 passes, so also O(n2)

5
Merge sort

• Recursive
• If only one item, return
• Sort first half by merge sort
• Sort second half by merge sort
• Merge the results

6
Keys to efficiency

• Recursion reduce problem to two problems of half
  the size
• Also known as divide and conquer
• Merge step only requires n comparisons, where n
  is number of items to be merged
• Need two arrays, but no more
• Do not construct arrays inside the recursive
  call!!! (The comprehensive edition of Liang does
  this)
• How should we implement this in Java?
• public static sort(what parameters?)
• We cannot pass half an array, so what to do?

7
Details for sort

• public static void sort(int left, int right,
  double x, double temp)
• if(left right)
• return
• int mid (leftright)/2
• sort(left, mid, x, temp) // sort first half of x
  using temp
• sort(mid1, right, x, temp) // sort second half
  of x using temp
• merge(left, mid, right, x, temp) // merge them
  into temp
• copy(left, right, temp, x) // copy temp back to
  x

8
Details for merge(left, mid, right, x, temp)

• Keep 3 ints i,j,k
• i initialized to left
• j initialized to mid1
• k initialized to left
• advance until i gt mid or j gt right
• copy the rest of one of the other half
• cute trick 2 additional while loops
• another trick do nothing if xmid lt
  xmid1
• for this reason, better move the copy inside
  merge

9
Number of comparisons

• Let C(n) number of comparisons to sort array of
  length n by merge sort
• Clearly, C(n) 2 C(n/2) (n-1), if n is even, gt
  0
• or a little less, if we program merge efficiently
• C(1) 0
• C(2) 1
• C(4) 5
• C(8) 17
• C(16) 49
• Hard to see a pattern, but when we double n, the
  number of comparisons is also doubled, an extra
  n-1 comparisons

10
Lets simplify this slightly

• Count the merge as n comparisons, instead of n-1
• C(n) 2 C(n/2) n, if n is even, gt 0
• C(1) 0
• C(2) 2
• C(4) 8
• C(8) 24
• C(16) 64
• Do you see a pattern now? Assume n is a power of
  2, say 2k
• By inspection, C(n) k n
• In other words, C(n) n log2(n)

11
Proof by induction on k

• Base case C(1) 1 log2(1) 0
• Inductive hypothesis suppose true for k-1
  C(2k-1) (k-1) 2k-1
• Just like assume the recursive magic works
• Want to prove true for nk
• C(2k) 2 C(2k-1) 2k
• 2 (k-1) 2k-1 2k
• (k-1) 2k 2k
• k 2k

12
Proof that all horses have same color

• proof by induction on number of horses
• if n1, true
• inductive hypothesis suppose true for n-1 if
  you have n-1 horses, they are all same color
• given n horses in a corral, take 1 out rest must
  have same color
• put it back and take a different one out rest
  must have same color
• therefore they are all the same color
• what is wrong with this?

13
An easier way to think about merge sort

• 1 call to sort array of length n
• 2 calls to sort array of length n/2
• 4 calls to sort array of length n/4
• 8 calls to sort array of length n/8
• n calls to sort array of length 1
• at each level, the merge operations take a total
  of at most n comparisons
• since the number of levels is log2n the total
  number of comparisons is at most n log2n

14
What if n is not a power of 2?

• Does not really matter mid (left
  right)/2 rounds down which is fine
• Number of comparisons is still approximately n
  log2n (which is not an integer)
• We say running time is O(n log2n)
• Base of logarithm doesnt matter much, because
  log10n log2n/log210

15
Quicksort

• Similar recursive idea, but avoids the need for 2
  arrays
• Chooses a pivot element, then divides array into
  two parts, one with elements pivot, and one
  with elements gt pivot
• Possible ways to choose pivot see next page
• To partition the array, loop from left to find
  first element gt pivot, and loop from right to
  find first element lt pivot
• Swap them and repeat
• Place pivot in right place
• Make two recursive calls

16
How to choose the pivot

• Goal want to choose pivot to divide array
  approximately in half, but want to do this fast
• Ideas?
• First position
• middle position
• average value
• median value
• randomly chosen
• median of items in first, middle and last
  positions

17
Number of comparisons for quicksort

• Assuming arrays divided approximately in half
  each time, same as merge sort
• Advantage only one array
• Disadvantage items with equal values may end up
  interchanged from their original value
• Doesnt matter for primitive types, but may
  matter for objects
• For this reason the Arrays.sort methods use
  quicksort for primitive types and merge sort for
  Comparable objects

18
Other sorts

• Heapsort slightly slower than quicksort on
  average but number of comparisons is n log2n even
  in worst case, like merge sort
• Radix sort

19
Big O Notation

• We say a function f(n) is O(g(n)) if there is a
  constant c so that f(n) c g(n) for all n
• Thus we say the number of comparisons for merge
  sort is O(n log n)
• We dont need to write the base
• Which of these functions are O of the others
• log n, 2n, 10n , n2, n3, n, n log n