Lecture 23: Sorting

1
Lecture 23Sorting
2
Binary search

• Its called a divide conquer algorithm because
  it divides the problem in half at each step
• First step is whole array, next is 1/2 of it,
  next is 1/4 of that, next is 1/8 of that, etc.b
• This makes time to solution MUCH faster!
• Heres Binary Search in Java

public int binarySearch(int array, int num, int
low, int high) if (low gt high)
return -1 else int mid (low
high) / 2 if (arraymid num)
// num is same as middle number return
mid else if (num lt arraymid)
// num is smaller than middle number
return binarySearch(array, num, low, mid - 1)
else // num is larger than middle
number return binarySearch(array, num,
mid 1, high)
3
How much work is it?

• Each step, we cut the portion we need to search
  in half.
• How many times can divide number in half before
  you get to 1?
• If you start with n, you divide to get n/2 then
  n/4, n/8, ... and eventually get 1.
• Let's suppose that n2k, then divide to 2k-1,
  2k-2, 2k-3, ..., 20 1 divide k times by 2.
• In general, you can divide n by 2 at most k times
  to get to 1, where k log2(n).
• So this is, in the worst case, log2(n) steps of
  work

4
Linear Search vs Binary Search

• Its not so significant at first, but as the
  number of elements gets larger, youre doing much
  less work
• For 1 million elements, linear search could have
  to check all 1,000,000
• Binary Search only has to check 20!
• Caveat remember that binary search only works if
  your numbers are sorted!
• Well learn how to be sure they are put that way
  next time.

5
Theres a catch

• Binary search is really good.
• log2(n) steps is way faster than n steps
• If we plan to do a lot of searches, well want to
  use this instead of linear search almost all the
  time
• But, this requires that your data be sorted!
• How do you make sure your data is sorted?
• Is it fast enough to be worth it?

6
Sorting

• We need some way to sort our arrays
• This way we can use algorithms like binary search
• There are actually lots of algorithms that can be
  made faster with sorted data
• Imagine youre sorting a deck of cards.
• How do you do it?
• Whats your algorithm?

7
Sorting Cards

• You probably
• Search for the first card (say, ace of spades)
• Move it to the top of the deck
• Search for the second card
• Move it to the second place in the deck
• Search for the third card

8
Selection Sort

• The procedure we just went over is actually
  called selection sort
• At each step
• select the item that goes in the next position in
  the array
• put it there
• We want to know 2 things
• How do we write this in Java?
• How fast is it?
• Well measure the number of steps

9
Selection Sort

• Selection Sort is a good recursive algorithm
• Well need
• A base case (when to stop)
• We know that an array of size 0 or 1 is sorted
  correctly
• Some sort of step to reduce the sorting problem
  to a smaller version of the same problem
• When we put one item in its place, weve
  increased the size of the sorted portion of the
  array
• We can look at the remaining portion of the array
  as an unsorted array that is one element smaller

10
Recursive step
Unsorted

• Given an unsorted array, as above, we want to
  sort from least to greatest

11
Recursive step
Unsorted

• Given an unsorted array, as above, we want to
  sort from least to greatest
• Well find the smallest element

12
Recursive step
Unsorted

• Given an unsorted array, as above, we want to
  sort from least to greatest
• Well find the smallest element
• Swap it with the first element

13
Recursive step
Unsorted

• Given an unsorted array, as above, we want to
  sort from least to greatest
• Well find the smallest element
• Swap it with the first element
• Now, the unsorted part of the array is one
  element smaller
• Just call selection sort recursively on the rest
  of the array.

14
Finding the smallest element
public int indexOfSmallest(int array, int
startIndex) int smallIndex startIndex
for (int i startIndex 1 i lt array.length
i) if (arrayi lt arraysmallIndex)
smallIndex i return
smallIndex

• We need to walk through the array from start to
  finish
• Remember the smallest element weve seen so far
• If we see something smaller, remember it instead
• This takes n steps, because we always walk over
  all n items in the array.

15
Finding the smallest element
Smallest so far
public int indexOfSmallest(int array, int
startIndex) int smallIndex startIndex
for (int i startIndex 1 i lt array.length
i) if (arrayi lt arraysmallIndex)
smallIndex i return
smallIndex
16
Finding the smallest element
Smallest so far
Smaller? Yes!
public int indexOfSmallest(int array, int
startIndex) int smallIndex startIndex
for (int i startIndex 1 i lt array.length
i) if (arrayi lt arraysmallIndex)
smallIndex i return
smallIndex
17
Finding the smallest element
Smallest so far
Smaller? No.
public int indexOfSmallest(int array, int
startIndex) int smallIndex startIndex
for (int i startIndex 1 i lt array.length
i) if (arrayi lt arraysmallIndex)
smallIndex i return
smallIndex
18
Finding the smallest element
Smallest so far
Smaller? No.
public int indexOfSmallest(int array, int
startIndex) int smallIndex startIndex
for (int i startIndex 1 i lt array.length
i) if (arrayi lt arraysmallIndex)
smallIndex i return
smallIndex
19
Finding the smallest element
Smallest so far
Smaller? Yes!
public int indexOfSmallest(int array, int
startIndex) int smallIndex startIndex
for (int i startIndex 1 i lt array.length
i) if (arrayi lt arraysmallIndex)
smallIndex i return
smallIndex
20
Finding the smallest element
Smallest so far
Smaller? No.
public int indexOfSmallest(int array, int
startIndex) int smallIndex startIndex
for (int i startIndex 1 i lt array.length
i) if (arrayi lt arraysmallIndex)
smallIndex i return
smallIndex
21
Finding the smallest element
Smallest so far
Smaller? No.
public int indexOfSmallest(int array, int
startIndex) int smallIndex startIndex
for (int i startIndex 1 i lt array.length
i) if (arrayi lt arraysmallIndex)
smallIndex i return
smallIndex
22
Finding the smallest element
Smallest so far
Smaller? No.
public int indexOfSmallest(int array, int
startIndex) int smallIndex startIndex
for (int i startIndex 1 i lt array.length
i) if (arrayi lt arraysmallIndex)
smallIndex i return
smallIndex
23
Finding the smallest element
Smallest so far
Smaller? No.
public int indexOfSmallest(int array, int
startIndex) int smallIndex startIndex
for (int i startIndex 1 i lt array.length
i) if (arrayi lt arraysmallIndex)
smallIndex i return
smallIndex
24
Finding the smallest element
Smallest so far
Smaller? No.
public int indexOfSmallest(int array, int
startIndex) int smallIndex startIndex
for (int i startIndex 1 i lt array.length
i) if (arrayi lt arraysmallIndex)
smallIndex i return
smallIndex
25
Finding the smallest element
Smallest.
Done.
public int indexOfSmallest(int array, int
startIndex) int smallIndex startIndex
for (int i startIndex 1 i lt array.length
i) if (arrayi lt arraysmallIndex)
smallIndex i return
smallIndex
26
Selection Sort

• First method just calls recursive version on the
  entire array
• Just passes a value of 0 for start.
• For convenience.
• selectionSort() method
• runs indexOfSmallest()
• swaps the smallest element with the first
• Runs selection sort on the rest of the array

public void sort(int array)
selectionSort(array, 0) void
selectionSort(int array, int startIndex)
if (startIndex lt array.length - 1) //
find smallest element in rest of array int
smallest indexOfSmallest(array, startIndex)
// move smallest to index startIndex
swap(array, smallest, startIndex) // sort
everything in the array after startIndex
selectionSort(array, startIndex 1)
void swap(int array, int i, int j) int
temp arrayi arrayi arrayj
arrayj temp
27
Swapping
void swap(int array, int i, int j) int
temp arrayi arrayi arrayj
arrayj temp

• Note that when you swap two values you have to
  remember one of them in a temporary variable.
• Why?
• If you dont, youll overwrite and lose one of
  the values.

28
How fast is selection sort?

• Well we know
• indexOfSmallest() takes exactly n steps
• selectionSort is run n times, and indexOfSmallest
  is a part of it.
• As a first guess, we might say that this takes n2
  steps
• You mightve noticed that we keep finding the
  smallest element in progressively smaller arrays
• indexOfSmallest really takes
• n-1 steps the first time
• n-2 steps the second time
• n-3 steps the third time, plus
• . . .
• 1 step the last time

29
How fast is selection sort?

• We can figure this sum out by adding it to
  itself, then dividing by two
• So, 2S (n-1) n
• So the selection sort takes

S (n-1) (n-2) (n-3) ... 3 2
1 S 1 2 3 ... (n-3)
(n-2) (n-1) -------------------------------
------------------------ 2S n n
n ... n n n
steps
30
How fast is selection sort?

• For computer scientists, this is about the same
  as n2 steps
• Its not very fast
• Its only a little less than half of n2 steps
• Half of a very large data set is still very large
• So the answer is, not very fast.

steps
steps
elements in list
31
Insertion Sort

• Insertion sort is a little different from
  Selection Sort
• Like selection sort, maintains sorted elements at
  front of the array
• This time instead of finding smallest element in
  the rest of the array, we just take each
  successive element and insert it in the right
  place in the front of the array
• The front of this is sorted
• We can just take 7 and insert it in the right
  place in the sorted part of the list

Unsorted
32
Insertion Sort

• We wont spend too much time on this
• Its not very good either
• Its about n2, just like selection sort
• These things take forever
• try the demo on the web page!

Unsorted
33
Merge Sort

• This ones trickier
• Well write it recursively, and we have
• Base Case An array of size 0 or 1 is,
  inherently, sorted.
• Neat trick We know we can merge two sorted
  arrays into a sorted array (Ill show you how)
• Recursive Step If we have an array with 2 or
  more elements
• break it into two halves
• sort the halves
• merge the two sorted arrays so that the whole
  thing is sorted.

34
First Merging
private void merge( int array, int left,
int middle, int right, int tempArray )
int indexLeft left int indexRight
middle 1 int target left // Copy
both pieces into tempArray. for (int i
left i lt right i) tempArrayi
arrayi // Merge them together
back in array while there are // elements
left in both halves. while (indexLeft lt
middle indexRight lt right) if
(tempArrayindexLeft lt tempArrayindexRight)
arraytarget tempArrayindexLeft
indexLeft else
arraytarget tempArrayindexRight
indexRight target
// Move any remaining elements from the left
half. while (indexLeft lt middle)
arraytarget tempArrayindexLeft
indexLeft target // Move
any remaining elements from the right half.
while (indexRight lt right) arraytarget
tempArrayindexRight indexRight
target

• Suppose Ive got two sorted arrays
• I can merge them into a single sorted array by
• starting at the beginning of each
• repeatedly
• comparing numbers
• putting the smaller one into the result array
• How fast do you think this is?

35
Merging
Sorted
Sorted
array
//Copy into temporary array for (int i left i
lt right i) tempArrayi arrayi
tempArray

• First, copy everything into a temporary array
• Were going to copy it all back, in order
• We need the temporary array so that we dont
  overwrite things in the original array

36
Merging
0
array
0
1
2
3
4
5
6
7
8
9
copy smaller value
tempArray
compare

• Compare successive values in sorted arrays and
  put the smaller ones into successive positions in
  the original array

37
Merging
0
1
array
0
1
2
3
4
5
6
7
8
9
copy smaller value
tempArray
compare

• Compare successive values in sorted arrays and
  put the smaller ones into successive positions in
  the original array

38
Merging
0
1
2
array
0
1
2
3
4
5
6
7
8
9
copy smaller value
tempArray
compare

• Compare successive values in sorted arrays and
  put the smaller ones into successive positions in
  the original array

39
Merging
0
1
2
3
array
0
1
2
3
4
5
6
7
8
9
copy smaller value
tempArray
compare

• Compare successive values in sorted arrays and
  put the smaller ones into successive positions in
  the original array

40
Merging
0
1
2
3
4
array
0
1
2
3
4
5
6
7
8
9
copy smaller value
tempArray
compare

• Compare successive values in sorted arrays and
  put the smaller ones into successive positions in
  the original array

41
Merging
0
1
2
3
4
5
array
0
1
2
3
4
5
6
7
8
9
copy smaller value
tempArray
compare

• Compare successive values in sorted arrays and
  put the smaller ones into successive positions in
  the original array

42
Merging
0
1
2
3
4
5
6
array
0
1
2
3
4
5
6
7
8
9
copy smaller value
tempArray
compare

• Compare successive values in sorted arrays and
  put the smaller ones into successive positions in
  the original array

43
Merging
0
1
2
3
4
5
6
7
array
0
1
2
3
4
5
6
7
8
9
copy smaller value
tempArray
compare

• Compare successive values in sorted arrays and
  put the smaller ones into successive positions in
  the original array

44
Merging
0
1
2
3
4
5
6
7
8
array
0
1
2
3
4
5
6
7
8
9
copy smaller value
tempArray
compare

• Compare successive values in sorted arrays and
  put the smaller ones into successive positions in
  the original array

45
Merging
Sorted!
0
1
2
3
4
5
6
7
8
9
array
0
1
2
3
4
5
6
7
8
9
copy remaining value
tempArray
compare

• Once one or the other indices runs off the end,
  just copy over the rest of the other array

46
How fast is merging?

• Well count the number of comparisons we do
• Add 1 to number of steps each time we compare 2
  numbers
• At most, we have to do one comparison for each
  element in the original list, except the last one
• So merge is worst case n-1 compares
• The example I showed you took n-1 comparisons
• We have to do 2n copies of elements
• n copies from the original array to the temporary
  array
• n copies back to original array from temp array
• Total weve got 3n-1 operations
• So this is linear
• Well just say its about n operations

47
Now Merge Sort

• The sort() method is for convenience
• creates temporary array
• calls mergeSort with correct bounds
• Repeatedly
• break array in half
• Sort the halves
• Merge them together
• Recursion stops when arrays are 0 or 1 elements
  long
• This is the base case
• We know that 0 or 1 element arrays are sorted

public void sort(int array) // create
tempArray for use in merging int tempArray
new intarray.length mergeSort(array, 0,
array.length-1, tempArray) public void
mergeSort( int array, int left, int right,
int tempArray ) if (left lt right) int
middle (right left) / 2 mergeSort(array,
left, middle, tempArray) mergeSort(array,
middle 1, right, tempArray) merge(array,
left, middle, right, tempArray)
48
Dividing
5
3
4
9
0
2
1
7
8
6
49
Dividing
50
Dividing
51
Dividing
5
3
4
9
0
2
1
7
8
6
52
Dividing
5
3
4
9
0
2
1
7
8
6
5
3
2
1

• Now all the subarrays on the bottom are sorted

53
Merging
3
5
4
9
0
1
2
7
8
6
5
3
2
1
54
Merging
3
5
4
9
0
1
2
7
8
6
55
Merging
56
Merging

• Done. The Arrays sorted now.
• But how much work did we do?

57
How fast is Merge Sort?

• On each level of this tree, were merging about n
  elements
• We already said that this takes about n steps
• How deep is the tree?
• We repeatedly divide by 2, so the tree is going
  to be log2(n) levels high

58
How fast is Merge Sort?

• So, complexity of merge sort is, worst case,
  n(log2(n)) steps
• This is MUCH better!
• Lets compare to selection sort and selection sort

59
How fast is Merge Sort?

• This is much faster than n2 sorts (insertion,
  selection)
• n2 curve grows much faster
• n(log2(n)) is also provably optimal complexity
  for sorting
• Upper bound on speed of as a general sort
• See how fast for yourself in the demo on the
  website.