Sorting Algorithms

1
Sorting Algorithms

• Bubble Sort (sinking sort)
• Selection Sort
• Bucket Sort
• Quick Sort
• Heap Sort
• Merge Sort
• Shell Sort

2
Bubble (Sinking) Sort

• for (pass 0 pass
• for (i0 i
• if (ai ai1)
• swap( ai, ai1)

Time Complexity for data size n O(n2)
3
DIY

• Modify the Bubble Sort program (Fig. 7.9) to
  demonstrate the changing of the array contents
• The changing explains why it is called bubble
  sort, or sinking sort.

4
Selection Sort

• SelectionSort(int array, int size)
• if (size1)
• array0 smallest_number( )
• / a function that finds the smallest number in
  the given array /
• SelectionSort(array1, size-1)

Time Complexity for data size n O(n2)
5
Discussion

• What is different between the bubble sort and the
  selection sort?
• Recursive mechanism is used in selection sort
• From the complexity O-notation viewpoint, these
  algorithms are the same inefficient

6
Bucket Sort

• for each digit position k (ones, tens, hundreds,
  thousands, etc.)
• for each value in the array
• place the value in row i based on position ks
  value / a number from 0 to 9 /
• gather the values back to the original array
  from row 0 to 9

7
Bucket Sort Examples (1)
19, 13, 5, 27, 1, 26, 31, 16, 2, 9, 11, 21
1, 31, 11, 21
2
13
5
26, 16
27
1, 31, 11, 21, 2, 13, 5, 26, 16, 27, 19, 9
19, 9
8
Bucket Sort Examples (2)
1, 2, 5, 9
1, 31, 11, 21, 2, 13, 5, 26, 16, 27, 19, 9
11, 13, 16, 19
21, 26, 27
31



1, 2, 5, 9, 11, 13, 16, 19, 21, 26, 27, 31

9
Bucket Sort Examples (3)
Array elements in original order 19, 2, 5,
27, 1, 126, 38, 16, 23, 9, 11, 21 Intermediate
steps 1, 11, 21, 2 , 23, 5, 126, 16, 27, 38,
19, 9 1, 2, 5, 9, 11, 16, 19, 21, 23, 126,
27, 38 1, 2, 5, 9, 11, 16, 19, 21, 23, 27,
38, 126 Array Elements in sorted order 1, 2,
5, 9, 11, 16, 19, 21, 23, 27, 38, 126
O(dn)
Array elements in original order 19, 2, 5, 27,
1000, 126, 38, 16, 23, 9, 11, 21 Intermediate
steps 1000, 11, 21, 2, 23, 5, 126, 16, 27, 38,
19, 9 1000, 2, 5, 9, 11, 16, 19, 21, 23, 126,
27, 38 1000, 2, 5, 9, 11, 16, 19, 21, 23, 27,
38, 126 2, 5, 9, 11, 16, 19, 21, 23, 27, 38,
126, 1000 Array Elements in sorted order 2, 5,
9, 11, 16, 19, 21, 23, 27, 38, 126, 1000
10
Quick Sort (1)

• Invented by C.A.R. Hoare in 1959-1960 when he was
  a British Council Visiting student at Moscow
  State University.
• Original usage use computers to provide mechanic
  translation from one natural language to another.
• Time Complexity in average for QuickSort is O(n
  log n), O(n2) in the worst case.

11
Quick Sort (2)

• The basic idea behind quick sort is to break the
  problem of sorting a sequence of n numbers into
  two smaller sorting problems.
• Can be described in two steps
• 1. Partitioning take the first element of the
  unsorted array and determine its final location
  in the sorted array. We then obtain one element
  is the proper position and two unsorted
  sub-arrays
• 2. Recursive Perform the partitioning step on
  each unsorted sub-arrays.

12
Quick Sort Example (1)
37 2 6 4 89 8 10 12 68 45
Starting from the right most element, compare
eachelement until a number less than 37 is
found. Swap 37 with the element less than 37.
12 2 6 4 89 8 10 37 68 45
Starting from the left most element after 12,
compare eachelement with 37 until a number
greater than 37 is found. Swap 37 with the
element.
12 2 6 4 37 8 10 89 68 45
13
Quick Sort Example (2)
12 2 6 4 37 8 10 89 68 45
Starting from the right most element before 89,
compare eachelement until a number less than 37
is found. Swap 37 with the element.
12 2 6 4 10 8 37 89 68 45
Starting from the left most element after 10,
compare eachelement with 37 until a number
greater than 37 is found. Since there are no
such element found, when compare 37 with itself,
we know 37 is in its final location
14
Quick Sort Examples Results
Initial array 37 2 6 4 89 8 10 12
68 45 Intermediate Steps 12 2 6 4 10
8 37 89 68 45 8 2 6 4 10 12 37
89 68 45 4 2 6 8 10 12 37 89 68
45 2 4 6 8 10 12 37 89 68 45 2
4 6 8 10 12 37 45 68 89 2 4 6
8 10 12 37 45 68 89 The sorted array 2
4 6 8 10 12 37 45 68 89
15
Heap Sort

• A heap is defined to be a complete binary tree
  with the property that the value of each node is
  at least as large as the value of its children
  nodes (if they exist) heap property
• A complete tree means every level of the tree is
  completely filled, except possibly the bottom
  level, and in this level, the nodes are in the
  leftmost positions.
• Every subtree of a heap is a heap
• The root of a heap is the largest key in the heap

9
8
5
6
2
3
16
Heap Sort (2)

• It has been shown that you can make an n-node
  complete tree into a heap in O(n) steps, called
  the sift-up process (Heapify).
• A heap is the basic data structure for the Heap
  Sort algorithm.
• Heap Sort takes O(n log n) time

17
Heap Sort Algorithm

• Take the n keys to be sorted and arrange them in
  a complete binary tree (not necessary a heap)
• Convert this tree into a heap (this takes O(n)
  steps)
• Repeatedly do the following steps until the heap
  is empty
• a) Remove the key at the root of the heap (which
  is the largest in the heap) and place it on an
  output queue
• b) detach from the heap the rightmost leaf node
  at the bottommost level, extract its key K, and
  replace the hey at the root of the heap with K
• c) apply the sift-up process to the root to
  convert the tree to a heap again

18
Heap Sort Example (1)
2
Original tree
Make it into a heap
1
3
6
4
5
Output queue
Replace the root by 3
6
19
Heap Sort Example (2)
Make it into a heap
Output queue
Replace the root by 2
6, 5
20
Heap Sort Example (3)
Make it into a heap
1
Output queue
Replace the root by 1
6, 5, 4
2
3
21
Heap Sort Example (4)
Make it into a heap
Output queue
1
Replace the root by 1
6, 5, 4, 3
2
22
Heap Sort Example (4)
Make it into a heap
Output queue
6, 5, 4, 3, 2
Replace the root by 1
Output queue
1
Empty tree
6, 5, 4, 3, 2, 1
23
Merge Sort

• Combining 2 sorted lists so that the resulting
  list is also sorted

File1 15, 20, 25, 35, 45, 60, 65, 70 File2
10, 30, 40 , 50 , 55
inputs
Read the first element x from File1, and the
first element y from File 2 Repeat until the end
of either File 1 or File2 is reached if x output x to File3 read a new x from File1
else output y to File3 read a new y from
File2 Copy the rest elements in the File
whose end is not reached to File3
Output
File3
24
Discussion

• What is the complexity for the above merge sort
  algorithm?
• What is the prerequisite?
• What about binary merge sort, which can be
  applied to any random list (without sorted)?

25
Binary Merge Sort
F 75, 55, 15, 20, 85, 30, 35, 10
F1 75, 15, 85, 35 F2 55, 20, 30, 10
split
(1)
55, 75
15, 20
30, 85
10, 35
(2)
split
15, 20, 55, 75
10, 30, 35, 85
(3)
split
15, 20, 55, 75
F1
10, 15, 20, 30, 35, 55, 75, 85
10, 30, 35, 85
F2
26
Discussion

• The main concern the size of F must be a power
  of 2
• This leads to the Natural Merge Sort algorithm

27
Natural Merge Sort
O(n log2 n)

• Break the list into segments that are already in
  order

F 75 55 15 20 85 30 35 10
F1 75 15 20 30 35 F2 55 85 10
F 55 75 15 20 85 10 30 35
F1 55 75 10 30 35 F2 15 20 85
F 15 20 55 75 85 10 30 35
F 10 15 20 30 35 55 75 85
28
Inserting Sorts

• The basic idea is to insert new elements into
  ordered list repeatedly insert a new element
  into an ordered list, the resulting list is still
  sorted.
• The binary insertion sort uses a binary search to
  locate the position in the sorted sublist for the
  new element.
• Shell sort (named after Donald Shell) is an
  insertion sort that uses linear insertion sort to
  sort small sublists to produce larger partially
  ordered sublists.

29
Homework

• Figure out the Shell Sort algorithm by yourself
  (text exercise 11.1 12, on page 584). Dont
  need to turn in this Shell Sort homework. But
  you should be responsible for your own work.
• Apply two of the Sorting algorithms (including
  the Shell Sort) you have learned to an
  application (your choice). Write programs to
  measure the performance of the algorithms.
  Compare the performance of different algorithms
  on the selected problem.
• It is expected that you understand every sorting
  algorithm discussed.
• Due April 20

30
Reference
Knuth D.E. The Art of Computer Programming
Vol.3Searching and Sorting, Addison-Wesley, 1973
(include 30 sorting methods) Floyd
R.W. Algorithm 245 Treesort 3, CACM 712,
December 1964, pp 701. Williams J.W.J.
Algorithm 232 Heapsort, CACM 76 June 1964, pp.
347-348