Chapter 9 Sorting

1
Chapter 9 Sorting

• Saurav Karmakar
• Spring 2007

2
Sorting

• The need to sort numbers, strings, and other
  records arises frequently in computer
  applications.
• The entries in any modern phone book were sorted
  by a computer.
• Databases have features that sort the records
  returned by a query, ordered according to any
  field the user desires.
• Google sorts your query results by their
  "relevance".
• Weve already seen how searching becomes easier
  for sorted data.
• Sorting is perhaps the simplest fundamental
  problem that offers a large variety of
  algorithms, each with its own inherent advantages
  and disadvantages.

3
Insertion Sort

• The list is assumed to be broken into a sorted
  portion and an unsorted portion
• Keys will be inserted from the unsorted portion
  into the sorted portion.

Unsorted
Sorted
4
Insertion Sort

• For each new key, search backward through sorted
  keys
• Move keys until proper position is found
• Place key in proper position

5
Insertion Sort Code
template ltclass Comparablegt void insertionSort(
vectorltComparablegt a ) for( int p 1 p
lt a.size( ) p ) Comparable tmp
a p int j for( j p j gt 0
tmp lt a j - 1 j-- ) a j a
j - 1 a j tmp
Fixed n-1 iterations
Worst case j-1 comparisons
Searching for the proper position for the new key
Move current key to right
Insert the new key to its proper position
6
Insertion Sort Analysis

• Worst Case Keys are in reverse order
• Do i-1 comparisons for each new key, where i runs
  from 2 to n.
• Total Comparisons 123 n-1

Comparison
7
Optimality Analysis I

• To discover an optimal algorithm we need to find
  an upper and lower asymptotic bound for a
  problem.
• An algorithm gives us an upper bound. The worst
  case for sorting cannot exceed ?(n2) because we
  have Insertion Sort that runs that fast.
• Lower bounds require mathematical arguments.

8
Optimality Analysis II

• Making mathematical arguments usually involves
  assumptions about how the problem will be solved.
• Invalidating the assumptions invalidates the
  lower bound.
• Sorting an array of numbers requires at least
  ?(n) time, because it would take that much time
  to rearrange a list that was rotated one element
  out of position.

9
Rotating One Element
Assumptions Keys must be moved one at a
time All key movements take the same amount of
time The amount of time needed to move one
key is not dependent on n.
2nd
1st
n keys must be moved
3rd
2nd
4th
3rd
?(n) time
nth
n-1st
1st
nth
10
Other Assumptions

• The only operation used for sorting the list is
  swapping two keys.
• Only adjacent keys can be swapped.
• This is true for Insertion Sort and Bubble Sort.

11
Inversions

• Suppose we are given a list of elements L, of
  size n.
• Let i, and j be chosen so 1?iltj?n.
• If LigtLj then the pair (i,j) is an inversion.

12
Maximum Inversions

• The total number of pairs is
• This is the maximum number of inversions in any
  list.
• Exchanging adjacent pairs of keys removes at most
  one inversion.

13
Swapping Adjacent Pairs
The only inversion that could be removed is the
(possible) one between the red and green keys.
The relative position of the Red and blue areas
has not changed. No inversions between the red
key and the blue area have been removed. The same
is true for the red key and the orange area. The
same analysis can be done for the green key.
14
Lower Bound Argument

• A sorted list has no inversions.
• A reverse-order list has the maximum number of
  inversions, ?(n2) inversions.
• A sorting algorithm must exchange ?(n2) adjacent
  pairs to sort a list.
• A sort algorithm that operates by exchanging
  adjacent pairs of keys must have a time bound of
  at least ?(n2).

15
Lower Bound For Average I

• There are n! ways to rearrange a list of n
  elements.
• Recall that a rearrangement is called a
  permutation.
• If we reverse a rearranged list, every pair that
  used to be an inversion will no longer be an
  inversion.
• By the same token, all non-inversions become
  inversions.

16
Lower Bound For Average II

• There are n(n-1)/2 inversions in a permutation
  and its reverse.
• Assuming that all n! permutations are equally
  likely, there are n(n-1)/4 inversions in a
  permutation, on the average.
• The average performance of a swap-adjacent-pairs
  sorting algorithm will be ?(n2).

17
Selection Sort

• Selection sort is equally simple, and also runs
  in quadratic time.
• We walk through unsorted list of items I and pick
  out the smallest item, which we append to the end
  of sorted list S.
• Algorithm
• Start with an empty list S and the unsorted
  list I of n input items.
• for (i 0 i lt n i)
• Let x be the item in I having smallest key.
• Remove x from I.
• Append x to the end of S.
• Whether S is an array or linked list, finding the
  smallest item takes Theta(n) time, so selection
  sort takes Theta(n2) time, even in the best case!
  Hence, it's even worse than insertion sort.

18
Shell Sort

• With insertion sort, each time we insert an
  element, other elements get nudged one step
  closer to where they ought to be
• What if we could move elements a much longer
  distance each time?
• We could move each element
• A long distance
• A somewhat shorter distance
• A more shorter distance next time
• This approach is what makes shellsort so much
  faster than insertion sort

19
Sorting nonconsecutive subarrays
Here is an array to be sorted (numbers arent
important)

• Consider just the red locations
• Suppose we do an insertion sort on just these
  numbers, as if they were the only ones in the
  array?
• Now consider just the yellow locations
• We do an insertion sort on just these numbers
• Now do the same for each additional group of
  numbers
• The resultant array is sorted within groups, but
  not overall

20
Doing the 1-sort

• In the previous slide, we compared numbers that
  were spaced every 5 locations
• This is a 5-sort
• Ordinary insertion sort is just like this, only
  the numbers are spaced 1 apart
• We can think of this as a 1-sort
• Suppose, after doing the 5-sort, we do a 1-sort?
• In general, we would expect that each insertion
  would involve moving fewer numbers out of the way
• The array would end up completely sorted

21
Example of shell sort
original 81 94 11 96 12 35 17 95 28 58 41 75 15
5-sort 35 17 11 28 12 41 75 15 96 58 81 94 95
3-sort 28 12 11 35 15 41 58 17 94 75 81 96 95
1-sort 11 12 15 17 28 35 41 58 75 81 94 95 96
22
Diminishing gaps

• For a large array, we dont want to do a 5-sort
  we want to do an N-sort, where N depends on the
  size of the array
• N is called the gap size, or interval size
• We may want to do several stages, reducing the
  gap size each time
• For example, on a 1000-element array, we may want
  to do a 364-sort, then a 121-sort, then a
  40-sort, then a 13-sort, then a 4-sort, then a
  1-sort
• Why these numbers?

23
Increment sequence

• No one knows the optimal sequence of diminishing
  gaps
• This sequence is attributed to Donald E. Knuth
• Start with h 1
• Repeatedly compute h 3h 1
• 1, 4, 13, 40, 121, 364, 1093
• This sequence seems to work very well
• Another increment sequence mentioned in the
  textbook is based on the following formula
• start with h the half of the containers size
• hi floor (hi-1 / 2.2)
• It turns out that just cutting the array size in
  half each time does not work out as well

24
Analysis

• What is the real running time of shellsort?
• Nobody knows!
• Experiments suggest something like O(n3/2) or
  O(n7/6)
• Analysis isnt always easy!

25
Merge Sort

• If List has only one Element, do nothing
• Otherwise, Split List in Half
• Recursively Sort Both Lists
• Merge Sorted Lists

Mergesort(A, l, r) if l lt r then q
floor((lr)/2) mergesort(A, l, q)
mergesort(A, q1, r)
merge(A, l, q, r)
26
The Merge Algorithm
Assume we are merging lists A and B into list C.
Ax 1 Bx 1 Cx 1 while Ax ? n and Bx ?
n do if AAx lt BBx then CCx
AAx Ax Ax 1 else CCx
BBx Bx Bx 1 endif Cx
Cx 1 endwhile
while only Ax ? n do CCx AAx Ax
Ax 1 Cx Cx 1 endwhile while only
Bx ? n do CCx BBx Bx Bx 1
Cx Cx 1 endwhile
27
Merging

• Merge.
• Keep track of smallest element in each sorted
  half.
• Insert smallest of two elements into auxiliary
  array.
• Repeat until done.

smallest
smallest
auxiliary array









A
28
Merging

• Merge.
• Keep track of smallest element in each sorted
  half.
• Insert smallest of two elements into auxiliary
  array.
• Repeat until done.

auxiliary array
A









G
29
Merging

• Merge.
• Keep track of smallest element in each sorted
  half.
• Insert smallest of two elements into auxiliary
  array.
• Repeat until done.

auxiliary array
A
G








H
30
Merging

• Merge.
• Keep track of smallest element in each sorted
  half.
• Insert smallest of two elements into auxiliary
  array.
• Repeat until done.

auxiliary array
A
G
H







I
31
Merging

• Merge.
• Keep track of smallest element in each sorted
  half.
• Insert smallest of two elements into auxiliary
  array.
• Repeat until done.

auxiliary array
A
G
H
I






L
32
Merging

• Merge.
• Keep track of smallest element in each sorted
  half.
• Insert smallest of two elements into auxiliary
  array.
• Repeat until done.

auxiliary array
A
G
H
I
L





M
33
Merging

• Merge.
• Keep track of smallest element in each sorted
  half.
• Insert smallest of two elements into auxiliary
  array.
• Repeat until done.

auxiliary array
A
G
H
I
L
M




O
34
Merging

• Merge.
• Keep track of smallest element in each sorted
  half.
• Insert smallest of two elements into auxiliary
  array.
• Repeat until done.

auxiliary array
A
G
H
I
L
M
O



R
35
Merging

• Merge.
• Keep track of smallest element in each sorted
  half.
• Insert smallest of two elements into auxiliary
  array.
• Repeat until done.

first halfexhausted
auxiliary array
A
G
H
I
L
M
O
R


S
36
Merging

• Merge.
• Keep track of smallest element in each sorted
  half.
• Insert smallest of two elements into auxiliary
  array.
• Repeat until done.

first halfexhausted
auxiliary array
A
G
H
I
L
M
O
R
S

T
37
Merging

• Merge.
• Keep track of smallest element in each sorted
  half.
• Insert smallest of two elements into auxiliary
  array.
• Repeat until done.

first halfexhausted
second halfexhausted
auxiliary array
A
G
H
I
L
M
O
R
S
T
38
Merging numerical example
39
Merge Sort Analysis

• Sorting requires no comparisons
• Merging requires n-1 comparisons in the worst
  case, where n is the total size of both lists (n
  key movements are required in all cases)
• Recurrence relation

40
Merge Sort Space

• Merging cannot be done in the same place
• In the simplest case, a separate list of size n
  is required for merging
• It is possible to reduce the size of the extra
  space, but it will still be ?(n)

41
Quick Sort

• Pick a pivot x in the set.
• Split List into Big (bigger than x) and
  Little (smaller than x).
• Put the Little group first, Big group second.
• Recursively sort the Big and Little groups.

Quicksort( A, l, r) if l lt r then q
partition(A, l, r) quicksort(A, l,
q-1) quicksort(A, q1, r)
42
Quick Sort

• Big is defined as bigger than the pivot point.
• Little is defined as smaller than the pivot
  point.
• The pivot point is chosen at random (But an
  educated guess following some strategy works
  better). In the following example, we pick up the
  middle element as the pivot.

43
Partitioning
2 97 17 39 12 37 10 55 80 42
46
Pick pivot 37
44
Partitioning
2 97 17 39 12 46 10 55 80 42
37
Step 1 Move pivot to end of array
45
Partitioning
2 97 17 39 12 46 10 55 80 42
37
Step 2 set i 0 and j array.length - 1
46
Partitioning
2 97 17 39 12 46 10 55 80 42
37
Step 3 move i right until value larger than the
pivot is found
47
Partitioning
2 97 17 39 12 46 10 55 80 42
37
Step 4 move j left until value less than the
pivot is found
48
Partitioning
2 10 17 39 12 46 97 55 80 42
37
Step 5 swap elements at positions i and j
49
Partitioning
2 10 17 39 12 46 97 55 80 42
37
Step 6 move i right until value larger than the
pivot is found
50
Partitioning
2 10 17 39 12 46 97 55 80 42
37
Step 7 move j left until value less than the
pivot is found
51
Partitioning
2 10 17 12 39 46 97 55 80 42
37
Step 8 swap elements at positions i and j
52
Partitioning
2 10 17 12 39 46 97 55 80 42
37
Step 9 move i left until it hits j
53
Partitioning
2 10 17 12 37 46 97 55 80 42
39
Step 10 put pivot in correct spot
54
Quicksort Best Case

• Pivot point may not be the exact median
• Finding the precise median is hard
• If we get lucky, the following recurrence
  applies (n/2 is approximate)

55
Quicksort Worst Case

• If the keys are in order, Big portion will have
  n-1 keys, Small portion will be empty.
• T(N) T(N-1) O(N) O(N2)
• N-1 comparisons are done for first key
• N-2 comparisons for second key, etc.
• Result

56
Choosing Pivot

• Wrong Way Choosing extreme elements
• (does not work if elements are presorted
  partly, not total random)
• Safe Way Find middle element.
• Better than Average Choice
• Three Median Partitioning.

57
A Better Lower Bound

• The ?(n2) time bound does not apply to
  Quicksort, Mergesort , on average.
• A better assumption is that keys can be moved an
  arbitrary distance.
• However, we can still assume that the number of
  key-to-key comparisons is proportional to the run
  time of the algorithm.

58
Sorting in Linear Time Counting Sort

• Counting sort assumes that each of the elements
  is an integer in the range 1 to k, for some
  integer k, When k O(n)
• The Counting-sort runs in O(n) time.
• The basic idea of Counting sort is to determine,
  for each input elements x, the number of elements
  less than x.
• This information can be used to place elements
  directly into its correct position.
• For example, if there 17 elements less than x,
  than x belongs in output position 18.

59
Counting Sort

• In the code for Counting sort, we are given array
  
• A1 . . n of length n.
• We required two more arrays, the array
• B1 . . n holds the sorted output and
• the array c1 . . k provides temporary
  working storage.

• COUNTING_SORT (A, B, k)
• for i ? 1 to k do
•     ci ? 0
• for j ? 1 to n do
•     cAj ? cAj 1
• //ci now contains the number of elements equal
  to i
• for i ? 2 to k do
•     ci ? ci ci-1
• // ci now contains the number of elements i
• for j ? n downto 1 do
•     BcAi ? Aj
•     cAi ? cAj - 1

60
Counting Sort
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
C 2 0 2 3 0 1
61
Counting Sort
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
C 2 0 2 3 0 1
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
C 2 2 4 7 7 8
62
Counting Sort
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
B
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
C 2 2 4 7 7 8
63
Counting Sort
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
B 3
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
C 2 2 4 6 7 8
64
Counting Sort
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
B 0 3
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
C 1 2 4 6 7 8
65
Counting Sort
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
B 0 3 3
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
C 1 2 4 5 7 8
66
Counting Sort
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
B 0 2 3 3
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
C 1 2 3 5 7 8
67
Counting Sort
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
B 0 0 2 3 3
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
C 0 2 3 5 7 8
68
Counting Sort
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
B 0 0 2 3 3 3
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
C 0 2 3 4 7 8
69
Counting Sort
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
B 0 0 2 3 3 3 5
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
C 0 2 3 4 7 7
70
Counting Sort
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
B 0 0 2 2 3 3 3 5
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5
C 0 2 2 4 7 7
71
Analysis Counting Sort

• The loop of lines 1-2   takes O(k) time
• The loop of lines 3-4   takes O(n) time
• The loop of lines 6-7   takes O(k) time
• The loop of lines 9-11 takes O(n) time
• Therefore, the overall time of the counting sort
  is O(k) O(n) O(k) O(n) O(k n)
• In practice, we usually use counting sort
  algorithm when have k O(n), in which case
  running time is O(n).

• COUNTING_SORT (A, B, k)
• for i ? 1 to k do
•   ci ? 0
• for j ? 1 to n do
•   cAj ? cAj 1
• //ci now contains the number of elements equal
  to i
• for i ? 2 to k do
• ci ? ci ci-1
• // ci now contains the number of elements i
• for j ? n downto 1 do
• BcAi ? Aj
• cAi ? cAj - 1

72
RADIX SORT

• Radix sort is a small method that many people
  intuitively use when alphabetizing a large list
  of names. (Here Radix is 26, 26 letters of
  alphabet). Specifically, the list of names is
  first sorted according to the first letter of
  each names, that is, the names are arranged in 26
  classes.
• Intuitively, one might want to sort numbers on
  their most significant digit. But Radix sort do
  counter-intuitively by sorting on the least
  significant digits first.
• On the first pass entire numbers sort on the
  least significant digit and combine in a array.
  Then on the second pass, the entire numbers are
  sorted again on the second least-significant
  digits and combine in a array and so on.

73
Radix Sort Example

• INPUT 1st pass 2nd pass 3rd pass

329 720 720 329 457
355 329 355 657 436
436 436 839 457 839
457 436 657 355
657 720 329 457 720 355
839 657 839
74
Analysis

• The running time depends on the stable used as an
  intermediate sorting algorithm.
• When each digits is in the range 1 to k, and k is
  not too large, so COUNTING_SORT  is the obvious
  choice.
• In case of counting sort, each pass over n
  d-digit numbers takes O(n k) time.
• There are d passes, so the total time for for
  each digit sort is O(nk) time.
• There are d passes, so the total time for Radix
  sort is O(dnkd). When d is constant and k
  O(n), the Radix sort runs in linear time.

• Pseudo code
• RADIX_SORT (A, d)
• for i ? 1 to d do    use a stable sort to sort A
  on digit i    // counting sort will do the job
• Consideration n-element array A has d digits,
  where digit 1 is the lowest-order digit and d is
  the highest-order digit