Radix and Bucket Sort

1
Radix and Bucket Sort

• Rekha Saripella
• CS566

2
History of Sorting

• Herman Hollerith (February 29, 1860 November
  17, 1929) is first known to have generated an
  algorithm similar to Radix sort.
• He was the son of German immigrants, born in
  Buffalo, New York and was a Census Statistician.
  He developed a Punch Card Tabulating Machine.
• Holleriths machine included punch, tabulator and
  sorter, and was used to generate the official
  1890 population census. The census took six
  months, and in another two years, all the census
  data was completed and defined.
• Hollerith formed the Tabulating Machine Company
  in 1896. The company merged with International
  Time Recording Company and Computing Scale
  Company to form Computer Tabulating Recording
  Company (CTR) in 1911. CTR was IBM's predecessor.
  CTR was renamed International Business Machines
  Corporation in 1924.
• Hollerith served as a consulting engineer with
  CTR until retiring in 1921.
• There are references to Harold H.Seward, a
  computer scientist, as being the developer of
  Radix sort in 1954 at MIT. He also developed the
  Counting sort.

3
History of Sortingcontd

• Quicksort algorithm was developed in 1960 by Sir
  Charles Antony Richard Hoare (Tony Hoare or
  C.A.R. Hoare, born January 11, 1934) while
  working at Elliot Brothers Ltd. in the UK.
• He also developed Hoare logic, and Communicating
  Sequential Processes (CSP), a formal language
  used to specify the interactions of concurrent
  processes.

Herman Hollerith
Sir Charles Antony Richard Hoare
4
Introduction to Sorting

• Sorting is the fundamental algorithmic problem in
  mathematics and computer science.
• It puts elements in a certain order. The most
  commonly used orders are numerical and
  lexicographical(alphabetical) order.
• Efficient sorting is important to optimize the
  use of other algorithms, as it is the first step
  in most of them.
• There are many sorting algorithms, but knowing
  which one to use depends on the specific problem
  at hand.
• Some factors that help decide the sort to use
  are

5
Introduction to Sortingcontd

• How many elements need to be sorted?
• Will there be duplicate elements in the data?
• If there are duplicate items in the array, does
  their order need to be maintained after sorting ?
• What do we know about the distribution of
  elements? Are they partly ordered, or totally
  random ? Based on the execution times of
  available sorting algorithms, we can decide which
  sorts should or should not be used. In class,
  weve seen that quick sort can be much worse than
  O(n2) if used to sort elements that are
  partially or nearly ordered.
• What resources are available for executing sorts
  ? Can we use more memory, more number of
  processors ?
• Most of the time, we do not know enough
  information about the elements to be sorted. In
  such cases, we need to look at the existing
  sorting algorithms, and figure out which one
  would be a good match.
• An algorithm whose worst case execution time is
  acceptable may be chosen when instance details
  are not known.

6
Classification of Sorting algorithms

• Sorting algorithms are often classified using
  different metrics
• Computational complexity classification is based
  on worst, average and best behavior of sorting a
  list of size (n).
• For typical sorting algorithms acceptable/good
  behavior is O(n log n) and unacceptable/bad
  behavior is O(n2).
• Ideal behavior for a sort is O(n).
• Memory usage (and use of other computer
  resources)
• Some sorting algorithms are in place", such that
  only O(1) or O(log n) memory is needed beyond the
  items being sorted.
• Others need to create auxiliary data structures
  for data to be temporarily stored. Weve seen in
  class that mergesort needs more memory resources
  as it is not an in place algorithm, while
  quicksort and heapsort are in place. Radix and
  bucket sorts are not in place.
• Recursion some algorithms are either recursive
  or non-recursive.(e.g., mergesort is recursive).
• Stability stable sorting algorithms maintain the
  relative order of elements/records with equal
  keys/values. Radix and bucket sorts are stable.
• General method classification is based on how
  sort functions internally.
• Methods used internally include insertion,
  exchange, selection, merging, distribution etc.
  Bubble sort and quicksort are exchange sorts.
  Heapsort is a selection sort.

7
Classification of Sorting algorithmscontd

• Comparison sorts A comparison sort examines
  elements with a comparison operator, which
  usually is the less than or equal to operator().
  Comparison sorts include
• Bubble sort
• Insertion sort
• Selection sort
• Shell sort
• Heapsort
• Mergesort
• Quicksort.
• Non-Comparison sorts these use other techniques
  to sort data, rather than using comparison
  operations. These include
• Radix sort (examines individual bits of keys)
• Bucket sort (examines bits of keys)
• Counting sort (indexes using key values)

8
Radix Sort

• Radix is the base of a number system or
  logarithm.
• Radix sort is a multiple pass distribution sort.
• It distributes each item to a bucket according to
  part of the item's key.
• After each pass, items are collected from the
  buckets, keeping the items in order, then
  redistributed according to the next most
  significant part of the key.
• This sorts keys digit-by-digit (hence referred to
  as digital sort), or, if the keys are strings
  that we want to sort alphabetically, it sorts
  character-by-character.
• It was used in card-sorting machines.
• Radix sort uses bucket or count sort as the
  stable sorting algorithm, where the initial
  relative order of equal keys is unchanged.
• Integer representations can be used to represent
  strings of characters as well as integers. So,
  anything that can be represented by integers can
  be rearranged to be in order by a radix sort.
• Execution of Radix sort is in ?(d(n k)), where
  n is instance size or number of elements that
  need to be sorted. k is the number of buckets
  that can be generated and d is the number of
  digits in the element, or length of the keys.

9
Classification of Radix Sort

• Radix sort is classified based on how it works
  internally
• least significant digit (LSD) radix sort
  processing starts from the least significant
  digit and moves towards the most significant
  digit.
• most significant digit (MSD) radix sort
  processing starts from the most significant digit
  and moves towards the least significant digit.
  This is recursive. It works in the following way
• If we are sorting strings, we would create a
  bucket for a,b,c upto z.
• After the first pass, strings are roughly sorted
  in that any two strings that begin with different
  letters are in the correct order.
• If a bucket has more than one string, its
  elements are recursively sorted (sorting into
  buckets by the next most significant character).
• Contents of buckets are concatenated.
• The differences between LSD and MSD radix sorts
  are
• In MSD, if we know the minimum number of
  characters needed to distinguish all the strings,
  we can only sort these number of characters. So,
  if the strings are long, but we can distinguish
  them all by just looking at the first three
  characters, then we can sort 3 instead of the
  length of the keys.

10
Classification of Radix Sortcontd

• LSD approach requires padding short keys if key
  length is variable, and guarantees that all
  digits will be examined even if the first 3-4
  digits contain all the information needed to
  achieve sorted order.
• MSD is recursive. LSD is non-recursive.
• MSD radix sort requires much more memory to sort
  elements. LSD radix sort is the preferred
  implementation between the two.
• MSD recursive radix sorting has applications to
  parallel computing, as each of the sub-buckets
  can be sorted independently of the rest. Each
  recursion can be passed to the next available
  processor.
• The Postman's sort is a variant of MSD radix sort
  where attributes of the key are described so the
  algorithm can allocate buckets efficiently. This
  is the algorithm used by letter-sorting machines
  in the post office first states, then post
  offices, then routes, etc. The smaller buckets
  are then recursively sorted.
• Lets look at an example of LSD Radix sort.

11
Example of LSD-Radix Sort
Input is an array of 15 integers. For integers,
the number of buckets is 10, from 0 to 9. The
first pass distributes the keys into buckets by
the least significant digit (LSD). When the first
pass is done, we have the following.
0 1 2 3 4
5 6 7 8 9
12
Example of LSD-Radix Sortcontd

We collect these, keeping their relative order
Now we distribute by the next most significant
digit, which is the highest digit in our example,
and we get the following.
0 1 2 3 4
5 6 7 8 9
When we collect them, they are in order.
13
Radix Sort

• Running time for this example is
• T(n) ? (d(nk))
• k number of buckets 10(0 to 9).
• n number of elements to be sorted 15
• d digits or maximum length of element 2
• Thus in our example, the algorithm will take
• T(n) ? (d(nk))
• ? (2(1510))
• ? (50) execution time.
• Pseudo code of Radix sort is

14
Bucket Sort

• Bucket sort, or bin sort, is a distribution
  sorting algorithm.
• It is a generalization of Counting sort, and
  works on the assumption that keys to be sorted
  are uniformly distributed over a known range (say
  1 to m).
• It is a stable sort, where the relative order of
  any two items with the same key is preserved.
• It works in the following way
• set up m buckets where each bucket is responsible
  for an equal portion of the range of keys in the
  array.
• place items in appropriate buckets.
• sort items in each non-empty bucket using
  insertion sort.
• concatenate sorted lists of items from buckets to
  get final sorted order.
• Analysis of running time of Bucket sort
• Buckets are created based on the range of
  elements in the array. This is a linear time
  operation.
• Each element is placed in its corresponding
  bucket, which takes linear time.
• Insertion sort takes a quadratic time to run.
• Concatenating sorted lists takes a linear time.

15
Bucket Sortcontd

• Execution time for Bucket sort is
• ?(n) for all the linear operations O(n2) time
  taken for insertion sort in each bucket.
• n-1
• T(n) ?(n) S O(n2)
• i0
• Using mathematical solutions, the above running
  time comes to be linear.
• Running time of bucket sort is usually expressed
  as
• T(n) O(mn) where
• m is the range of input values
• n is the number of elements in the array.
• If the range is in order of n, then bucket sort
  is linear. But if range is large, then sort may
  be worse than quadratic.

16
Example of Bucket Sort

The example uses an input array of 9 elements.
Key values are in the range from 10 to 19. It
uses an auxiliary array of linked lists which is
used as buckets. Items are placed in appropriate
buckets and links are maintained to point to the
next element. Order of the two keys with value 15
is maintained after sorting.
17
Bucket Sort

• Pseudo code of Bucket sort is

18
Advantages and Disadvantages

• Advantages
• Radix and bucket sorts are stable, preserving
  existing order of equal keys.
• They work in linear time, unlike most other
  sorts. In other words, they do not bog down when
  large numbers of items need to be sorted. Most
  sorts run in O(n log n) or O(n2) time.
• The time to sort per item is constant, as no
  comparisons among items are made. With other
  sorts, the time to sort per time increases with
  the number of items.
• Radix sort is particularly efficient when you
  have large numbers of records to sort with short
  keys.
• Drawbacks
• Radix and bucket sorts do not work well when keys
  are very long, as the total sorting time is
  proportional to key length and to the number of
  items to sort.
• They are not in-place, using more working
  memory than a traditional sort.

19
Addendum Count sort

• Count sort is a sorting algorithm that takes
  linear time ?(n), which is the best possible
  performance for a sorting algorithm.
• It assumes that each of the n input elements is
  an integer in the range 0 to k, where k is an
  integer. When k O(n), the sort runs in ?(n)
  time.
• This is a stable, non-comparison sort.
• It works as follows
• Set up an array of initially empty values, its
  length being the range of keys in input array.
  This is the count array.
• Suppose input array 0,5,2,8,3,1,0,4
• Count array size 9, and has placeholders for
  occurrences of keys from 0 (minimum element
  value) to 8 (maximum element value).
• Each element in count array will store the number
  of times elements occur in input array, starting
  from least key value to the maximum key value.
• Go over the input array, counting occurrences of
  elements. Populate count array with counts of the
  elements.
• After population, count array
  2,1,1,1,1,1,0,0,1
• Iterate over input array in order, and put
  elements from input array into the result array,
  using count array for the number of occurrences.

20
Addendum Count sortcontd

• Count sort uses auxiliary data structures
  internally (for count array and result array),
  and is a resource-intensive algorithm.
• Pseudo code of Count sort is

21
Addendum - some uses of Sorting

• Indexes in relational databases.
• Since index entries are stored in sorted order,
  indexes help in processing database operations
  and queries. Without an index the database has to
  load records and sort them during execution. An
  index on keys will allow the database to simply
  scan the index and fetch rows as they are
  referenced. To order records in descending order,
  the database can simply scan the index in
  reverse.
• File comparisons.
• Data in files is first sorted, and then
  occurrences in both files are compared and
  matched.
• Grouping items.
• Items with the same identification are grouped
  together using sorting. This rearrangement of
  data allows for better identification of the
  data, and aids in statistical studies.

22
Bibliography and References

• http//www.cs.umass.edu/immerman/cs311/applets/vi
  shal/RadixSort.html - demonstration of Radix
  Sort.
• http//users.cs.cf.ac.uk/C.L.Mumford/tristan/Count
  ingSort.html - demonstration of Count sort.
• Art of Programming Volume 3 by Donald Knuth.
• Introduction to Algorithms by Thomas H. Cormen,
  Charles E. Leiserson, Ronald L. Rivest, Clifford
  Stein.
• http//www.cs.ubc.ca/harrison/Java/sorting-demo.h
  tml - demonstration of different sorting
  algorithms by James Gosling, Jason Harrison, Jack
  Snoeyink. Jim Boritz, Denis Ahrens, Alvin Raj
• http//en.wikipedia.org/wiki/Sorting_algorithm
• http//www.cs.cmu.edu/adityaa/211/Lecture12AG.pdf
  - Introduction to Sorting
• http//www-03.ibm.com/ibm/history/history/year_191
  1.html - History of IBM
• http//www.w3c.rl.ac.uk/pasttalks/A_Timeline_of_Co
  mputing.html - timeline of computing history.
• http//www.nist.gov/dads/HTML/radixsort.html -
  radix sort
• http//www.cs.purdue.edu/homes/ayg/CS251/slides/ch
  ap8c.pdf - Radix and Bucket sorts.
• http//www.cse.iitk.ac.in/users/dsrkg/cs210/applet
  s/sortingII/radixSort/radix.html - Radix sort.
• http//www.cs.cmu.edu/afs/cs.cmu.edu/academic/clas
  s/15451-s07/www/lecture_notes/lect0213.pdf -
  Radix and Bucket sorts.
• http//www.cs.cmu.edu/afs/cs.cmu.edu/academic/clas
  s/15451-f03/www/lectures/lect0923.txt - Radix and
  Bucket sorts.
• http//www.cs.berkeley.edu/kamil/sp03/042803.pdf
  - Sorting.