Advanced Data Structures Fall 20052007

1
Advanced Data Structures(Fall 2005-2007)

• Sorting in Linear Time

Last Modified October 19, 2007, by Cataltepe
2
Sorting

• A number of sorting algorithm that can sort n
  numbers in T(nlog2n) time have been introduced.
• A sorting algorithm of time complexity of T(n)?
• A new approach?
• Previous approach comparison sort
• In insertion sort, merge sort, quicksort,
  heapsort numbers are compared to determine the
  relative order of the numbers

3
Counting Sort

• Assume that n integers in the range of 0 to k
  for some k.

4
Counting sort

• counting_sort(A, B, k)
• for i ? 0 to k
• do Ci ? 0
• for j ? 1 to lengthA
• do CAj ? CAj 1
• / Ci contains of elem. eq. to i /
• for i ? 1 to k
• do Ci ? Ci Ci1
• / Ci cont. of el. eq. to or less than i /
• for j ? lengthA downto 1
• do BCAj ? Aj
• CAj ? CAj 1

5
Counting sort an example
1
2
3
4
5
6
7
8
9
MaxAi5 ? k 5
A
4
5
1
0
1
3
2
4
1
1
2
3
4
5
0
for i ? 0 to k do Ci ? 0
C
0
0
0
0
0
0
for j ? 1 to lengthA do CAj ? CAj
1
C
0
0
0
0
0
0
1
1
1
1
2
1
1
2
3
for i ? 1 to k do Ci ? Ci Ci1
C
1
4
5
6
8
9
1
2
3
4
5
6
7
8
9
A
for j ? lengthA downto 1 do BCAj ?
Aj CAj ? CAj 1
4
5
1
0
1
3
2
4
1
B
-
-
-
-
-
-
-
-
-
1
4
2
3
1
0
1
4
5
1
2
3
4
5
0
C
1
4
5
6
8
9
3
7
4
5
2
0
1
6
8
6

• Counting sort preferred when kO(n). Thus

for i ? 0 to k do Ci ? 0
T(k)
T(n)
for j ? 1 to lengthA do CAj ? CAj
1
T(k)
for i ? 1 to k do Ci ? Ci Ci1
for j ? lengthA downto 1 do BCAj ?
Aj CAj ? CAj 1
T(n)
T(nk)
T(n)
7
RADIX SORT
8
Stability of sorting?

• Stable Elements (numbers) having the same value
  appear in the output list in the same order as
  they do in the input list.
• Important if a satellite data are attached to
  the element being sorted.
• Counting sort is stable !

9
Radix sort

• Origin
• Goes back to the late 19th century Herman
  Holleriths card-sorting
• machine for the 1890 US census
• Approach
• Sort numbers digit-by-digit
• Start with the least significant digit

10
Radix sort
3 2 9 4 5 7 6 5 7 8 3 9 4 3 6 7 2 0 3 5 5
7 2 0 3 5 5 4 3 6 4 5 7 6 5 7 3 2 9 8 3 9
7 2 0 3 2 9 4 3 6 8 3 9 3 5 5 4 5 7 6 5 7
3 2 9 3 5 5 4 3 6 4 5 7 6 5 7 7 2 0 8 3 9
11
Radix Sort

• radix_sort(A,d)
• for i ? 1 to d
• do use a stable sort to sort array A on
  digit i
• Why should it be stable sorting?

12
Inductive analysis of radix sort

• Say it is sorted for the first n digits
  (columns), consider sorting for the n1th digit
  (column)

7 2 0 3 2 9 4 3 6 8 3 9 3 5 5 4 5 7 6 5 7
3 2 9 3 5 5 4 3 6 4 5 7 6 5 7 7 2 0 8 3 9
Case 2 The two number are the same in digit n1
Case 1 The two number differ in digit n1
It is obvious
Sorting must be stable!
13
Running Time of Radix Sort

• Given
• n d-digit number
• each digit can take k possible values
• Radix sort sorts these numbers in T(d(nk)) time

14
Running Time of Radix Sort

• The running time depends upon the intermediate
  stable sorting algorithm run for each digit
• Assume that counting sort is used for
  intermediate sorting
• Each pass over n d-digit number takes T(nk)
  time.
• Since there are d passes the total running time
  for radix sort is T(d(nk))

15
Further Analysis of Radix Sort

• Given
• n b-bit number to sort
• r b
• Radix sort sorts these numbers in T((b/r)(n2r))
  time

16
Further Analysis of Radix Sort

• Words can viewed as having
  digits of r bits each.
• Each digit can be considered as an integer
  between 0 to 2r-1
• i.e. k2r 1
• Each pass of counting sort T(nk) T(n2r)
• There are d passes
• Total running time
• T(d(n2r)) T((b/r)(n2r))

17
Further Analysis of Radix Sort

• Given b and n,
• minimize the running time, T((b/r)(n2r)), where
  r b.
• If any r b
• ? (n2r) T(n)
• if rb, T((b/r)(n2r)) T(n2b) T(n)

18
Further Analysis of Radix Sort

• Given b and n,
• minimize the running time, T((b/r)(n2r)), where
  r b.
• If , gives the best time
• if running time T(bn/log n),
• if running time O(bn/log n),
• if running time T(n),

19
Bucket Sort

• Runs in linear time when the input is drawn from
  a uniform distribution
• Assumption
• Input is uniformly distributed
• Approach
• Divide distribution interval into n equal-sized
  intervals, i.e. buckets.
• Sort the numbers in each bucket
• Go through the buckets in order

20
Bucket Sort Algorithm

• bucket_sort(A) //to sort Ai in 01
• //If your inputs are not in 01
• //you need to change the Bindx computation
• n?lengthA
• for i?1 to n
• do Bindx floor(nAi)
• insert Ai into list BBindx
• for i?0 to n-1
• do sort list Bi with insertion sort
• concatenate the lists B0, B1, ,Bn-1
  together in order

21
Bucket Sort

• List Ptr. to
• Buckets Buckets
• .78 null
• .17 ? .12? .17null
• .39 ? .21? .23? .26null
• .26 ? .39null
• .72 null
• .94 null
• .21 ? .68null
• .12 ? .72? .78null
• .23 null
• .68 ? .94null

22
Running time of Bucket Sort
for i?1 to n do Bindx floor(nAi) insert
Ai into list BBindx
for i?0 to n-1 do sort list Bi with insertion
sort
concatenate the lists B0, B1, ,Bn-1
together in order
23
Bucket Sort

• Taking expected values of both sides
• ? T(n) n O(2 - 1/n) T(n)

Not straightforward, See proof on pp 175-176
24
Data Structures

• You are expected to know at least (but not
  limited to) the fundamental data structures given
  in chapter 10 of Cormens book!
• Skim through the chapter and refresh your
  knowledge on data structures.
• Hands on practice is suggested.