CSE621 : Parallel Algorithms

1
CSE621Parallel Algorithms Lecture 2Sorting
September 6, 1999
2
Overview

• Review of previous lectures
• Sorting on the CRCW and CREW PRAMs
• Odd-Even Merge Sort on the EREW PRAM
• Sorting on the One-Dimensional Mesh
• Sorting on the Two-Dimensional Mesh
• Sorting Networks
• Summary

3
Summary of previous lecture

• Parallel Algorithm Design Approach
• Constraints in Designing Parallel Algorithm
• Computation Model
• Distributed System Topology
• Examples
• Search on PRAM
• Search on Distributed Memory
• Performance Issues
• Speedup
• Efficiency

4
Sorting on the CRCW and CREW PRAMs

• Sort on the CRCW PRAM
• Similar idea used to design MIN-CRCW
• Need more powerful (less realistic) model of
  resolving concurrent writes that sums the value
  to be concurrently written
• Use n(n-1)/2 processors and in a constant time
• Drawback processors and powerful model
• Sort on the CREW PRAM
• uses an auxiliary two-dimensional array
  Win1n,1n
• separate write and separate sum in O(1) and O(log
  n) using O(n2) processors

5
(No Transcript)
6
Odd-Even Merge Sort on the EREW PRAM

• Sort EREW PRAM model use too many processors
• Merge Sort on the EREW PRAM
• Based on the idea of divide-and-conquer
• Idea divide a list into two sub-lists,
  recursively sort and merge
• Use n processors with complexity Q(n)
• S(n) Q(log n), C(n) Q(n2)
• Odd-Even Merge Sort on the EREW PRAM
• Speedup the Merge process
• Make Odd-index list and Even-index list, and
  recursive merge
• Last do a even-odd compare exchange step

7
Odd-Even Merge Sort on the EREW PRAM contd

• Odd-Even Merge Sort on the EREW PRAM
• number of processors used n
• W(n) 1 2 log n Q((log n)2)
• C(n) Q(n log2n)

8
(No Transcript)
9
(No Transcript)
10
Sorting on the One-dimensional Mesh

• Any comparison-based parallel sorting algorithm
  on Mp must perform at least n-1 communications
  steps to properly decide between the relative
  order of the elements in P1 and Pn.
• Can achieve a speedup of at most log n on the
  one-dimensional mesh
• Two algorithms
• Insertion Sort
• Odd-Even Transposition Sort

11
Sorting on the One-dimensional Mesh Insertion
Sort

• Insertion sort
• Input from the left as a pipeline
• Each processor compares the received value from
  the left with the holding value and transfer the
  larger one to the right.
• W(n) 2n-1
• C(n) n(2n-1)
• S(n) Q(log n)

12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
Sorting on the One-dimensional Mesh Odd-Even
Sort

• Odd-Even Transposition Sort
• Assume that all data are already distributed.
• N iterations of following two parallel steps
• Odd-Even Exchange For all odd i, compare and
  exchange if not ordered
• Even-Odd Exchange For all even I, compare and
  exchange if not ordered
• W(n) n
• C(n) n2
• S(n) Q(log n)

16
(No Transcript)
17
Sorting on the Two-dimensional Mesh

• Order in the two-dimensional mesh
• Row, column-major orders
• Snake order

• Snake-order sorting
• Repetition of row and column sort
• When column sort, the direction is snake-order
  direction
• W(n) (\ceil(log n) 1)\root(n) \root(n)
• C(n) nW(n)
• S(n) \root(n)

18
(No Transcript)
19
(No Transcript)
20
Sorting on the Two-dimensional Mesh contd

• Consideration I Is row-major order sorting
  possible using the same idea?
• Consideration II Why does the repetitive row and
  column sorting work?

21
Sorting Networks

• Made up of simple processors, called comparators
• Construction of sorting networks from Merging
  Networks
• Odd-Even Sorting Network
• Same to the Odd-Even Merge PRAM algorithm
• See the figure
• Bitonic Merge Sort Network
• Same to the Bitonic Merge PRAM algorithm

22
(No Transcript)
23
(No Transcript)
24
Bitonic MergeSort EREW PRAM

• What is Bitonic List?
• A sequence of numbers, x1, , xn, with the
  property that
• there exists an index I such that x1ltx2ltltxi and
  xigtxi1gtgtxn or else
• there exists a cyclic shift of indices so that
  condition (1) holds
• What is rearrangement?
• For X(x1, , xn), X (x1, ., xn) is
  defined as
• xi min (xi, xin/2), xin/2 max(xi,
  xin/2)
• Property
• Let A and B be the sub-lists of X after the
  rearrangement
• Then A and B are bitonic lists.
• And any element in B is larger than all elements
  in A.

25
(No Transcript)
26
Bitonic MergeSort EREW PRAM contd

• Bitonic Sort
• Input A bitonic list
• Output A sorted list
• Algorithm
• Recursive rearrangement
• Bitonic Merge
• Merge of two increasing-order lists as one sorted
  list
• Change the index for rearrangement (i n/2)
  gt (n-i1)
• results are two bitonic sub-lists
• Call Bitonic Sort for two different lists
• Bitonic MergeSort Algorithm
• Input a random list
• Output a sorted list
• Recursive call of Bitonic MergeSort and following
  call of Bitonic Merge

27
(No Transcript)
28
(No Transcript)
29
Bitonic MergeSort EREW PRAM contd

• Bitonic MergeSort Complexity
• W(n) Q((log n)2)
• C(n) Q(n log2 n))
• S(n) Q(n / log n)
• Bitonic Merge Sorting Network
• See the figure

30
(No Transcript)
31
Review of the lecture

• Sorting on the CRCW and CREW PRAMs
• Odd-Even Merge Sort on the EREW PRAM
• Sorting on the One-Dimensional Mesh
• Insertion sort
• Transposition sort
• Sorting on the Two-Dimensional Mesh
• snake-order row-column sort
• Sorting Networks
• odd-even merge sort network
• bitonic sort network