Hypercubes and Their Algorithms

1
Hypercubes and Their Algorithms
by Shietung Peng
2
Introduction

• The hypercube architecture has played an
  important role in the development of parallel
  processing and is still quite popular and
  influential. The logarithmic diameter, linear
  bisection, and highly symmetric recursive
  structure of the hypercube support a variety of
  elegant and efficient parallel algorithms that
  often serve as benchmarks for evaluating
  algorithms on other architectures.

3
Definition

• Hypercubes are also called binary q-cubes, or
  q-cubes, where q indicates the number of
  dimensions.
• A q-cube can be defined recursively as depicted
  below

4
Main Properties

• A node x in a q-cube has a unique label, its
  binary ID, that is a q-bit binary number.
• The labels of any two neighboring nodes differ in
  exactly 1 bit.
• Two nodes whose labels differ in k bits are
  connected by a shortest path of length k (have a
  Hamming distance k).
• Hypercube is both node- and edge- symmetric.
• Any two nodes with Hamming distance k have k
  node-disjoint paths that connect them.
• Some other interesting properties will be
  explored in the exercises.

5
Embeddings

• Given the architectures A and A, embedding A
  into A includes a node mapping and an edge
  mapping (an edge (u,v) in A is mapped into a path
  u ? v in A, where u and v are given by the
  node mapping).
• Embedding a 7-node binary tree into 2D meshes of
  various sizes is depicted below.

6
Embeddings

• In order to gauge the effectiveness of an
  embedding with regard to algorithm performance,
  various measures has been defined.
• The most important ones are listed below with the
  numerical values of the examples in the previous
  page.

7
Embedding of Arrays and Trees

• We show how meshes, tori, and binary trees can be
  embedded into hypercube in such a way as to allow
  a hypercube to run mesh, torus, and tree
  algorithms efficiently, i.e., with very small
  dilation and congestion.
• Any p-node graph that can embed a p-ring with
  dilation and load factor of 1 is said to be
  Hamiltonian.
• To prove the q-cube is Hamiltonian is equivalent
  to construct a q-bit Gray code. A q-bit Gray code
  can be built by induction as follows

8
Cross-product Graphs

• The nodes of the product graph G are labeled with
  k-tuples, where the ith element of the k-tuple is
  chosen from the node set of the ith component
  graph. The edges of the product graph connect
  pairs of nodes whose labels are identical in all
  but the j-th element, and the two nodes
  corresponding to the j-th elements are connected
  by an edge in the j-th component graph.

9
Cross-product Graphs

• Examples of product graphs

10
Embedding of Meshes

• An h-D mesh, where the number of nodes in each
  dimension is a power of 2, is a sub-graph of the
  q-cube.
• The proof is based on product graph and is stated
  as follows

11
Embedding of Meshes

• The 4-by-4 mesh is a sub-graph of the 4-cube as
  depicted below.

12
Embedding of Trees

• The -node double-rooted complete binary tree
  is a sub-graph of the q-cube. The proof is by
  induction, and is depicted by the following
  figure

13
Embedding of Trees

• Embeddings do not have to be 1-to-1 to be
  efficient or useful. For example, an embedding of
  the -leaf complete binary tree in the q-cube
  is shown below. This embedding has dilation 1,
  congestion q, and load factor q1. this embedding
  is efficient if the hypercube is to emulate a
  tree algorithm in which only nodes at a single
  level of the tree are active at any given time.

14
Some Simple Algorithms

• This is an ascend algorithm, each node
  successively communicates with its neighbors in
  dimension order from 0 to q -1.

15
Some Simple Algorithms

• Parallel prefix computation on a 3-cube.

16
Some Simple Algorithms

• A second algorithm for parallel prefix
  computation on the hypercube using a recursive
  formulation.

17
Matrix Multiplication

• Consider the problem of multiplying m-by-m
  matrices A and B on a q-cube, where
  .
• The hypercube algorithm copies the elements of A
  and B into all other processors that need them to
  perform parallel multiplications. A
  recursive doubling scheme is used to make the
  required copies in O(q) steps. This can be done
  by three for loops as shown in the next page. The
  last for loop computes the sum of the m terms
  that form each of the elements of C using
  recursive doubling within (q/3)-dimensional
  subcubes of the original q-cube.
• The running time of the algorithm is O(q) O(lg
  p).

18
Matrix Multiplication
19
Matrix Multiplication

• Multiplying two 2-by-2 matrices on a 3-cube.

20
Exercise 7

• Consider a q-cube, with q even, in which each
  link is replaced by two unidirectional links
  going in opposite directions. and then outgoing
  links along odd (even) dimensions have been
  removed for nodes with odd (even) labels. This
  leads to a unidirectional network with node
  in-degree and out-degree of q/2, called a pruned
  q-cube.
• Find the diameter of such a pruned q-cube.
• Find the bisection width of such a pruned q-cube.

21
Exercise 7

• Embedding mesh into hypercube
• Show that the 3-by-3 mesh is a sub-graph of the
  4-cube.
• Show that 3-by-3 torus is a sub-graph of the
  4-cube.
• Show that the 3-by-5 mesh is not a sub-graph of
  the 4-cube.
• Show an embedding of the 8-by-8-by-8 torus into
  the 8-cube. What is the dilation and congestion
  of the embedding?
• Show that the in-order labeling of the nodes of
  a (p -1)-node complete binary tree corresponds to
  a dilation-2 embedding in the q-cube.

22
Exercise 7

• Develop a hypercube algorithm for solving a
  triangular system of linear equations via back
  substitution.