Application Paradigms: Unstructured Grids CS433 Spring 2001

1
Application ParadigmsUnstructured
GridsCS433Spring 2001

• Laxmikant Kale

2
Unstructured Grids

• Typically arise in finite element method
• E.g. Space is tiled with variable-size-and-shape
  triangles
• in 3D may be tetrahedra, or hexahedra
• Allows one to adjust the resolution in different
  regions
• The base data structure is a graph
• Often, represented as bipartite graph
• E.g. Triangles (Elements) and Nodes

3
Unstructured grid computations

• Typically
• Attributes (stresses, strains, pressure,
  temperature, velocities) are attached to nodes
  and elements
• Programs loop over elements and loop over nodes,
  separately
• Each time you visit an element
• Need to access, and possibly modify, all nodes
  connected to it.
• Each time you visit a node
• Typically, access and modify only node attributes
• Rarely access/modify attributes of elements
  connected to it

4
Unstructured grids parallelization issues

• Two concerns
• The unstructured grid graph must be partitioned
  across processors
• vproc (virtual processor, in general)
• Boundary values must be shared
• What to partition and what to duplicate (at the
  boundaries)
• Partition elements (so each element belongs to
  exactly one vproc)
• Share nodes at the boundary
• Each node potentially has several ghost copies
• Why is this better than partitioning nodes, and
  sharing elements?

5
Partitioning unstructured grids

• Not so simple as structured grids
• by rows, by columns, rectangular, .. Dont
  work
• Geometric?
• Applicable only if each node has coordinates
• Even when applicable, may not lead to good
  performance
• What performance metrics to use?
• Load balance the number of elements in each
  partition
• Communication
• Number of shared nodes (Total)
• Maximum number of shared nodes for any one
  partition
• Maximum number of neighbor partitions for any
  partition
• Why? per message cost
• Geometric difficult to optimize both

6
Grid partitioning

• When communication costs are relatively low
• Either because the data-set is large or the
  computation per element is large
• Geometric methods can be used
• Orthogonal Recursive Bisection (ORB)
• Basic idea Recursively divide sets into two
• Keep shapes squarish as long as possible
• For each set
• Find bounding box (Xmax, Xmin, Ymax, Ymin, ..)
• Find the longer dimension (X or Y or ..)
• Find a cut along the longer dimension that will
  divide the set equally
• Doesnt have to be at the midpoint of the section
• Partition the element in the two sets based on
  the cut
• Repeat for each set
• Variation non-power-of-two processors

7
Grid partitioning quad/oct trees

• Another Geometric technique
• At each step, divide the set into 2xD subsets,
  where D is the number of physical dimensions\
• In 2-D 4 quadrants
• Dividing line goes thru geometric midpoint of the
  box.
• Bounding box is NOT recalculated each time in the
  recursion
• Comparison with ORB

8
Grid partitioning Graph partitioners

• CHACO and METIS are well-known programs
• Optimize both load imbalance and communication
  overhead
• But often ignore per-message cost, or the
  maximum-per-partition costs
• Earlier algorithm KR (Kernigham-Ritchie)

9
Crack Propagation

• Explicit FEM code
• Zero-volume Cohesive Elements inserted near the
  crack
• As the crack propagates, more cohesive elements
  added near the crack, which leads to severe load
  imbalance
• Framework handles
• Partitioning elements into chunks
• Communication between chunks
• Load Balancing

Decomposition into 16 chunks (left) and 128
chunks, 8 for each PE (right). The middle area
contains cohesive elements. Pictures S.
Breitenfeld, and P. Geubelle
10
Crack Propagation
Decomposition into 16 chunks (left) and 128
chunks, 8 for each PE (right). The middle area
contains cohesive elements. Both decompositions
obtained using Metis. Pictures S. Breitenfeld,
and P. Geubelle