Anthony Mezzacappa

1
SciDAC Business as Usual?
Perspectives from the TeraScale Supernova
Initiative

• One Discovery
• Three Examples
• Multi-Physics
• Applied Mathematics
• Computer Science
• and a little Philosophy

2
Scientific Discovery through Advanced Computing
Stationary Accretion Shock Instability (SASI)

• Supernova shock wave may become unstable.
• Instability will
• help drive explosion,
• define explosions shape.
• Operates between the proto-neutron star and the
• supernova shock wave.
• Blondin, Mezzacappa, and DeMarino (2003)

Completely unexpected. Discovered through 2D and
3D (TeraScale) hydrodynamics simulations. 1024-cub
ed problem 1 week on 200 X1
processors. Generating data at the staggering
rate of 500 Mbps (5 TB/day).
3
Initial shock location/strength depend on
knowledge of nuclear states and their
occupation during core collapse.
This is a challenge in nuclear computation being
addressed by TSIs nuclear theorists.
This challenge is exacerbated by the fact that
nuclei increase in size (neutron and proton
number) /complexity (population of states,
collective excitations) during collapse.
4
Significant change in initial shock location and
strength and stellar core profiles when state of
stellar core nuclei computed with more realistic
nuclear models and when this new nuclear physics
is included in the supernova models. Hix et al.
2003, Physical Review Letters, 91,
201102. Langanke et al. 2003, Physical Review
Letters, 90, 241102.
Merger of two fields at their respective states
of the art. (SciDAC enabled.)
5

• Combining Scalable Algorithms and Code
  Performance Analysis
• In conjunction with TOPS, TSI applied
  mathematicians have developed
• scalable preconditioners and solvers for the
  sparse linear systems that
• arise in our neutrino transport solvers.
• 2D/3D Multigroup Flux-Limited Diffusion (MGFLD)
  Transport
• Sparse Approximate Inverse Preconditioner
• Saylor, Smolarski, and Swesty (2004)
• Successfully implemented in 2D MGFLD code (V2D).
• 2D/3D Boltzmann Transport
• ADI Preconditioner
• DAzevedo et al. (2004)
• Successfully implemented in 1D Boltzmann code
  (AGILE-BOLTZTRAN).
• Dense LU factorization was used for dense blocks
  (DAzevedo).
• Being implemented in 2D/3D Boltzmann code
  (GenASiS).
• Sparse incomplete LU factorization for dense
  blocks (DAzevedo, Eijkhout).

6

• As TSI enters production mode managing its
  Workflows has become a paramount issue.
• Ideally, we would like to automate these
  workflows.

Data Management Networking Visualization
These must be viewed together.

• Collaboration between
• SDM
• Arie Shoshani
• Nagiza Samatova
• Guru Kora
• Ian Watkins
• Mladen Vouk
• Networking
• Beck
• Atchley
• Moore
• Rao
• Visualization (TSI)
• Blondin
• Toedte

7
Addressing TSIs Bulk Data Transfer Needs
Current Data Generation Rate 500 Mbps (5 TB/day).

• Logistical Networking provided a
• Light-Weight
• Low-Level
• Deployable solution.
• New Paradigm
• Integrate storage and networking.
• Multi-source, multi-stream.
• Easy for TSI members to share data.

• Data transfer rates 200-300 Mbps using TCP/IP!
• Limit set by ORNL firewall.
• Greater rates expected
• outside firewall,
• other protocols (e.g., Sabul).
• Direct impact on TSIs workflow!

Atchley, Beck, and Moore (2003)
8
Conclusions

• SciDAC has certainly lived up to its name.
• Enabled scientific discovery through
  high-performance computing.
• SciDAC has brought whole communities of
  researchers together.
• Has taken science to an entirely new level of
  realism.
• e.g., state of the art nuclear physics in
  astrophysics models
• Has enabled science that otherwise could not have
  been done.
• e.g., state of the art data management and
  networking technologies
• enabling astrophysics simulations
• Under SciDAC
• We are growing a new community.
• We are growing a new culture.
• What is that feeling?
• The excitement of meeting new people.
• The excitement of learning new things.
• Its the teamwork, the camaraderie.
• The excitement of participating in what is a
  jewel of a program.