http://www.phy.ornl.gov/tsi/

1
TeraScale Supernova Initiative
http//www.phy.ornl.gov/tsi/
2
Investigator Team

• Cross-Cutting Team
• Long-Term Collaborations
• Structured like SciDAC

TOPS

• Linear System/Eigenvalue Problem Solution
  Algorithms for Radiation Transport and Nuclear
  Structure Computation
• Dongarra (UT, ORNL)
• Saied (UIUC, NCSA)
• Saylor (UIUC, NCSA)

• Radiation Transport/
• Radiation Hydrodynamics
• Blondin (NC State)
• Bruenn (FAU)
• Hayes (UCSD)
• Mezzacappa (ORNL)
• Swesty (SUNYSB)

• Supernova Science
• Blondin
• Bruenn
• Fuller
• Haxton
• Hayes
• Lattimer
• Meyer (Clemson)
• Mezzacappa
• Swesty

TOPS
CCA PERC TSTT

• Nuclear Structure Computations
• for EOS and Neutrino-Nucleus/
• Nucleon Interactions
• Dean (ORNL, UT)
• Fuller (UCSD)
• Haxton (INT, Washington)
• Lattimer (SUNYSB)
• Prakash (SUNYSB)
• Strayer (ORNL, UT)

SDM

• Visualization
• Baker (NCSA)
• Toedte (ORNL)

3

• Goal
• Ascertain the explosion mechanism(s).
• Reproduce supernova phenomenology (element
  synthesis neutrino,
• gravitational wave, and gamma ray signatures
  neutron star kicks
• gamma ray burst connection)

• Relevance
• Dominant source of many elements in the
  Universe.
• Given sufficiently well developed models, serve
  as laboratories for
• fundamental nuclear and particle physics
  that cannot be explored
• in terrestrial laboratories.
• Driving application in computational science
  (radiation transport,
• hydrodynamics, nuclear physics, applied
  mathematics, computer
• science, visualization).

• Paradigm
• Result from stellar core collapse and
• bounce in massive stars.
• Radiatively driven (perhaps some are
• MHD driven, or both).

4
Convection

• Need Boltzmann Solution
• Need Angular Distribution
• Need Spectrum
• Gray Schemes Inadequate
• Spectrum Imposed
• Limited Angular Information
• (Few Moments)
• Parameterized
• (No First Principle Solution)
• The bar is high! (10 effects can
• make or break explosions.)

5
1D
1D
0D
0D
Neutrino Energy
Lightbulb
FLD
MGFLD
MGBT
D
Space
Burrows, Hayes, and Fryxell
Janka and Mueller
Mezzacappa et al.
1D
Herant et al.
TSI Year 1
TSI Year 2
2D
Swesty
Fryer and Heger
Past Transport in 2D Models D Diffusion FLD
Flux-Limited Diffusion MGFLD Multigroup
FLD MGBT Boltzmann Transport
TSI Year 3
TSI Year 2
3D
Gray Models
6

• Latest TSI 2D/3D Models
• Hydrodynamics only.
• Focused on understanding 2D/3D flow and its
• coupling to shock wave.
• Convectively stable.
• 2D model exhibits bipolar explosion (due to
• nonlinear flow-shock interaction).
• 3D model exhibits similar long-wavelength
• behavior. Key finding.
• New rolling flows identified.
• AAS Meeting Ap.J. Submitted

2D Model
3D Model
7
Explosion Mechanism Open Questions

• What is the Recipe for Explosion?

Neutrino Heating
Convection
General Relativity
Rotation
Magnetic Fields

• Are there multiple mechanisms?
• Neutrino-driven supernovae
• MHD-driven supernovae
• Supernovae driven by both neutrinos and MHD
  effects

• One mechanism for a class of stars?

• Is the mechanism tailored to the individual star?

8
Nuclear and Weak Interaction Physics Needs
High-Density EoS
Nuclear MatterOpacities
Thomas Fermi (Classical)
Classical treatment of many-body problem.
Ensembles
Hartree-Fock
Density of States
Lowest order solution to the quantum mechanical
many-body problem.
e-capture
n-nucleus
Shell Model Monte Carlo
b-decay
Shell Model Diagonalization
Time
Advanced solutions to the many body problem.
Bloch-Horowitz
Solve exact many-body problem.
9
Supernova Nucleosynthesis
R-Process Breakthrough

• r-process can occur in symmetric environment
  (equal numbers of protons and
• neutrons) under certain conditions (high
  entropy, fast expansion).
• Meyer, PRL Submitted

10
Supernova Science
Hydrodynamics Explicit Differencing Reactive
Flows Newtonian General Relativistic
Nuclear, Weak Interaction Physics Thermodynamics (
Composition), Neutrino Sources and Interactions
Radiation Transport Implicit Differencing MGFLD Pr
econditioners Sparse System Solvers MGBT Precondit
ioners Sparse System Solvers (Matrix Free)
11
Integration of Technologies
Generation 3
High-Resolution 3D MGFLD with Full Integration
of Components (Ensemble of Nuclei, State of the
Art Neutrino-Matter Interactions, ...)
Increasing Integration
Generation 2
Inclusion of state of the art neutrino
interactions in Generation 1 MGBT/ MGFLD
Simulations

Generation 1
Increasing Integration
2D MGFLD Simulation with Naive Neutrino
Interactions and Single-Nucleus Equation of
State Computation of State of the Art
Neutrino-Matter Interactions
12
Supernova Simulation Timeline
Year 0
Year 1
Year 2
Year 3
Year 4
Year 5
3D MGFLD Models w/ AMR (4D)
2D MGFLD Models (3D)
3D MGFLD Models (4D)
2D Boltzmann Models (5D)
1D Boltzmann Models (3D)
3D Boltzmann Models (6D)
13
ISIC Collaborations TOPS

• Nonlinear Algebraic Equations
• Linearize
• Solve via Multi-D Newton-Raphson Method
• Large Sparse Linear Systems

Boltzmann Equation nonlinear integro-PDE

• Implicit Time Differencing
• Extremely Short Neutrino-Matter
• Coupling Time Scales
• Neutrino-Matter Equilibration
• Neutrino Transport Time Scales

Memory Requirements (assuming matrix-free
methods) 10s Gb up to 1/2 Tb
Progress Sparse Approximate Inverses for 2D
MGFLD (Saylor, Smolarski, Swesty J. Comp.
Phys.) ADI-Like Preconditioner for Boltzmann
Transport (DAzevedo et al. Precond 2001,
NLAA) AGILE-BOLTZRAN, V2D codes turned over to
TOPS for analysis and development.
14
ISIC Collaborations CCTTSS

• TSI Code
• F90 MPI Code
• Object-Oriented Design for Interoperability and
  Reuse
• Application Framework
• IBEAM Interoperability Based Environment for
  Adaptive Meshes
• NASA HPCC-Funded Project (PI Swesty)
• AMR PARAMESH

Goal Develop our framework to be
CCA-compliant. Initiated discussions with ANL,
LLNL, and ORNL members of CCTTSS.
15
ISIC Collaborations PERC

• Assess Code Performance on Parallel Platforms
• Identify Code Optimizations to Increase
  Performance

• TSI Code Suite
• Hydrodynamics
• VH-1 (PPM)
• ZEPHYR (Finite Difference)
• Neutrino Transport
• AGILE-BOLTZTRAN 1D General Relativistic Adaptive
  Mesh
• Hydrodynamics with 1D Boltzmann
  Transport
• V2D 2D MGFLD Transport Code
• V3D 3D MGFLD Transport Code (Under Development)
• 2D/3D Boltzmann Code (Under Development)

VH-1 numerical hydrodynamics algorithm scales
well.
Results for VH-1
16
ISIC Collaborations SDM

• Use PROBE environment for staging data between
  simulation platforms and
• end-user visualization platforms.
• Develop new data analysis techniques/tools
  tailored to our application, allowing
• (a) data reduction and (b) discovery
  potential.
• Use of agent technology for distributed data
  analysis (data analysis must be
• done in parallel to achieve reasonable
  throughputs).

17
ISIC Collaborations TSTT
Adaptive Quadratures (Direction Cosines) for
Multidimensional Radiation Transport

• Greatest challenge to completing 3D Boltzmann
• simulations is memory.
• Minimize number of quadratures to minimize
• memory needs while maintaining physical
• resolution. (Also important for 1D/2D MGBT.)
• Optimization Problem

Results for 1D Boltzmann Transport on Milne
Problem (DAzevedo)
Extended Core
Compact Core
18
Collaboration with Supporting Base Projects
Networking

• Identify Optimal Paths in Our Collaborative
  Visualization Server-Client Model
• Maximize Bandwidth along these Paths (Not
  Achieved Using Current Protocols)

• Participated in ORNL Workshop on DoE
  High-Performance Network RD
• and Applications
• Convey TSI Needs to Networking Team
• Participate in White Paper to Define and
  Develop Interface between Efforts