Space Weather Modeling Framework and BATSRUS

1
Space Weather Modeling Framework and BATS-R-US

• Gábor Tóth
• Tamas Gombosi, Darren De Zeeuw, Ken Powell
• Center for Space Environment Modeling

2
Outline

• Challenges for modeling space weather
• Space Weather Modeling Framework
• BATS-R-US MHD code
• Multifluid MHD algorithm
• Dissemination of codes and algorithms
• Conclusions

3
The Challenge of Scales

• Temporal scale
• 228 2.5x108
• Length scale
• 228 2.5x108
• Volume ratio
• 284 2x1025

4
Space Weather Modeling Framework
5
Coupled Models in SWMF
Physics Domain ID Model(s)

• Solar Corona SC BATS-R-US
• Eruptive Event Generator EE BATS-R-US
• Inner Heliosphere IH BATS-R-US
• Solar Energetic Particles SP Kóta FLAMPA
• Global Magnetosphere GM BATS-R-US
• Inner Magnetosphere IM RCM HEIDI
• Ionosphere Electrodynamics IE RIM
• Upper Atmosphere UA GITM
• Radiation Belt RB RBE SALAMMBO
• Polar Wind PW PWOM
• Plasmasphere PS DGCPM
• Lower Atmosphere LA SOCOL

6
The SWMF Architecture
7
Parallel Layout and Execution
LAYOUT.in for 20 PE-s
SC/IH GM IM/IE
ID ROOT LAST STRIDE COMPONENTMAP SC 0
9 1 IH 0 9 1 GM 10 17
1 IE 18 19 1 IM 19 19
1 END
8
SWMF Code Summary

• Source code
• 263,000 lines of Fortran in the physics models
• 34,000 lines of Fortran 90 in the core of the
  SWMF
• 18,000 lines of IDL plotting scripts
• 13,000 lines of Perl and shell scripts for
  installation, configuration, data conversion,
  parameter checking, etc.
• 11,000 lines of Fortran 90 in the wrappers and
  couplers
• 8,000 lines of XML description of input
  parameters
• 6,000 lines of PHP scripts in the SWMF GUI
• User manual with example runs and full
  documentation of input parameters
• Fully automated nightly testing on 9 different
  machine/compiler combinations
• SWMF runs on any Unix/Linux based system with a
  Fortran 90 compiler, MPI library, and Perl
  interpreter
• SWMF can run on a laptop with one or two
  components and scales well to several hundreds or
  even thousands of processors of the worlds
  fastest supercomputers (like Columbia) with all
  components running together.

9
GUICode Control

• Components of the framework can be easily
  selected for use.
• Some components have multiple versions or even
  alternative models.

10
Parameter Editor GUI
11
GUI Visualization

• Visualize run output using quality templates with
  many customizable options.
• Outputs postscript, png, animated gif,

12
MHD Code BATSRUS
Block Adaptive Tree Solar-wind Roe Upwind Scheme

• Conservative finite-volume discretization
• Shock-capturing TVD schemes
• Parallel block-adaptive grid
• Cartesian and generalized coordinates
• Explicit and implicit time stepping
• Classical, semi-relativistic and Hall MHD
• Multi-species and multi-fluid MHD
• Splitting the magnetic field into B0 B1
• Various methods to control divergence B
• Conservative and/or non-conservative energy
  equation

13
Multifluid MHD Solver

• General equation modules now allows an arbitrary
  number of ion and neutral fluids in BATS-R-US.
• Solve for global ion fluid and individual ion and
  neutral fluids.
• Use conservative scheme for global ion fluid, and
  non-conservative (there is no other choice) for
  the individual ion fluids.
• Distribute total mass and pressure among ion
  fluids proportionally to the individual
  solutions. Distributing momentum is not yet
  working.
• Empty regions where some fluids do not occur are
  very difficult to model.
• Density is set to a small fraction (10-4) of the
  total ion density. Velocity and temperature are
  set to the same as for the total ion fluid.
• Tests suggest that this recipe is sufficiently
  robust.
• Fluids interact via collisions and/or the
  magnetic field
• Source terms can be handled with a general
  point-implicit scheme.
• Or equations can be solved with a fully implicit
  scheme.

14
Parallel Implicit Solver(Toth et al. JCP 217,
722, 2006)

• BDF2 second-order implicit time-stepping scheme
  requires solution of a large nonlinear system of
  equations at each time step.
• Newton linearization allows the nonlinear system
  to be solved by an iterative process in which
  large linear systems are solved.
• Matrix free Jacobian evaluation works for all
  equations and solvers.
• Krylov solvers (GMRES, BiCGSTAB) with
  preconditioning are robust and efficient for
  solving large linear systems.
• Schwarz preconditioning allows the process to be
  done in parallel
• Each adaptive block preconditions using local
  data only
• MBILU preconditioner constructed from numerical
  derivatives (any equation)
• Implicit and explicit schemes can be used
  together on a block-by-block basis.

15
SWMF is Fully Operational

• SWMF is available for the community
• It is publicly available at http//csem.engin.umic
  h.edu/swmf
• It is available in Runs-on-Request mode at CCMC
• There have been gt700 SWMF/BATS-R-US runs at CCMC
• SWMF/BATS-R-US is running 24/7 in real-time mode
  at CCMC since 2002
• SWMF is operating at several partner institutions
  (JPL, Rice, U. Alberta, George Mason U.)
• SWMF is user friendly and well documented
• GUI
• Parameter editor
• 200 page User Manual
• Peer reviewed publications are describing
  methodology and algorithms (available at
  http//csem.engin.umich.edu/pubzilla)

16
SWMF is Fully Operational

• SWMF is available for the community
• It is publicly available at http//csem.engin.umic
  h.edu/swmf
• It is available in Runs-on-Request mode at CCMC
• There have been gt700 SWMF/BATS-R-US runs at CCMC
• SWMF/BATS-R-US is running 24/7 in real-time mode
  at CCMC since 2002
• SWMF is operating at several partner institutions
  (JPL, Rice, U. Alberta, George Mason U.)
• SWMF is user friendly and well documented
• GUI
• Parameter editor
• 200 page User Manual
• Peer reviewed publications are describing
  methodology and algorithms (available at
  http//csem.engin.umich.edu/pubzilla)

17
Summary

• The Space Weather Modeling Framework uses
  efficient and flexible methods to couple and
  execute the physics models.
• BATS-R-US is a state-of-the-art parallel MHD code
  with adaptive grid, implicit time stepping, and
  multi-physics capabilities. Both SWMF and
  BATS-R-US are freely available.
• AISRP helps us improving the physics and the
  algorithms of the BATS-R-US code and in effect of
  the whole SWMF.