M' Satoh

1
Next generation climate model Development of the
nonhydrostatic icosahedral atmospheric model at
Frontier Research System for Global Change
M. Satoh H. Tomita K. Goto The Integrated
Modeling Research Program Frontier Research
System for Global Change
EU-Japan Symposium on Climate Research 4-5 March
2003, Brussels, Belgium
2
Contents
Nonhydrostatic ICosahedral Atmospheric Model
(NICAM)

• Introduction
• Nonhydrostatic modeling
• Icosahedral grid modeling
• Runs on ES

3
Motivation

• Mission
• Development of a high resolution atmospheric
  global model on the Earth Simulator
• 10 km or less in horizontal, 100 levels in
  vertical
• Cloud resolving global model
• Climate study
• Strategy of development
• Quasi-uniform grid the icosahedral grid
• Spectral method is inefficient in high resolution
  simulations.
• Legendre transformation
• Massive data transfer between computer nodes
• The latitude-longitude grid point method has the
  pole problem.
• Severe limitation of time interval by the CFL
  condition.
• Inhomogeneous near the poles.
• Non-hydrostatic equations system a new
  conservative scheme

4
Development procedure
Coupling with Ocean, Land, Ice, Bio models
Computational tuning on ES
2003-
2002
Non-hydrostatic Global Climate Model Dynamical
Core Physics
Dynamical Core of Global Non?hydrostatic
Model Icosahedral grid Nonhydrostatic equations
2000-01
Global Shallow Water Model Icosahedral or
Conformal cubic grids
Regional Cartesian Nonhydrostatic model New
dynamical scheme
StrechedRegionalClimateModel
Study of physical processes using regional
nonhydrostatic model
Physical tuning
5
Outline of a nonhydrostatic modeling
Non-hydrostatic modeling

• Characteristics of the non-hydrostatic model
• Dry formulation and results
• Moist formulation
• Squall line experiments

6
Characteristics of the nonhydrostatic model (1)

• Fully compressible non-hydrostatic equations
• Horizontally explicit and vertically implicit
  time integration with time splitting
• The Helmholtz equation is formulated for vertical
  velocity not for pressure
• a switch for a hydrostatic/non-hydrostatic option
  can be introduced.
• Conservation of the domain integrals.
• The finite volume method using flux form
  equations of density, momentum and total energy.
• Tracer advecion
• Third order upwind, or UTOPIA
• Consistency with Continuity
• Exact treatment of moist thermodynamics (Ooyama
  1990, 2001).
• Dependency of latent heat on temperature and
  specific heats of water substance
• Transports of water, momentum, and energy due to
  rain.
• An accurate transport scheme for rain.
• Conservative Semi-Lagrangian scheme with 3rd order

7
Characteristics of the nonhydrostatic model (2)

• Physics
• Warm rain (bulk method), no ice yet
• Turbulence Mellor and Yamada Level 2, 2.5
  Deardorff Smagorinsky
• Surface flux Louis(1982)
• Radiation MSTRN8 (Nakajima et al, 2000)
• A subset of the three-dimensional global
  non-hydrostatic model
• A test bed of new dynamical schemes. Development
  of the conservative scheme.
• Physics cloud schemes (warm/ice), radiation,
  turbulence
• Study of cloud-radiation interaction and cumulus
  parameterization
• Radiative-convective equilibrium experiments
• Model hierarchy can be used as 1D-vertical,
  2D-horizontal-vertical, and 3D-regional models.

8
Dry formulation

• Conservative flux form equations for density R,
  momentum V, and internal energy E

where and
9
Time integration scheme time splitting

• Large time step t, small time step t
• Leap-frog
• or RK2

10
The flux division method (Klemp et al.2000)
11
Small time integration(1)

• Explicit for U and V
• Implicit for R, W, E using
• 1D-Helmholtz eq. for W

a0Hydrostatic option
12
Small time integration(2)

• Integrate for R in the flux form
• Energy correction integrate for total energy in
  the flux form
• where E internal energy, K kinetic energy, and
  G potential energy

13
Density current experiment (Straka et al, 1993)
Initial cold bubble ? ?15K ?x ?z 50m ?t
0.1s
14
Moist formulation with warm rain

• Prognostic variables
• water vapor qv
• cloud water qc
• rain water ql
• total density ?
• momentum V (U, V, W) (?u, ?v, ?w)
• Sensible part of internal energy Ea
• Effects of specific heats of water substance
  are considered

15
Governing equations (Ooyama, 1990,2000)
16
Squall line exp. 2D, ?x1.25km
Cloud water and rain
Precipitation
Water Energy budgets
17
Squall line exp. 3D 100km x 125km x 21km
t150min
qc z7.3km
? z0.1km
qc z1.4km
t200min
18
Outline of an icosahedral grid modeling
Icosahedral grid modeling

• Grid generation
• Advection terms and Coriolis term
• Life cycle of extratropical cyclones experiment
• Held and Suarez experiments

19
Grid Generation Method

• Grid generation
• Each side of icosahedron whose vertices are on a
  sphere is projected onto the sphere. (glevel-0)
• By connecting the mid-points of the geodesic
  arcs, four sub-triangles are generated.
  (glevel-1)
• By iterating this process, a finer grid
  structure is obtained. (glevel-n)
• of gridpoints
• 11 interations are requried to obtain the 5km
  grid interval.

(0) grid division level 0 (1) grid
division level 1
(2) grid division level 2 (3) grid
division level 3
20
level 8 (28km)
level 7 (56km)
level 10 (7km)
level 9 (14km)
21
Advection of momentum and Coriolis term
Only the advection term is evaluated with the
Cartesian components.
22
Grid Optimization by Spring Dynamics (2)

• Another application of spring grid
• We can construct the clustered grid by tuning the
  spring.

Example of the clustered grid (a) High
resolution hemisphere (b) Low
resolution hemisphere
? For the regional prediction or climate model
23
Model configuration
24
Life Cycle of Extratropical Cyclone Exp.(1)
Polvani and Scott(2002)
glevel 10 ?x7km
glevel 6 ?x112km
glevel 8 ?x28km
25
Life Cycle of Extratropical Cyclone Exp.(2)
glevel 6
glevel 8
glevel 10
26
Held Suarez Dynamical Core Exp.(1)

• Test configuration
• Radiation
• We use a simple radiation as Newtonian Cooling of
  temperature field
• where
• Equilibrium temperature is zonally symmetric as
• where
• Surface fricrion
• Surface friction is imposed in the lower
  atmosphere as a Rayleigh damping
• where

27
Held Suarez Dynamical Core Exp.(4)

• Zonal mean of zonal wind

• glevel-5 (b) glevel-6
  (c) glevel-7
• ?x240km 120km
  60km

GME(DWD) IFS(ECMWF)
ni64(g-level 6) T106
28
Energy spectrum
Comparison with spectral model(2)
glevel-6 vs T159 2?x2p/N240km
N 2?x
N/2 4?x
29
Computational Performance (1)

• Performance on the Earth Simulator
• Earth Simulator
• Massively parallel super-computer
• based on NEC SX-5 architecture.
• 640 computational nodes.
• 8 vector-processors in each of nodes.
• Peak performance of 1CPU 8GFLOPS
• Total peak performance 8x8x640 40TFLOPS

30
Computational Performance (2)

• Scalability of our model

• Configuration
• Horizontal resolution glevel-8
• Vertical layers 100
• Fixed
• The used computer nodes increases from
  10 to 80.
• Results
• Green ideal speed-up line
• Red actual speed-up line

? Our model has a good scalability!
31
Comparison with spectral model (1)
AFES AGCM for the Earth Simulator A very fast
spectral model in the world

• Computational time for 1step

• AFESL 2pR / N
• O(N3)
• NICAM L 2?x
• NICAM L 4?x
• O(N2)

• NICAM is more efficient than AFES
• at least for T1279 or glevel10

32
Comparison with spectral model(2)

• Maximum time step and one-day simulation time

If L 2 pR / N 4 ?x

• Time step of NICAM can be larger than AFES

At T1279 glevel-10, NICAM is faster than AFES.
33
Summary

• A new regional non-hydrostatic model using a new
  conservative scheme.
• Conservation of mass and total energy.
• A newly tuned icosahedral grid.
• Quasi uniform grid using the spring dynamics.
• A stretched grid gt a regional climate model
• A new dynamical core of the nonhydrostatic
  icosahedral grid model Validation of the
  dynamical core
• The Life cycle of extratropical cyclones
  experiment.
• the Held Suarez experiment.
• Measurement of computational performance on the
  Earth Simulator.
• A very good scalability and a good sustained
  performance ( 40 of peak performance ).
• Superior to a spectral model.

34
References

• Icosahedral grid
• Tomita et al., (2001) Shallow Water Model on
  a Modified Icosahedral Geodesic Grid by Using
  Spring Dynamics, J. Comput. Phys., 174, 579-613
• Tomita et al., (2002) An Optimization of the
  Icosahedral Grid Modified by the Spring
  Dynamics, J. Comput. Phys., 183, 307-331
• Nonhydrostatic scheme
• Satoh (2002) Conservative scheme for the
  compressible non-hydrsostatic model with
  horizontally explicit and vertically implicit
  time integration scheme, Mon.Wea.Rev., 130,
  1227-1245
• Satoh (2003) Conservative scheme for a
  compressible non-hydrsostatic models with moist
  processes, Mon.Wea.Rev., in press.
• Global nonhydrostatic icosahedral model
• Tomita et al., (2002a) Development of a
  nonhydrostatic general circulation model using an
  icosahedral grid, Parallel CFD 2002, in press
• Goto et al., (2002) Computational performance
  of dynamical part of next generation climate
  model using an icosahedral grid on the Earth
  Simulator, Parallel CFD 2002, in press
• Tomita et al., (2002b) The Non-hydrostatic
  Icosahedral Global Model for the Earh Simulator,
  Max-Planck Institute for Meteorology technical
  Report 2002
• Tomita et al., (2002c) Global nohydrostatic
  dynamical core on the icosahedral grid Part I
  Model description and fundamental tests, in
  preparation
• Physical processes
• Nasuno et al., (2002) Resolution Dependence
  of a Tropical Squall Line, submitted to
  Mon.Wea.Rev