Outline

1
Outline

• 1. Earth Simulator Project GeoFEM
• 2. Computational Strategies and Performance
• Hybrid parallel / vector
• Visualization mesh subdivision
• 3. Solid Earth Applications
• Geodynamo
• Horizontal velocity of Japan islands
• Rupture of faults (quasi-static and dynamic)
• 4. Feasibility study of pluggable function
• 5. Summary Future Plan
• SE on ES
• Towards HPC-MW project and Grid computing

2
Multi-Scale in Solid Earth
Mantle-Core Dynamics Plumes Target 107 nodes
(Dh 10 km order)
Crustal movements tectonic deformation Target
109 nodes (Dh 1 km )
Seismic wave generation propagation Target
1011 nodes (Dh 20 m )
3
Geodynamo
Electrically conductive fluid in Earth's outer
core
Entire mesh
Earths interior
Enlarged view
Partitioning
Insulated area (inner core, red)
Conductive fluid shell (outer core, green)
4
Spectral method by Christensen et al. (2001)
GeoFEM
Radial magnetic fields at the outer boundary of
fluid shell
5
Convection and Magnetic Field Patterns
Magnetic field (Bz) on equator plane
Intensity of z-component of vorticity on equator
plane
Radial magnetic field on CMB

• NS equations with Boussinesq approximation,
  Coriolis and Lorentz terms, the thermal diffusion
  equation, Ohm's law and Maxwell's equations in
  MHD
• 2.2M nodes, Ekman number5.0E-5, Pr1, Ra
  300, Pm 1

6
Animation for Core Dataset
Time evolution of z-component of vorticity
7
Geometry of Spherical Shell
8
Treatment of the Magnetic Field

• FE mesh for the magnetic field is considered for
  both outside and inside of the fluid shell.
• Vector potential in the fluid and insulator is
  solved simultaneously.

Mesh for the fluid shell
Entire mesh
Grid pattern for center
9
Basic Equations for GeoFEM/MHD
Coriolis term
Lorentz term
for conductive fluid
for conductor
for insulator
10
Time Integration Algorithm
3x3 Solver for conductor and insulator
Poisson Solver for conductor and insulator
Poisson Solver for fluid
3x3 solver for fluid
Poisson solver for fluid
11
Dynamo Benchmark Test(Christensen et al., 2001)

• Only one benchmark test for MHD simulation in a
  rotating spherical shell
• Low energy and steady dynamo
• All reported results are done by the spherical
  harmonic expansion

12
Dynamo Benchmark Test (cont.)(Christensen et
al., 2001)

• 3 benchmark tests
• Case0 Non-magnetic field
• Inner core co-rotates with mantle
• Case1 Simple MHD dynamo
• Inner core Co-rotates with mantle Electrically
  insulated
• Case2 More realistic simulation
• Inner core Rotated by the viscous and Lorentz
  torque Same conductivity as the outer core

13
Dynamo Benchmark Test (cont.)- Requested Data -

• Average kinetic and magnetic energy in the fluid
  shell
• Drift velocity of the convection pattern w
• Local T, uf? and Bq?at a points where the
  following conditions are satisfied

Velocity field on equatorial plane
14
Kinetic and magnetic energies averaged over the
fluid shell
15
Convergence of solutions
Averaged magnetic energy
Magnetic field at a specific point
R third root of DOF for scalar valuables
Solutions are converging to the reference
solution as the mesh resolution goes fine.
16
Performance on the ES
17
Performance on the ES
To peak ratio
18
Multi-Scale in Solid Earth
Mantle-Core Dynamics Plumes Target 107 nodes
(Dh 10 km order)
Crustal movements tectonic deformation Target
109 nodes (Dh 1 km )
Seismic wave generation propagation Target
1011 nodes (Dh 20 m )
19
Horizontal Velocity of Japan Islands
Computed by Prof.Hirahara ( Nagoya Univ.) using
GeoFEM
20
FE Mesh
?South West Japan
?North East Japan
24,255 nodes, 21,600 elements
23,520 nodes, 21,080 elements
21
Results (Horizontal Velocity)
?North East Japan
Observation
Computation
22
Results (Horizontal Velocity)
?South West Japan
Observation
Computation
23
Friction Force Accumulation and Slip due to
Rupture of Faults of North East Japan

• Quasi-static analysis
• Augmented Lagrange method to treat contact
  between mantle and plate
• Iterative solver with selective blocking
  preconditioning
• 2M nodes

24
Friction Force
3 Fault Patches
Slip Ratio

• Friction force and slip ratio on plate boundary (
  view from bottom )
• Friction coefficients on three fault patches are
  assumed.
• Larger-scale model refined by PMR will be solved.

25
Dynamic Fault Rupture with Slip-Weakening Law
m drops from 0.606 to 0.6 at patch area
26
Dynamic Fault Rupture with Slip-Weakening Law
Case of 2,300,000 nodes
Case of 250,000 nodes
Mesh dependency should be further studied.
27
Simulation model of South West Japan

• Most realistic FE model developed so far.
• Numerical instability encountered.
• Re-modeling and analysis on going.

28
Multi-Scale in Solid Earth
Mantle-Core Dynamics Plumes Target 107 nodes
(Dh 10 km order)
Crustal movements tectonic deformation Target
109 nodes (Dh 1 km )
Seismic wave generation propagation Target
1011 nodes (Dh 20 m )
29
Earthquake Simulation of Tokyo Bay Area
Domain of Computation
Earthquake
Aug.11, 1999 0928AM (35.4N, 139.8E) M4.0
30km
Obsevation Point Chitose-park (35.4338N,
139.6372E)
Depth 60km
40km
30
FE Mesh Partitioning
lt Metis 8 domainsgt
lt RCB 8 domains (X-Y-Z) gt
lt manual 16 gt (XXYYZ)
lt RCB 16 gt (XXYYZ)
31
Strong Motion of Tokyo Bay Area
- NS 24 km, EW 30 km, Depth 60 km - Displacement
norm on ground surface(16 28 s, animation)
32
(No Transcript)
33
Heat transfer coefficient
Steady heat conduction
0
Temperature (?)
Time (sec)
Tubesheet
Transient BCs
34
Examples of conventional FE modeling
( 30symmetry )
35
Tubesheet Small_Model
54,084 nodes 40,416 elements
36
Tubesheet Middle_Model
540,590 nodes 474,756 elements
37
Tubesheet Large_Model
1,053,906 nodes 949,512 elements
38
Speed up
ideal
Large_Model
Middle_Model
Small_Model
Speed-up Sn
Number of PEs
Rate of CPU Usage
 
39
495-pin Micro-PGA package Intel Mobile Pentium
iii Processor Photo http//www6.tomshardware.com
/cpu/00q4/001107/mobilecpu-19.html
Top view
Bottom view
40
Refined by PMR 7.8 M nodes, 7.6 M
elements Mises stress
41
(No Transcript)
42
495-pin Micro-PGA package