A High-Fidelity, High-Performance Turbulent Dispersion Framework

1
A High-Fidelity, High-Performance Turbulent
Dispersion Framework

• Damian Rouson
• Combustion Modeling Scaling
• U.S. Naval Research Laboratory, Washington, DC
• Prof. Joel Koplik Karla Morris
• Depts. of Physics Mechanical Engineering
• The City College of CUNY, New York, NY
• Sponsors NSF, NRL

2
Overview

• Motivation Objectives
• Previous work
• Physics
• Fire suppression
• Quantum turbulence
• Software
• Componentization strategy
• O.O. Fortran 90/95 Implementation
• Future Work
• Study architecture??complexity??performance
• Study fine-grained parallelism (OpenMP)

3
Motivation
Motivation Objectives Previous
Work Physics Software Future Work

• A broad class of phenomena involve interactions
  between fluid turbulence N-dimensional objects
• Algorithms data structures for these problems
  exhibit a high degree of commonality.
• Exploiting software design commonalities can lead
  to greater code reuse and more efficient
  interoperation

N Objects
0 Point masses particles, droplets
1 Quantum vortices, snails
2 Interfaces (fluid/fluid fluid/solid)
3 Two-fluid mixtures
4
Objectives
Motivation Objectives Previous Work Physics Softwa
re Results Conclusions Future Work

• To build an extensible set of software components
  suitable for the N0,1 subset of turbulent
  dispersion problems.
• To quantify the influence of object-oriented
  design componentization on software complexity
  and performance.
• To simulate new physics improve engineering
  models
• Fire suppression
• Quantum turbulence
• Boundary-layer combustion

5
Previous Work
Motivation Objectives Previous Work Physics Softwa
re Future Work

• Rouson Eaton (2001) On the preferential
  concentration of solid particles in a
  fully-developed turbulent channel flow, J. Fluid
  Mech.
• Rouson Xiong (2004) Design metrics in quantum
  turbulence simulations How physics influences
  software architecture, Scientific Programming.
• Decyk, Norton Szymanski (1997-) Object-oriented
  programming in Fortran 90, ACM Fortran Forum,
  etc.
• Akin (2003), Object-Oriented Programming via
  Fortran 90/95, Cambridge U. Press.
• Armstrong et al. (1999-) Toward a common
  component architecture for high-performance
  scientific computing

6
Particle-Laden Flow
Motivation Objectives Previous Work Physics Softwa
re Future Work
Direct Numerical Simulation (DNS) of
Navier-Stokes Equations
Lagrangian Particle Equation of Motion
Isotropic turbulence
Spectral Spatial Representation
3rd-Order Runge-Kutta Time Advancement
7
Particle-Laden Flow
Motivation Objectives Previous Work Physics Softwa
re Future Work

• Preferential concentration
• Particles move independently from each other and
  slip relative to flow
• Coherent flow structures segregate particles
  according to their inertia
• e.g. cyclone separation by vortices

Particles near centerplane of a channel flow
8
Quantum Turbulence
Motivation Objectives Previous Work Physics Softwa
re Future Work

• Below 2.17 K, liquid helium behaves as a
  two-fluid mixture
• Normal viscous fluid
• Inviscid superfluid
• characterized by discrete (quantized) vortex
  motions
• vortex core 1 A across
• can be modeled by
• 1. Nonlinear Schrodginger Eq.
• 2. Euler equation
• 3. Discrete vortex equations
• Surprisingly, turbulence statistics
• match those of classical flows.

Density ratio of each fluid
9
Quantum Turbulence
Motivation Objectives Previous Work Physics Softwa
re Future Work
Normal fluid DNS of Navier-Stokes equations.
Superfluid vortex filament method.
Vortex filament
Increasing x
Arbitrary origin
Equations of motion
(Biot-Savart Law)
Material properties a, a, k Quantum of
circulation k h/mHe 9.9710-4 cm2/sec
10
Quantum Turbulence
Motivation Objectives Previous Work Physics Softwa
re Future Work
As a prelude to turbulence, we studied the steady
Arnold-Beltrami-Childress (ABC) flow
ABC0.748cm/s, l0.2cm
Animation
Barenghi et al. (1997)
Our result
11
Componentization
Motivation Objectives Previous Work Physics Softwa
re Future Work
Fluid
Provides d_dt(), u(x)
Uses f(x)
Cloud
Provides d_dt(), f(x)
Uses u(x)
Tangle
Provides d_dt(), f(x)
Uses u(x)
Integrator
Provides euler RK3
Uses d_dt()
VortexPoint


VectorField


Droplet


Grid


12
OOP in Fortran 90/95
Motivation Objectives Previous Work Physics Softwa
re Future Work
OOP Concept Fortran Construct
Class MODULE
Abstract data type Derived type
Inheritance Aggregation
Polymorphism Generic interfaces
13
Fortran Implementation
Motivation Objectives Previous Work Physics Softwa
re Future Work
MODULE Cloud_Class USE Droplet_Class
PRIVATE PUBLIC d_dt(), operator(),
operator(-) TYPE Cloud PRIVATE
TYPE(Droplet), DIMENSION(),
ALLOCATABLE drop END TYPE Cloud
INTERFACE d_dt MODULE
PROCEDURE d_dt_Cloud END INTERFACE
CONTAINS FUNCTION d_dt_Cloud(this)
RESULT(dState_dt) TYPE(Cloud), INTENT(IN)
this TYPE(Cloud), POINTER
dState_dt ... DO i1,n
dState_dtdrop(i) d_dt(thisdrop(i))
END DO
Inheritance
Encapsulation
Polymorphism
14
Fortran Implementation
Motivation Objectives Previous Work Physics Softwa
re Future Work
MODULE Droplet_Class PRIVATE
PUBLIC d_dt(), operator(),
operator(-) TYPE Droplet
PRIVATE REAL
St REAL, DIMENSION(3) s
REAL, DIMENSION(3) v END TYPE Droplet
CONTAINS FUNCTION d_dt_Droplet(this,u)
RESULT(dState_dt) TYPE(Droplet),POINTER
dState_dt
TYPE(Droplet),INTENT(IN) this
REAL ,INTENT(IN) ,DIMENSION(3) u
dState_dtSt 0.0 dState_dts
thisv dState_dtv (u-thisv)/thisSt

15
Fortran Implementation
Motivation Objectives Previous Work Physics Softwa
re Future Work
MODULE Integrator_Class USE Cloud_Class USE
Fluid_Class PRIVATE
PUBLIC euler_Integrator TYPE Integrand
PRIVATE TYPE(Cloud) , POINTER cloud_ptr
TYPE(Fluid) , POINTER fluid_ptr END
TYPE Integrand CONTAINS SUBROUTINE
euler_Integrator(this,dt) TYPE(Integrand),
INTENT(INOUT) this REAL(NBYTES)
dt IF (ASSOCIATED(thisflu
id_ptr) ) THEN thisfluid_ptr
thisfluid_ptr dtd_dt(thisfluid_ptr)
ELSE IF (ASSOCIATED(thiscloud_ptr)) THEN
Run-time polymorphism
via dynamic dispatching
16
Future Work
Motivation Objectives Previous Work Physics Softwa
re Future Work

• Complete componentization of F90 code
• Study performance via fine-grained
  parallelization
• Several recent developments presage a potentially
  renewed importance of OpenMP
• Multi-core chips (Intel)
• Hardware support for multithreading (Suns
  Niagara)
• Efficient emulation of SMP via NUMA clusters
  (SGIs Altix)