SHARP TH Simulation Effort

1
SHARP TH Simulation Effort

• Paul Fischer
• Mathematics and Computer Science Division
• Argonne National Laboratory
• J. Lottes, A. Siegel, S. Thomas, C. Verma

Work sponsored by U.S. Department of Energy
Office of Nuclear Energy, Science Technology
2
Outline

• Long term objectives / Overview
• 2007 Accomplishments
• Code Development
• Nek5000
• Low-Dimensional Code
• Simulations
• DNS
• LES
• RANS
• Low-Dimensional Models

3
Long Term Objectives

• Exploit DOEs Petascale computing facilities ( P
  gt 100,000 processors) and state of the art
  simulation tools to improve TH predictive
  capabilities at the design level
• temperature distributions, under a broad range of
  loading conditions
• pressure drops and flow resistance through the
  system
• Provide validated predictive capabilities based
  on a fidelity hierarchy
• DNS ? LES ? RANS ? low-dimensional modeling
• enable investigation of new designs (e.g.,
  outside validated range of current codes)
• Coupled simulation capability
• spanning a range of scales,
• integrated with other physics (e.g., neutronics,
  structural mechanics, )
• integrated with other codes
• Allow simultaneous coupling of say, LES in some
  areas low-dimensional models elsewhere
  neutronics
• Ultimately, simulate full reactor

4
Petascale Computing at DOE

• Argonne
• 100 Tflops IBM BG/P Nov. 07
• 32,000 processors, 850 MHz
• 500 Tflops IBM BG/P Aug. 08
• 140,000 processors, 850 MHz
• Oak Ridge
• 100 Tflops Cray XT4 Now
• 23,000 processors, 2.6 GHz
• 1 Petaflops Cray XT4 Late 08
• 200,000 processors, 2.6 GHz
• Its time to be thinknig about Exaflops

5
Overview, SHARP Thermal-Hydraulics Plan

• Develop design analysis capabilities that span
  desktop ? Petaflop
• Design rapid turn-around reactor scale
• Analysis detailed simulations providing
  information previously accessible only
  through experiment.
• Input to design codes
• Understanding of basic phenomena (e.g., thermal
  striping)
• Design validation
• Large scale multiphysics simulations at reactor
  scale (out years, PFLOPS)
• Reduce of experiments, not replace.

6
Targeted Range of Simulation Capabilities

• Target Platform Model
• Desktop Subchannel Modeling
• Conservative
• low-resolution
• DG codes
• RANS
• LES
• Petaflops DNS

7
Targeted Range of Simulation Capabilities

• Target Platform Model Current Capabilities /
  Efforts
• Desktop Subchannel SAS
  (T. Fanning)
• Modeling
• Conservative Starting w/ Nek (S.
  Thomas)
• low-resolution
• DG codes
• RANS Star CD (D.
  Pointer)
• LES Nek (F., D. Sheeler,A. Siegel)
• Petaflops DNS Prism (C. Pantano-UIUC)

8
Approaches to TH analysis of subassemblies

• DNS direct numerical simulation of all scales
  parameter-free
• LES large eddy simulation dissipation
  parameter-free
• RANS Reynolds-averaged Navier-Stokes
  tuning required
• Subchannel modeling empirical input
• 400 x 200 subchannels in the core
• Subchannel analysis will continue to be used for
  reactor design.
• RANS will inform design process.
• LES can help to validate / inform RANS and
  subchannel analysis.

impractical 107 p. per channel 105 p. per
channel steady state 100 p. per
channel steady state
9
Current TH Capabilities within ANL SHARP team

• Nek5000 ANL code for fluids / heat
  transfer (Fischer, Lottes, Thomas)
• High-order accuracy
• Scales to P gt 10,000 processors
• State of the art multilevel solvers
• 2 decades of development / verification /
  validation
• Supports conjugate heat transfer, variable
  properties, MHD, ALE, URANS
• Extensive reactor TH experience (Fanning,
  Pointer, Yang)
• RANS modeling Star CD
• Subchannel codes (SAS)

10
Validation Nek5000 ComputationsRod bundle
flow at Re30,000 w/ C. Tzanos (ANL)

• Low-speed streaks in a rod bundle
• Log-law profiles

11
Rod Bundle Validation Nek5000 Comparison w/
Experimental Data (F. Tzanos,
05)
12
Outline

• Long term objectives / Overview
• 2007 Accomplishments
• Code Development
• Nek5000
• Low-Dimensional Code
• Simulations
• DNS
• LES
• RANS
• Low-Dimensional Models

13
Code Development Efforts 07

• Nek5000
• Improved parallel coarse-grid solver for
  multigrid solution of pressure
• work in progress low-memory but not scaling
  as expected
• Working with European collaborators on low-Mach
  number formulation for non-Boussinesq thermal
  expansion effects
• New mesh reading capabilities for large element
  counts and non-native mesh generators
• Coupled to VisIt (D. Bremer, LLNL)
• Low-Dimensional Modeling
• Surrogate mass-conserving velocity fields derived
  from LES/RANS used for thermal transport in
  larger systems (i.e., full-length fuel
  assemblies)
• Developing a conservative super-parametric
  formulation that will be volume preserving
  (non-faceted geometries) with few
  degrees-of-freedom

14
Simulations 07

• First Simulation Study wire-wrapped fuel pins
• DNS
• LES
• RANS
• Low-Dimensional Models

15
First TH Study analysis of wire wrapped pins in
subassembly

• Starting point for TH simulation development and
  deployment
• Uniformity of temperature controls peak power
  output
• A better understanding of flow distribution
  (interior, edge, corner) can lead to improved
  subchannel models.
• Wire wrap geometry is relatively complex

16
Objectives for LES / RANS
From Bogoslovskaya et al.

• Potential surrogate for benchtop experiments
• Provide geometry-specific input to subchannel
  codes
• Consider sequence of 7, 19, , 217 pins to
  provide a detailed picture of the hydrodynamics
  and heat transfer in a single assembly.

17
Approaches to TH analysis of subassemblies

• DNS direct numerical simulation of all scales
  parameter-free
• LES large eddy simulation dissipation
  parameter-free
• RANS Reynolds-averaged Navier-Stokes
  tuning required
• Subchannel modeling empirical input
• 400 x 200 subchannels in the core
• Subchannel analysis will continue to be used for
  reactor design.
• RANS will inform design process.
• LES can help to validate / inform RANS and
  subchannel analysis.

impractical 107 p. per channel 105 p. per
channel steady state 100 p. per
channel steady state
18
Direct Simulation of Wire in Turbulent Channel
with Crossflow
Carlos Pantano UIUC

• Channel-wire flow model
• Model turbulent flow around wires in reactor core
• Target large DNS with accurate spatio-temporal
  resolution
• Derive turbulence statistics for validation of
  RANS/LES models
• Preliminary results (spectral element code)

• Domain size Lx4 ?, Ly 2, Lz2 ?
• 15th order polynomial, 52 elements in x-y plane,
  64 Fourier modes (750K grid points)
• Bulk Reynolds numbers Rex500 and Rez1200 (?
  67o)
• Friction Reynolds numbers 42 and 86 (core flow
  region)

19
Flow visualization
Presence of spiral recirculation bubbles
(isocontours of mean spanwise velocity and
streamlines of transverse velocity)
20
Turbulence statistics

• Mean velocity components

Mean Velocity Components

Normal Reynolds stresses
Kolmogorov scale in false color logarithmic scale
(dark regions denote smaller ???not fully
converged statistics)
21
LES of Single and 7 Pin Wire Wrap Nek5000

• Single Pin
• Mimics infinite array (no assembly walls)
• Cheap, first case for exploratory convergence
  studies, etc.
• 7-Pin
• Geometry is current ARR design
• P/D 1.135
• H/D 17.74 (2/3 of current ARR design)

22
Relationship to Inflow / Outflow Configuration

• Flow establishes a fully turbulent state within
  1 flow-through time
• ? spatial development length H/D
• To be checked by multi-pitch inflow / outflow
  simulations

23
Cross-Sectional Velocity Distributions

• Flow tends to follow in the wake of the wire
• Near the contact point, the flow separates and
  forms a strong standing vortex in the assembly
  cross section, as also reported in RANS
  computations of Ahmad Kim

24
Subchannel Interchange Velocities

• Interchange velocity distributions
• left instantaneous
• right time-averaged

25
Subchannel Interchange Velocities

• Close fit to sinusoid, with amplitudes
• H / D 13.4 a 0.290 Uz
• H / D 20.1 a 0.225 Uz
• H / D 26.8 a 0.150 Uz
• Amplitude higher than predicted by geometric
  factors alone

H/D 26.8
20.1
13.4
26
7 Pin Simulatons

• E132,000, N 7
• nv 44 M
• np 28 M
• niter 30 / step

27
7 Pin Visualization

• Time-averaged axial (top) and transverse
  (bottom) velocity distributions.

Snapshot of axial velocity
28
Subchannel Interchange Velocities 7-Pin, with
Sidewalls

• Inter-channel exchange is no longer a simple
  sinusoid
• Edge channels have non-zero mean ? swirling flow

C
D
C
D
B
B
A
A
29
Subchannel Interchange Velocities 7-Pin, with
Sidewalls

• Inter-channel exchange is no longer a simple
  sinusoid
• Edge channels have non-zero mean ? swirling flow

Single- (Infinite-) Pin Distributions
H/D 17.7
30
7-Pin RANS Using Star CD D. Pointer (ANL)
31
Fine Polyhedral Mesh

• 2.5 million cells
• Based on fine triangulated surface
• Surface extrusion layer not used in current cases
  to allow use of high Re and two-layer k-epsilon
  turbulence models. Will be used with low Re
  models.
• Generated from fine triangulated surface using
  Star-CCM meshing tools

32
Coarse Polyhedral Mesh

• 1 million cells
• Based on coarse triangulated surface
• Surface extrusion layer not used in current cases
  to allow use of high Re and two-layer k-epsilon
  turbulence models. Will be used with low Re
  models.
• Generated from coarse triangulated surface using
  Star-CCM meshing tools

33
Fine Polyhedral Mesh Results

• Re15000 (Vmean 1, Dpin1)
• H/D 26.6

34
Coarse Polyhedral Mesh Results

• Re15000 (Vmean 1, Dpin1)
• H/D 26.6

35
LES / RANS Comparison

• Same basic features
• Significant scaling discrepancies (1.5 x due to
  different H/D, rest tbd)

Star CD RANS Model (note scale difference)
H/D 26.6
H/D 17.7
36
Low-Dimensional Representations

• A step towards subchannel modeling
• allows full-core simulations
• less geometric detail (no wire)
• Wire-induced transport compensated by
  interchannel exchange velocities
• currently generated by helical forcing
• future projection onto LES/RANS results
• Intra-channel mixing enhanced diffusion
• Allows rapid turn-around of coupled multi-physics
  simulations
• Some issues
• How to smear wire-wrap volume into reduced
  geometry?
• Increased clad thickness?
• Maintain cross-sectional area?
• Other

37
Low-Dimensional Models, Full Length Subassemblies

• Effects of interchannel mixing with
• no-wire vs. wire-wrap
• pin conductivity
• thermal loading
• large pin counts
• Sacrifices detailed intra-channel mixing
• Surrogate velocity field generated by spiral
  forcing to match effect of wire-wrap
• Desktop (or small cluster)

38
Conclusions

• Software Development
• Advances to Nek5000 to incorporate additional
  physics, low-resolution conservative formulations
  underway
• Pushing the envelope on problem size and
  processor count
• Continually comparing with commercial and other
  codes as reality check
• Simulations
• First 7-pin LES study is near completion
• RANS LES comparison underway
• 19-pin simulations within the next few weeks
  (EDF)
• Low-resolution TH w/ 7 pins ready to couple with
  UNIC
• Low-resolution 217-pin simulation nearly ready