FSL Mesoscale Ensemble Forecast System for MDSS

1
FSL Mesoscale Ensemble Forecast System for MDSS

• Paul Schultz
• Brent Shaw

2
Overview

• System Configuration
• Hardware Requirements
• LAPS Diabatic Initialization
• Setup and Operations
• QA

3
System Configuration
4
Configuration Design Goals

• Use affordable computing technology
• Maximize use of off-the-shelf components
• Maintain configuration flexibility (resolution,
  models used, geographic domain, etc.)
• Appropriate balance of
• Hardware constraints
• Target forecast length
• Desired spatial resolution
• Number of ensemble members
• Update frequency of each ensemble member

5
FSL Configuration

• Based on hardware availability and forecast
  goals
• 2003-2004 Iowa Domain Setup
• Hourly LAPS data assimilation
• 1 MM5 and 1 WRF run each hour
• 15-h forecast length with 1-h output increment
• 144 x 144 x 35 grid points, 12-km grid distance
• 2004-2005 CDOT E-470 Demo
• Increased forecast length to 24-h
• Requires addl hardware to run WRF domain to full
  24-h period
• Reduced grid points to 128 x 128 x 35
• Upgraded LAPS to v 0-23-16
• Upgraded WRF to WRF v2.0.2 (NCAR EM core)

6
FSL Configuration
2003-2004 Iowa Domain 12-km 144x144x35
2004-2005 Colorado Domain 12-km 128x128x35
7
System Components

• 22 node Linux cluster
• FSL LAPS Data Assimilation Package
• NCAR/PSU MM5 Modeling System
• Community WRF Model and SI Package
• PGI FORTRAN compiler, gcc, netCDF, and MPICH
  packages
• Custom run scripts and directory structures
• FSL public data feed and LDM services

8
Hardware Requirements
9
Hardware Architecture

• High-performance parallel computing system
• Distributed memory parallelism
• Cluster of commodity processors
• High-speed interconnect for MPI communication
• Linux OS
• Shared RAID array controlled by cluster
  management node

10
Minimum Requirements

• Multi-node Linux cluster
• Dual 1.2 GHz Pentium III class processors per
  node
• 1 GB RAM per node
• 1 node for cluster management and file server
• 1 node for LAPS run
• 1 node for pre/post processing per 2 ensemble
  members
• 8 nodes per ensemble member
• Recommend spare nodes for failover capability

11
Minimum Requirements

• 30 GB disk storage per ensemble member
• File system shared amongst all nodes
• High-speed interconnect
• Separate interface dedicated to MPI comms
• Can be Gigabit Ethernet for lt 16 nodes
• Myrinet or similar technology for gt 16 nodes
• Computer room with adequate cooling and power

12
Recommended Hardware Type

• Intel or AMD-based processors running Linux are
  most common and thoroughly tested
• LAPS, MM5, and WRF thoroughly tested on IBM
  AIX-based systems
• SGI is a possibility, but less experience at FSL
  with SGI/Irix
• Sun Solaris problematic for WRF at this time

13
FSL Demo Hardware Architecture
FSL Public Data
Titan Local RAID
FRD NetApp
LDM Server
NCAR
Titan Cluster
Graphics, Verification, MSS
14
LAPS Diabatic Initialization
15
LAPS Overview

• Local
• Can run on workstations or Linux PC
• Configurable for local needs and data sources
• Analysis
• Observation ingest and quality control
• Current, high-resolution 3D depiction of
  atmosphere
• Prediction
• High-resolution 3D forecasts
• Coupling with many state-of-the-art model (MM5,
  RAMS, etc.)
• System
• Fully integrated into operations

16
LAPS Components

• Data Ingest and Quality Control
• 3D atmospheric analysis
• 3DVAR Dynamic Balance
• Forecast Component

17
LAPS Flow Diagram
18
LAPS Data Sources

• 20km RUC 1-h Forecast (First Guess)
• Multiple Level-II WSR-88D Sites (Z and Vr)
• Multiple Level-III WSR-88D Sites (Z)
• GOES visible, 3.9, and LWIR imagery
• GOES sounder data
• NESDIS derived products
• ACARS
• Wind profilers
• Surface obs (METARs, mesonets)
• GPS Total Precipitable Water

19
Quality Control

• State and local mesonets present QC challenges
• Poor siting
• Poor maintenance
• Poor communications
• Result Inaccurate, irregular observations,
  leading to inconsistent analysis products and
  poor forecast initialization

20
Solution Kalman Filter

• Provides a continuous station estimate of
  observation based on self trend, buddy trends,
  and NWP use for quality checking
• With missing obs maintain constant station
  count

Kalman ob
Possible bad ob
Station value
Kalman continuous model
Obs error
Time
21
Analysis Steps

• Obtain first guess (RUC 1-h forecast)
• Perform univariate analyses
• 3D temperature and heights
• 3D winds
• Cloud analysis, typing, and vertical velocity
  estimate
• Variational moisture analysis
• Surface analysis
• Create derived products
• Perform 3D variational dynamic balance
• Prepare forecast model initialization fields

22
LAPS 3D Cloud Analysis
METAR
METAR
METAR
23
Cloud Analysis Example
W-E cross-section cloud coverage, liq, ice
sp. hum., precip, precip type
24
Cloud Typing and Vertical Motions
25
LAPS 3DVAR Dynamic Balance
FH FL
26
Results
3D Simulated Clouds
00Hr Fcst, Valid 28 Mar 01/00Z
01Hr Fcst, Valid 28 Mar 01/00Z
27
Setup and Operations
28
Setup Steps

• Configure cluster
• Install compilers
• Build MPICH and netCDF libraries
• Install MDSS software package
• Run LAPS, WRF, and MM5 installation scripts
• Edit MDSS/Config files for various paths
• Localize the model domain
• Dimensions, projection, resolution
• Run separately for LAPS, MM5, and WRF, but could
  be scripted to share a single config file in the
  future
• Create data ingestors for LAPS if not using FSL
  public netCDF formats

29
Operation

• Start LAPS analysis cron jobs (hourly or more
  frequently)
• GRIB Preprocessors
• WRFSI grib_prep program
• MM5 pregrid program
• Run in cron for each new cycle of Eta and/or GFS
• Cron job for each member
• Use MDSS/Scripts/run_enemble_member.sh

30
Operation

• run_ensemble_member performs
• Execution of desired member model run and all
  pre/post processing
• Runs the GRIB parser script to extract a subset
  of fields from the full GRIB files and write them
  to a directory for transmission to the MDSS NCAR
  fuzzy logic system

31
Operation

• Load balancing for MPI jobs
• Each WRF and MM5 member uses its own
  mpi_machines.conf file to specify nodes
• run_enemble_member automatically adjusts to
  removal/addition of processors from the list
• Operational logging
• Script includes reliability log capability

32
Questions?