Migration of LaRC Icing Products to Columbia Supercomputer

1
Migration of LaRC Icing Products to Columbia
Supercomputer
Louis Nguyen, Patrick Minnis, John Murray, and
Rabindra Palikonda NASA Langley Research
Center Hampton, Virginia, USA
2
Objectives
Transition and Integrate the Langleys Satellite
Products to Operations in the operational
National Weather Service Aviation Weather Center
CIP product

• Improve processing system to produce reliable
  icing products
• Increase temporal and spatial resolution of these
  products
• Improve latency time
• Port and migrate code to Project Columbia
  Supercomputer
• Transition research code to operational code

3
Outline

• Overview of LaRC Icing processing environment
• Project Columbia Supercomputer
• Migration plan
• Utilize parallel processing
• Benefits from improved computer capability
• Summary and future plans

4
Overview of LaRC Processing Environment
Processing Environment Uses the Man computer
Interactive Data Access System (McIDAS, developed
by SSEC Univ. Wisc) McIDAS is a programmable
suite of software and interactive data
visualization tool used for displaying,
analyzing, and acquiring datasets Runs on SGI
Unix platform
Real-time Data Acquisition System GOES-East
West data from via McIDAS ADDE server (SSEC)
RUC data obtained from SSEC and NOAA Forecast
Systems Lab Snow maps from NOAA via FTP site
Transition to Operations, utilize the NOAA
NESDIS operational 24/7 ADDE servers to ingest
satellite and model data
5
Overview of LaRC Icing Processing (cont)
Source Code Code Modularization (R.
Palikonda) - satellite modules for GOES (8-12),
MSG, AVHRR, MODIS - sounding module for RUC,
ECMWF, GFS/AVN - helps isolates sections of the
algorithm to improve cloud mask, corr-k,
cloud parameterization, etc. - most modules are
written in C Main driver program is in McIDAS
environment Cloud algorithms are written in
Fortran Data inputs and outputs utilizes
McIDAS API Approximately 75,000 lines of code
6
Overview of LaRC Icing Processing (cont)
Current Computer Resources Used for ASAP SGI
Orgin 3800 with 40 CPUs Only 8 CPU is available
for Icing (other CPUs are used by CERES CALIPSO
project)
Limitation of Current Computer Resources Run
code every half hour during the day and hourly at
night Process GOES-West (10) at 8km res 30
mins Process GOES-East (12) at 8km res 35
mins
Need Additional Resources Run at full 4km
resolution every 15 mins Would take 70 min to
process GOES-West and 80 min for GOES-East at
full resolution Project Columbia Supercomputer
offers needed resource
7
Project Columbia Supercomputer

• NASAs newest supercomputer is one of the worlds
  fastest
• Supports the agencys 4 mission directorates
  (Aeronautics, Exploration, Science, Space),
  NASA Engineering and Safety Center (NESC), and
  Vision for Space Exploration
• Support general science community GSFC runs a
  General Circulation Model to generate real-time
  numerical weather prediction (NWP) to improve
  hurricane tracking and intensity forecast for
  Hurricane Center
• Funds for Columbia pre-paid for
• thru FY05-07 from the 4 mission
• directorates and NESC
• Requested 35,000 cpu hrs FY05-06
• from Aviation Safety Program
• allotment

8
Project Columbia Supercomputer
Technical Hardware Description

• 20 SGI Altix 3700 superclusters, each with 512
  processors
• 10,240 Intel Itanium-2 processors running at 1.5
  GHz
• Capable of 51.9 teraflops, ranked second behind
  DOE Blue Gene
• 20 Terabyte of Memory
• Linix based operation system
• 40 TB of fiber channel RAID storage
• Capable of storing 10 petabytes of archive
  storage

9
Migration to Project Columbia

• NASA Advance Supercomputing (NAS) accounts
  requested
• McIDAS software installation
• Over 1000 programs and subroutines
• Testing the software components
• Port LaRC Icing code to Columbia platform
• Data ingest system (satellite, model, input maps)
• Processing script and aux program package
• LaRC cloud algorithms
• Test and check for consistency of Icing products
  produced from LaRC and Columbia
• Possible security/firewall issues related to
  ingest and dissemination
• of data and products via ADDE and FTP ports

10
Parallel Processing

• GOES-12 GOES-10 are
• processed using single CPU
• at 8km takes 30 mins
• at 4km takes 80 mins
• Time includes 5-8 mins for
• Input and output

GOES-10 Phase
GOES-12 Phase

• Two parallel processing approach
• run sub-domains using multiple CPUs (use same
  binary)
• multi-threaded process for looping through each
  grid box
• Parallel processing will use 16 CPUs (8 for ea
  GOES)
• Columbia CPU are 2.5x faster

11
Parallel Processing (cont)
Full Resolution 80 min process should be reduced
to 9-12 mins
12
Summary and Future Plans

• LaRC Satellite products available near real-time
  in several formats (McIDAS ADDE server, RUC
  Gridded netCDF, and LaRC Packed Binary)
• LaRC Icing code will be adapted to run in
  parallel processing mode
• Parallel processing will allow GOES Icing
  products to run at full resolution every 15 min
• Porting to Project Columbia Supercomputer to be
  completed by end of FY05 (ASAP project milestone)
• Work with the NASA Applied Sciences Program, the
  FAA and NOAA to integrate and transition the LaRC
  Icing products to the operational NWS Aviation
  Weather Center CIP product by end of FY06

This research was support by the NASA Aviation
Safety Program.