Title: Transparent Grid Enablement of Weather Research and Forecasting S. Masoud Sadjadi1, Liana Fong6, Rosa M. Badia2, Javier Figueroa1,9, Javier Delgado1, Xabriel J. Collazo-Mojica8, Khalid Saleem1, Raju Rangaswami1, Shu Shimizu4, Hector A. Duran Limon5, Pat
1Transparent Grid Enablement ofWeather Research
and ForecastingS. Masoud Sadjadi1, Liana Fong6,
Rosa M. Badia2, Javier Figueroa1,9, Javier
Delgado1, Xabriel J. Collazo-Mojica8, Khalid
Saleem1, Raju Rangaswami1, Shu Shimizu4, Hector
A. Duran Limon5, Pat Welsh3, Sandeep Pattnaik10,
Anthony Praino6, David Villegas1, Selim Kalayci1,
Gargi Dasgupta7, Onyeka Ezenwoye1, Juan Carlos
Martinez1, Ivan Rodero2, Shuyi Chen9, Javier
Muñoz1, Diego Lopez1, Julita Corbalan2, Hugh
Willoughby1, Michael McFail1, Christine Lisetti1,
and Malek Adjouadi11 Florida International
University (FIU), Miami, Florida, USA 2
Barcelona Supercomputing Center, Barcelona,
Spain 3 University of North Florida,
Jacksonville, Florida, USA 4 IBM Tokyo Research
Laboratory, Tokyo, Japan 5 University of
Guadalajara, CUCEA, Mexico 6 IBM T. J. Watson,
NY, USA 7 IBM IRL, India 8 University of
Puerto Rico, Mayaguez Campus, Puerto Rico 9
University of Miami, Coral Gables, Florida, USA
10 Florida State University, Tallahassee,
Florida, USAContact sadjadi_at_cs.fiu.edu
2Outline
- Motivation
- Grid Enablement
- Application and Scenario
- System Overview
- Remaining Challenges Lessons Learned
3Motivation
- Weather Prediction can
- Save Lives
- Help Business Owners
- How?
- Accurate Results
- Precise Location Information
- What do we have?
- WRF Weather Research Forecast
- The Weather Research and Forecasting (WRF) Model
is a next-generation mesocale numerical weather
prediction system designed to serve both
operational forecasting and atmospheric research
needs.
4Motivation (Cont.)
- WRF Status
- Single Machine/Cluster
- Single Domain
- Fine Resolution -gt Resource Requirements
- How to Overcome this?
- Through Grid Enablement
- Expected Benefits to WRF
- More available resources Different Domains
- Faster results
- Improved Accuracy
5Grid Enablement
- Grid-enabling is the practice of taking existing
applications, which currently run on a single
node or on a cluster of homogeneous nodes, and
adapt them (either automatically or manually) so
that they can be deployed over non-homogeneous
computing resources connected through the
Internet across multiple organizational
boundaries (e.g., multiple clusters from
different organizations) without major
modifications to the underlying source code. - Grid-enablement process successful if the
resulting Grid-enabled application performs
better than the original application. - Performs better can be interpreted differently
- Improved execution time, better resource
utilization, enabling collaboration,
6Application and ScenarioThree-Layer Nested Domain
7Application and ScenarioThree-Layer Nested Domain
15 km
5 km
1 km
8Application and ScenarioThree-Layer Nested Domain
9System Overview
- Web-Based Portal
- Grid Middleware (Plumbing)
- Job-Flow Management
- Meta-Scheduling
- Profiling and Benchmarking
- Development Tools and Environments
- Transparent Grid Enablement (TGE)
- TRAP Static and Dynamic adaptation of programs
- TRAP/BPEL, TRAP/J, TRAP.NET, etc.
- GRID superscalar Programming Paradigm for
parallelizing a sequential application
dynamically in a Computational Grid
10 System Architecture
Grid Middleware
11 Web-Based Portal Screenshot
Meteorologist Login Interface
12 Web-Based Portal Screenshot
Business Owners/Emergency Officials Login
Interface
13Grid Middleware
- Middleware
- A layer between network operating systems and
applications that aims to resolve heterogeneity
and distribution - Examples CORBA, Javas RMI and .NET.
- Grid Middleware
- Middleware for Grid Enablement
- Examples Globus, Legion, Condor-G, etc.
14Peer-to-Peer Inter-Domain Interactions
Meteorologist
Meteorologist
BSC
FIU
Web-Base Portal
Web-Base Portal
Job-Flow Manager
Job-Flow Manager
Peer-to-peer Protocols
Meta-Scheduler
Meta-Scheduler
Resource Policies
Resource Policies
Loca scheduler
Loca scheduler
Loca scheduler
Loca scheduler
Local Resources
Local Resources
Local Resources
Local Resources
C
15Peer-to-Peer Inter-Domain Interactions
Meteorologist
Meteorologist
BSC
FIU
Web-Base Portal
Web-Base Portal
Job-Flow Manager
Job-Flow Manager
Peer-to-peer Protocols
Meta-Scheduler
Meta-Scheduler
7
Resource Policies
Resource Policies
Loca scheduler
Loca scheduler
Loca scheduler
Loca scheduler
Local Resources
Local Resources
Local Resources
Local Resources
C
16Peer-to-Peer Inter-Domain Interactions
17Fault-Tolerant Job-Flow Management
18Job Flow Management Architecture
19The Meta-Scheduling Protocol
20FIU Meta-Scheduler Internal Architecture
21Better Scheduling by Modeling WRF Behavior
Mathematical Modeling
An Iterative Process
An Incremental Process
Profiling
Code Inspection Modeling
Modeling WRF Behavior
Start
Parameter Estimation
Texe ( ?0 ?1 / nodes ) ( ?0 ?1 / clock )
22ResultsExecution Time Vs Allocated CPU
23ResultsModel Validation A Linear Model!
24Challenges remain to be addressed
- High latency of Internet compared to high-speed
LANs - High overhead of the Grid middleware software
- Risking compatibility with future WRF versions
- High volume of the WRF sources code
- Compiling WRF on unsupported platforms
25Lessons Learned
- No current and complete methodology for Grid
Enablement - Grid enabling cluster applications Issues LAN vs
WAN - WRF lack of enough documentation, old
programming techniques - Mathematical Model May Optimize Speedup but
also Error Margin More Clusters Needed - Still on early stage of Concrete Scenario for
Forecast Ensemble
26Acknowledgements
- We are thankful to the following individuals for
their - contributions to some of the ideas presented in
this paper Yanbin Liu, Norman Bobroff, Balaji
Viswanathan, Steve Luis, Shu-Ching Chen, Lloyd
Trinish, Jason Liu, Alex Orta, T. N.
Krishnamurti, Eric Johnson, and Donald Llopis. - This work was supported in part by IBM (SUR and
Student Support awards), the National Science
Foundation (grants OISE-0730065, OCI-0636031,
REU-0552555, and HRD-0317692). This work is part
of the Latin American Grid (LA Grid) project
27- Contact Information
- S. Masoud Sadjadi
- http//www.cs.fiu.edu/sadjadi/
- sadjadi_at_cs.fiu.edu
- Thank you!
- and
- Questions?