Using the Weather to Teach Computing Topics

1
Using the Weather to Teach Computing Topics

• B. Plale,
• Sangmi Lee,
• AJ Ragusa
• Indiana University

2
Outline

• Forecasting Severe Storms
• Why a better computing infrastructure is needed
• Grid computing addresses the problem
• Work being done in context of LEAD project
• http//lead.ou.edu
• Computing architecture to enable better weather
  forecasting
• Demo

3
Motivation for LEAD

• Each year, mesoscale weather floods, tornadoes,
  hail, strong winds, lightning, and winter storms
  causes hundreds of deaths, routinely disrupts
  transportation and commerce, and results in
  annual economic losses gt 13B.

4
Conventional Numerical Weather Prediction

• OBSERVATIONS
• Radar Data
• Mobile Mesonets
• Surface Observations
• Upper-Air Balloons
• Commercial Aircraft
• Geostationary and Polar Orbiting Satellite
• Wind Profilers
• GPS Satellites

5
Conventional Numerical Weather Prediction

• Analysis/Assimilation
• Quality Control
• Retrieval of Unobserved
• Quantities
• Creation of Gridded Fields

• OBSERVATIONS
• Radar Data
• Mobile Mesonets
• Surface Observations
• Upper-Air Balloons
• Commercial Aircraft
• Geostationary and Polar Orbiting Satellite
• Wind Profilers
• GPS Satellites

6
Conventional Numerical Weather Prediction

• Analysis/Assimilation
• Quality Control
• Retrieval of Unobserved
• Quantities
• Creation of Gridded Fields

Prediction PCs to Teraflop Systems

• OBSERVATIONS
• Radar Data
• Mobile Mesonets
• Surface Observations
• Upper-Air Balloons
• Commercial Aircraft
• Geostationary and Polar Orbiting Satellite
• Wind Profilers
• GPS Satellites

7
Conventional Numerical Weather Prediction

• Analysis/Assimilation
• Quality Control
• Retrieval of Unobserved
• Quantities
• Creation of Gridded Fields

Prediction PCs to Teraflop Systems

• Product Generation,
• Display,
• Dissemination

• OBSERVATIONS
• Radar Data
• Mobile Mesonets
• Surface Observations
• Upper-Air Balloons
• Commercial Aircraft
• Geostationary and Polar Orbiting Satellite
• Wind Profilers
• GPS Satellites

8
Conventional Numerical Weather Prediction

• Analysis/Assimilation
• Quality Control
• Retrieval of Unobserved
• Quantities
• Creation of Gridded Fields

Prediction PCs to Teraflop Systems

• Product Generation,
• Display,
• Dissemination

• OBSERVATIONS
• Radar Data
• Mobile Mesonets
• Surface Observations
• Upper-Air Balloons
• Commercial Aircraft
• Geostationary and Polar Orbiting Satellite
• Wind Profilers
• GPS Satellites

• End Users
• NWS
• Private Companies
• Students

9
Conventional Numerical Weather Prediction

• Analysis/Assimilation
• Quality Control
• Retrieval of Unobserved
• Quantities
• Creation of Gridded Fields

Prediction PCs to Teraflop Systems

• Product Generation,
• Display,
• Dissemination

• OBSERVATIONS
• Radar Data
• Mobile Mesonets
• Surface Observations
• Upper-Air Balloons
• Commercial Aircraft
• Geostationary and Polar Orbiting Satellite
• Wind Profilers
• GPS Satellites

The Process is Entirely Serial and Pre-Scheduled
No Responseto Weather!

• End Users
• NWS
• Private Companies
• Students

10
The LEAD Vision No Longer Serial or Static

• Analysis/Assimilation
• Quality Control
• Retrieval of Unobserved
• Quantities
• Creation of Gridded Fields

Prediction PCs to Teraflop Systems

• Product Generation,
• Display,
• Dissemination

• OBSERVATIONS
• Radar Data
• Mobile Mesonets
• Surface Observations
• Upper-Air Balloons
• Commercial Aircraft
• Geostationary and Polar Orbiting Satellite
• Wind Profilers
• GPS Satellites

• End Users
• NWS
• Private Companies
• Students

11
The LEAD Vision No Longer Serial or Static

• Analysis/Assimilation
• Quality Control
• Retrieval of Unobserved
• Quantities
• Creation of Gridded Fields

Prediction PCs to Teraflop Systems

• Product Generation,
• Display,
• Dissemination

• OBSERVATIONS
• Radar Data
• Mobile Mesonets
• Surface Observations
• Upper-Air Balloons
• Commercial Aircraft
• Geostationary and Polar Orbiting Satellite
• Wind Profilers
• GPS Satellites

• End Users
• NWS
• Private Companies
• Students

12
The Value of Being Able to Respond to the
Weather Dynamic Adaptivity
13
Radar Observationsof a Storm System In Kansas
on 20 June 2001
14
11-hr Forecast
15
9-hr Forecast
16
5-hr Forecast
17
3-hr Forecast
Moral Need to do more short forecasts, because
they are more accurate
18
The Value of Local Observations
19
What Do Operational Forecast Models Currently
Predict?
Bands of rain, and high and low pressure, but
thats about it.
20
What Causes the Problems?
Do we really understand the conditions that
result in a funnel cloud?
21
Why the Lack of Detail in Current Forecasts?
22
Why the Lack of Detail in Current Forecasts?
23
The Solution.
Fine-Scale Local Observations

Fine Grid Spacing in Forecast Models
24
Example The March 28, 2000 Fort Worth Tornado
25
TV Radar Image of the Hook Echo
26
NWS 12-hr Forecast Valid Near Tornado
Time(shading indicates precipitation)
27
Radar

Hourly Radar Observations (Fort Worth Shown by
the Pink Star)
28
6 pm
Radar
2 hr
Computer Forecast
29
6 pm
7 pm
Radar

3 hr
2 hr

Computer Forecast
30
Radar

4 hr
3 hr
2 hr

Computer Forecast
31
Radar


4 hr
3 hr
2 hr

Fcst w/o Radar
32
Outline

• The Weather
• Why cyberinfrastructure is needed
• LEAD project addressing the problem
• Cyberinfrastructure based on a web service
  architecture for the Grid
• Prototype demo

33
What is the Grid?

• A collection of resources (computers, databases,
  telescopes, etc.) that can be used by a wide
  range of users with a wide range of skills.
• More than the Internet
• Built on top of the Internet
• The Grid is a collection of web services
  layered on top of the Internet.

Web Services layer
Data Management Service
Security
Registries
Policy
Accounting Service
Logging
Administration Monitoring
Reservations And Scheduling
Grid Orchestration
Event Service
Internet
Physical Resource Layer
34
Predicting Severe Storms
NWS RiverForecast Centers
Historical Observations and Model Output
Digital Library Holdings
NOMADS NCDC
DLESE
I D D
I D D
SuomiNet
I D D
Abilene/NGI
HydrologicData Server
GPS Meteorological Data Server
Wisconsin/SSEC NASA, NOAAPort EROS Data Center
The LEAD project Univ. of Oklahoma
I D D
I D D
NEXRAD RadarData Server
SatelliteData Server
Large scale, real-time Simulation Grid
Lightning Data Server
Surface and Upper-Air Data Server
I D D
I D D
SUNY Albany
Air Quality Data Server
DemographicData Server
I D D
Operational Model Grids and Server
Oceanographic Data
I D D
I D D
Field Program User Generated Data
EPA
DODS
I D D
Project CONDUIT
UCAR/JOSS Individual Investigators
35
Very Simple Scenario to Run Forecast

• Search for data set, run simulation, and catalog
  results.
• Query metadata catalog for dataset
• Use result of query a large WRF simulation
• Allocate storage on remote resource
• Move WRF output to that allocated space
• Record output location and computation history in
  a metadata catalog.
• How does a user describe such a scenario as a
  workflow or distributed application?
• How do we free the user from details of
  distributed computing in a service oriented
  architecture?
• What does a service architecture mean in this
  context?
• Can it be done by a component composition
  approach?

36
Web Services

• Why does the web work?
• A language with few verbs (get, put, post) and
  many nouns (documents).
• Corba Java RMI are object models which present
  a problem.
• Object identity and lifetime is bound to its
  container,
• Whereas a web address is persistent.
• RPC/RMI requires too much synchronization
• For reliability make connections implicit.
• Communicate with simple standard message
  exchanges.

37
So what is a web service?

• A network endpoint, i.e. server, that
  implements one or more ports
• Each port is defined by the message types that
  accepts and the messages it returns.
• A Web Service is specified by a Web Service
  Definition Language xml document.
• Given the WSDL for a web service you know all you
  need to interact with it.
• Web Service Standards exist for security, policy,
  reliability, addressing, notification,
  choreography and workflow.
• It is the basis for MS .NET, IBM Websphere, SUN,
  Oracle, BEA, HP,
• It is the basis for the new Grid standards like
  WSRF and OGSA.

38
Web Site vs Web Service

• The Web Site
• Designed to pass http get/post/put request to
  between a browser and a web server.
• Google has a web site.

• The Web Service
• Designed for services to talk to other services
  by exchanging xml messages
• Google also provides a web service so Google may
  be used in distributed apps

Web Server
Web Service
Web Service
Web Service
39
An Example
query

• The program
• Run a query against a metadata catalog and
  extract simulation boundary conditions
• Allocate storage for simulation output
• Run the simulation
• Save result metadata reference for output to the
  metadata catalog.
• Record event log of execution to the catalog.
• Services/components in our example are
• Metadata catalog
• Storage Allocator
• WRF Simulation Engine
• Execution history recorder

input
Metadata Catalog
output
Query results
reference
mdata
input
Metadata Catalog
output
notification
40
The Workflow as specified by the scientist
Experiment Name (Notification Topic)
Event Listener
query
Notification Broker
Metadata Catalog
Metadata Catalog
done
done
Output URL
Final URL
WRF Factory
File Mover
Parameter file
done
Space Allocator
Storage requirements
Resource info
done
41
The Portal

• Users View of the Grid
• A very sophisticated web browser.
• Lets a classroom teacher create an experiment (to
  run a forecast model for Hurricane Ivan), then
  submit the experiment to the Grid for
  computing. The results can be viewed graphically
  within the portal.

42
Portal as Point of Access to Grid
Keeps information about all the different users
Grid Portal Server
https
SOAP WS-Security
Web Services Resource Framework Web Services
Notification
Physical Resource Layer
43
Portal Architecture (OGCE)

• Building on Standard Technologies
• Portlet Design (JSR-168) IBM, Oracle, Sun, BEA,
  Apache
• Grid standards Java CoG, Web/Grid Services
• User configurable, Service Oriented
• Based on Portlet Design
• A portlet is a component within the portal that
  provides the interface between the user and some
  service
• Portlets can be exchanged, interoperate

Grid
Java
Protocols
Java
Local
CoG
COG
Grid Services
Portlets
API
Kit
GRAM,
MDS
-
LDAD
Portal contaner
MyProxy
Grid Services
Grid Service
SOAP ws call
Portlets
Web Services
44
Putting it together
myLEAD portlet
Factory
myLEAD service
myLEAD agent
workflow
LEAD Portal service
WRF model
Data mining task
Storage Repository service
/var/tmp/wrf_tmp
NCSA
IU
45
Managing Workflow

1. Portlets exist to submitjobs to a condor
   web-service and monitorresults
2. BPEL4WS is web-serviceworkflow standard.
   Interface is underdevelopment.
3. CCA componentscan also be managedfrom the
   portal.

46
Science Portal Deployments in Collaboration with
OGCE, DOE Fusion Portal, NCSA, NPACI/SDSC and
others
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
Thank You