SCECCME Pathway 2 Meeting SCECCME Framework

1
SCEC/CME Pathway 2 MeetingSCEC/CME Framework
Phil Maechling 9 June 2003 maechlin_at_usc.edu http/
/www.scec.org/cme
2
Agenda

• Elements of the Community Modeling Environment

3
Elements of the SCEC/CME

• Provide standardized access to the Community
  Models (CVM, CFM, CBM, CMM) etc.
• Add processing algorithms (solvers) to these data
  models (SHA codes, AWM, RDM) etc.
• Provide repository for datasets with sufficient
  metadata to preserve usefulness of datasets.

4
Elements of the SCEC/CME

• Integrated framework where unsophisticated users
  can access Community Data Models, Solvers, Grid
  Computing, Knowledge Representation and
  Reasoning, and Digital Library
• Work environment that provides enough value
  (easy to use, available algorithms, available
  data, archiving, computing resources etc) that
  geoscientist want to use it perform research.

5
Agenda

• SCEC/CME Integrated CME Framework Development

6
SCEC/CME Integrated Framework Development

• A browser based system that enables users to
  construct and run computational pathways.
  Necessarily integrates the seismic hazard
  analysis codes, grid computing, knowledge
  representation and reasoning, and digital library
  technology.

7
Example Scientist Use Case

• User Selects Calculation program (called a
  operator within the SCEC/CME) that they wish to
  run.
• SCEC/CME system displays a list of input
  datatypes required by operator.
• For each input datatype, system displays all
  known instances (called datasets) of that
  datatype. (Presumably the datasets are stored in
  a digital library).
• User selects datasets they wish to use for
  calculation.
• User specifies Run.
• SCEC/CME system calls web service-based operators
  with input parameters of URLs to input datasets.
• Operator program performs calculation, creates
  output datasets, and registers output datasets
  with digital library.

8
Integrated Framework Development

• Integrated Framework may not be appropriate or
  useful environment for scientist doing daily
  work.
• For example, KRR is showing very promising
  concepts of identifying an initial data type, and
  a destintation data type, and the KRR helps you
  develop a computational path between them. Most
  AWM dont need or want this extra layer.
• Script based, batch file based, interaction with
  system may be appropriate.
• The scriptable access to system should be access
  portions of the Intergrated systems capabilities.

9
SCEC/CME Framework Concepts

• Framework is modeled as Resources (as in data
  resources) and Operators (programs that perform
  calculations).
• Resources are stored in Digital Libraries or
  data repositories, along with their metadata.
• Operators are exposed as web services (or grid
  services). Operators the perform significant
  computations are grid enabled.

10
SCEC/CME Service Based Architecture

• SCEC/CME is based on a Service-based system
  architecture.
• Datasets and algorithms are made available
  through a standard interface.
• Users can access Services (data or data
  processing capabilities) remotely.
• Example is the GMT service used by SHA.
• Advantage is that the user is able to access
  without hosting.
• Service Architectures are being formalized as
  Web Services using widely accepted standards
  (e.g. SOAP, XML, WSDL, UDDI).

11
SCEC/CME Current Capabilities

• SCEC/CME is extending the SCEC Community
  collaboration to algorithms and to computing
  resources
• convertServices (wsdl) (http//iowa.usc.edu/ws/ser
  vices/convertServices?wsdl)
• getLatLong
• getUTM
• getUTMwithZone
• HazardMapWS (wsdl) (http//epicenter.usc.edu/axis/
  services/Version?wsdl)
• getHazardMapCurves

12
SCEC/CME Current Capabilities

• Demonstration Coordinate Conversion Web Service
  Program available on SCEC/ITR systems
• http//iowa.usc.edu/ws/services/convertServices?ws
  dl

13
Computational Step Concept
Computational Step modeled like a dataflow
diagram Datatype inputs, transforming program,
datatypes outputs
Map Configuration Params Dataset
Map File - JPEG Dataset
GMT Map Generator
Gridded Intensity Measure Dataset
Input Datatypes Operator
Output Datatypes
Data Type Data formatted for use by a
computational programs. Dataset an instance of
a datatype accessible with a URL. Operator
Inputs datatypes input and outputs datatype
14
SCEC/CME Current Capabilities
SCEC/CME Build 1 Computational Pathway
Gridded Region Definition
Calculate Hazard Curves
Extract IMT Value at POE
Plot Hazard Map
9000 Hazard Curve files (9000 x 0.5 Mb 4.5Gb)
IMR Definition
1 File with 3000 datapoints (100Kb)
1 JPEG (38Kb)
Probability of Exceedence and IMT Definition
ERF Definition
15
SCEC/CME Current Capabilities

• SCEC/CME is developing environment in which User
  can setup a computational pathway using
  existing datasets and algorithms.

16
SCEC/CME Current Capabilities

• Demonstrate Computational Testbed SHA Computation
• http//epicenter.usc.edu/cmedb/PathwayComputationS
  etup.jsp

17
SCEC/CME Computational Model

• Translate this diagram into Services and data
  types.

18
SCEC/CME Pathway 2 Model

• Computational Step 1
• Input Data types
• AWM Problem statement
• Velocity Mesh
• Source Description
• Output Data Types
• AWM Wavefield Data
• Computational Step 2
• Input Data Types
• AWM Wavefield Data
• Output Data Type
• Standard ETree Format Wavefield Data

19
SCEC/CME Required Capabilities

• Establishing Standard Interfaces to Models (e.g.
  Velocity Model)
• valuetype VelocityVolume GeoVolume         
  public long numXpts         public long
  numYpts         public long numZpts        
  public float deltaX         public float
  deltaY         public float deltaZ        
  private Velocity3D velocities        
  VelocityStruct getVelocityAt(in long x, in long
  y, in long z)struct VelocityFile         
  sequenceltoctetgt bytes interface
  VelocityModel          VelocityStruct
  getVelocityAtPoint(in Location atPoint)        
  VelocitySurface getVelocityForSurface(in Surface
  forSurface,                                      
                                                    
    in ParameterSeq properties) raises (
  InvalidParameterException )        
  VelocityVolume getVelocityForVolume(in GeoVolume
  forVolume,                                       
                                            in
  ParameterSeq properties raises (
  InvalidParameterException )

20
SCEC/CME Application Architecture
SCEC/CME Testbed Grid
Users Computer
SCEC/CME Testbed Portal
Apache AXIS
Apache Httpd
Apache Tomcat
Apache Tomcat
Web Service Implementations
HTML/HTTP
XML/SOAP
Apache Struts
Browser Based User Interface
JSPs
Digital Library I/F
Java Web-Service Client Action Methods
Globus Grid Scheduler
Digital Library I/F
mySQL RDBMS
Application Programs
21
SCEC/CME Current Capabilities

• We have not yet determined how to grid-enable our
  web services. Queuing jobs to the grid from a web
  server requires special software technology.
• Our beginning example is openSHA Hazard Curve
  Calculation.

22
SCEC/CME Digital Resource IDs

• Unique Identifiers for Digital Resource
• We use the concept and term URI to mean a WWW
  unique identifier for both a dataset and a
  operator. DOIs are a type of URI.
• We plan to store URIs for all datasets, and all
  operators in our RDBMS.
• We will generate our own URIs for each new
  dataset using a convention www.scec.org/ResourceT
  ypeName/PK. Design allows us to use other URIs
  if appropriate.
• We will maintain a URI to URL mapping table. URL
  may be updated as object is moved.
• System must maintain URI to file mapping except
  naming the file using the URI. Concerned about
  over-writing non-unique datafile, and breaking
  URI-URL mapping.

23
Using SCEC/CME from Script

• Running of programs frequently script based.
• Will try to support script based interface to
  system.
• Will require some type of scripting language.
• Should be able to access Web Services through
  scripts
• Results are typically written to Community data
  storage and must be retrieved by use of URL.
• Enforcement of Metadata generation will be
  necessary.