A Services Oriented Architecture for Water Resources Data

1
A Services Oriented Architecture for Water
Resources Data

• David R. Maidment and Timothy L. Whiteaker
• Center for Research in Water Resources
• University of Texas at Austin

Waters Network Meeting Baltimore, Oct 22, 2007
2
Collaborators

• San Diego Supercomputer Center
• Ilya Zaslavsky, David Valentine, Tom Whitenack
• Utah State University
• David Tarboton, Jeff Horsburgh, Kim Schreuders
• Drexel University
• Michael Piasecki, Bora Beran, Yoori Choi
• University of South Carolina
• Jon Goodall

3
A Services Oriented Architecture for Water
Resources Data

• WATERS Network Information System
• Observations data model
• Data Services

4
A Services Oriented Architecture for Water
Resources Data

• WATERS Network Information System
• Hydrologic Information Server
• Data Services

5
Waters Network Testbed Sites
6
Waters Observation Networks

• 16 observation networks (some testbeds have more
  than one network)
• Provides data from 1246 sites
• Of these, 167 sites are operated by WATERS
  investigators

7
Florida Santa Fe Watershed
Nitrate Nitrogen (mg/L)
Millpond Spring
PI Wendy Graham, . DM Kathleen McKee, Mark
Newman
8
North Carolina Albemarle Pamlico Sound
Salinity
Mod Mon and Ferry Mon networks
PI Hans Paerl DM Rodney Guajardo
9
Chesapeake Information Management System (Johns
Hopkins, Drexel, Penn State Universities)
http//www.hydroseek.org
PI Michael Piasecki, Bill Ball, Pat Reed, Kevin
Dressler, Chris Duffy DM Bora Beran, Yoori Choi
10
Baltimore Gwynns Falls Watershed
15-min Precipitation at Carroll Park
PI Claire Welty, .. DM Mike McGuire
11
Susquehanna Upper Juanita Basin
Net Radiation (W/m2)
Oct 05
May 06
PI Chris Duffy, Pat Reed DM Bora Beran, Yoori
Choi
12
Iowa Clear Creek Watershed
Uses streaming data loader
Precipitation
PI Craig Just, Marian Muste, Anton Kruger DM
Marian Muste, Dong Su Kim, Nick Arnold
13
Minnesota Minnehaha Creek
Nitrate Nitrogen (mg/L)
PI Miki Hondzo, Bill Arnold, . DM Jim Kang,
Sung-Chul Kim
14
Montana Crown of the Continent
Snow Depth (m)
Glacier on ice weather station
4
0
2007 July August
PI Johnnie Moore, DM Toby Meierbachotol,
Aaron Deskins
15
Utah Little Bear River and Mud Lake
Turbidity
David Stevens, Jeff Horsburgh, David Tarboton,
Nancy Mesner, Kim Schreuders
16
Sierra Nevada San Joaquin River
Transect of measurements across the river
PI Roger Bales DM Xiande Meng
17
Corpus Christi Bay - Hypoxia
DO (mg/L)
PI Barbara Minsker, Paul Montagna, Jim Bonner,
Ben Hodges DM Kevin Nelson
18
A Services Oriented Architecture for Water
Resources Data

• WATERS Network Information System
• Observations data model
• Data Services

19
Hydrologic Information Server
DASH data access system for hydrology
WaterOneFlow services
ArcGIS Server
Geospatial Data
Observations Data
Microsoft SQLServer Relational Database
20
Point Observations Information Model
http//www.cuahsi.org/his/webservices.html
Utah State Univ
Data Source
GetSites
Network
Little Bear River
GetSiteInfo
Sites
Little Bear River at Mendon Rd
GetVariables
GetVariableInfo
Variables
Dissolved Oxygen
GetValues
Values
9.78 mg/L, 1 October 2007, 6PM
Value, Time, Qualifier, Offset

• A data source operates an observation network
• A network is a set of observation sites
• A site is a point location where one or more
  variables are measured
• A variable is a property describing the flow or
  quality of water
• A value is an observation of a variable at a
  particular time
• A qualifier is a symbol that provides
  additional information about the value
• An offset allows specification of measurements
  at various depths in water

21
CUAHSI Observations Data Model http//www.cuahsi.o
rg/his/odm.html
22
Loading Data into ODM
MyDB
ODDataLoader
Database
23
New Methods for Data Loading
SQL/Server Integration Services
DataTurbine
Streaming Data Loader
24
How to Get an HIS Server

• Options
• Virtual Server
• If you dont have a server available, a virtual
  server can be allocated for you at SDSC
• SDSC Staff sets up your server for you
• Ship your server to SDSC and they will install
  HIS then ship it back to you.
• Provide remote desktop access to SDSC staff and
  they install and configure your server remotely.
  
• Do it yourself
• Download the components and data and configure
  your server on your own.
• Documentation and downloads available here
  http//river.sdsc.edu/wiki/
• You must provide software licenses for
• MSDN Academic Alliance 700/yr
• ArcGIS Server and Desktop (institutional site
  license)

Contact Tom Whitenack at SDSC whitenac_at_sdsc.edu
25
(No Transcript)
26
A Services Oriented Architecture for Water
Resources Data

• Waters Network Information System
• Observations Data Model
• Data Services

27
Definition
The CUAHSI Hydrologic Information System (HIS) is
a geographically distributed network of data
sources and functions that are integrated using
web services oriented architecture so that they
operate as a connected whole.
28
Services Oriented Architecture
(from Wikipedia)

• Service-oriented Architecture (SOA) is an
  architectural design pattern that concerns itself
  with defining loosely-coupled relationships
  between producers and consumers.
• A major focus of Web services is to make
  functional building blocks accessible over
  standard Internet protocols that are independent
  from platforms and programming languages.
• The Web Services Description Language (WSDL,
  pronounced 'wiz-d?l' or spelled out, 'W-S-D-L')
  is an XML-based language that provides a model
  for describing Web services.

Defined by the World Wide Web Consortium (W3C)
29
Web Pages and Web Services
http//www.safl.umn.edu/
Uses Hypertext Markup Language (HTML)
Uses WaterML (an eXtended Markup Language for
water data)
30
WaterML and WaterOneFlow
STORET
Data
GetSiteInfo GetVariableInfo GetValues
NAM
Data
NWIS
Data
WaterML
WaterOneFlow Web Service
Data Repositories
Client
EXTRACT
TRANSFORM
LOAD
WaterML is an XML language for communicating
water data WaterOneFlow is a set of web services
based on WaterML
31
WaterOneFlow

• Set of query functions

• Returns data in WaterML

Ilya Zaslavsky and David Valentine, SDSC
32
Data Heterogeneity

• Syntactic mediation
• Heterogeneity of format
• Use WaterML to get data into the same format

• Semantic mediation
• Heterogeneity of meaning
• Each water data source uses its own vocabulary
• Match these up with a common controlled
  vocabulary
• Make standard scientific data queries and have
  these automatically translated into specific
  queries on each data source

33
Objective

• Search multiple heterogeneous data sources
  simultaneously regardless of semantic or
  structural differences between them

What we are doing now ..
Michael Piasecki Drexel University
34
Semantic Mediator
What we would like to do ..
GetValues
GetValues
GetValues
GetValues
generic request
GetValues
GetValues
Michael Piasecki Drexel University
GetValues
GetValues
35
Hydroseekhttp//www.hydroseek.org
Bora Beran, Drexel
Supports search by location and type of data
across multiple observation networks including
NWIS and Storet
36
HydroTagger
Ontology A hierarchy of concepts
Each Variable in your data is connected to a
corresponding Concept
37
HIS to Google Earthdeveloped by Peter Fitch,
CSIRO, Australia
http//www.watersnet.org/wtbs/ODMKMLGatway.html
A web application housed in Canberra, Australia,
that operates over the WATERS Network data
services
38
NCSA CyberIntegrator
Web Services
Data Sets
Contextual information about a specific data set,
tool or step.
Tools
Barbara Minsker, Peter Bajcsy and Luigi Marini,
UIUC
Workflow
39
(No Transcript)
40
Conclusion Web services work!
The CUAHSI Hydrologic Information System (HIS) is
a geographically distributed network of
hydrologic data sources and functions that are
integrated using web services oriented
architecture so that they function as a connected
whole.
For more information http//www.cuahsi.org/his.ht
ml
41
Conclusions

• Hydrologic Information Server is functioning at
  all testbed sites
• Data are published in a consistent format
  (WaterML) and are thematically synthesized in
  Hydroseek with water agency data
• Applications and analyses can operate seamlessly
  over the Waters Network data services
• A lot more to be done GIS, weather and climate,
  remote sensing, simulation modeling, interpretive
  analysis, .. Digital Watershed development!