David R. Maidment (PI)

1
CUAHSI Hydrologic Information System Update

• David R. Maidment (PI)
• David G Tarboton
• Ilya Zaslavsky
• Michael Piasecki
• Jon Goodall
• With support from collaborators, postdocs and
  graduate students Rick Hooper, Jon Duncan, David
  Valentine, Tom Whitenack, Jeff Horsburgh, Bora
  Beran, Tim Whiteaker, Ernest To, Cedric David

http//www.cuahsi.org/his.html
2
Definition
The CUAHSI Hydrologic Information System (HIS) is
a geographically distributed network of
hydrologic data sources and functions that are
integrated using web services so that they
function as a connected whole.
3
CUAHSI HIS Update

• Current Status of the Project
• HIS 1.0 Components
• WaterOneFlow web services
• Observations data model
• Digital Watershed

4
CUAHSI HIS Update

• Current Status of the Project
• HIS 1.0 Components
• WaterOneFlow web services
• Observations data model
• Digital Watershed

5
HIS Progress

• Phase 1 grant, 2.5 year grant completed
• Phase 2 grant, 5-yr, 4.5M to Maidment (Texas)
• Co-PIs
• Ilya Zaslavsky (San Diego Supercomputing Center)
• David Tarboton (Utah State)
• Michael Piasecki (Drexel)
• Jon Goodall (Duke)
• WATERS Testbeds serve as beta-testers for
  software
• First community-wide distribution in 12 18
  months (dependant on establishing support
  services)

6
CUAHSI HIS Components(in development)
http//www.cuahsi.org/his.html
7
HIS Team and its Cyberinfrastructure Partners
Government USGS, EPA, NCDC, USDA
Industry ESRI, Kisters, Microsoft
CUAHSI HIS
HIS Team Texas, SDSC, Utah, Drexel, Duke
Super Computer Centers NCSA, TACC
Domain Sciences Unidata, NCAR LTER, GEON
8
WATERS Network Information System
HIS Team
WATERS Testbed
9
HIS, WATERS and the CUAHSI Community
Government USGS, EPA, NCDC, USDA
Industry ESRI, Kisters, OpenMI
CUAHSI HIS
WATERS Network Information System
HIS Team
WATERS Testbed
Super computer Centers NCSA, TACC
Domain Sciences Unidata, NCAR LTER, GEON
10
International Partners
European Commission Water database design and
model integration (HarmonIT and OpenMI)
CSIRO Land and Water Resources Water Resources
Observations Network (WRON)
11
CUAHSI HIS Update

• Current Status of the Project
• HIS 1.0 Components
• WaterOneFlow web services
• Observations data model
• Digital Watershed

12
Water Data
Water quantity and quality
Rainfall Snow
Soil water
Modeling
Meteorology
Remote sensing
13
Water Data Web Sites
14
NWISWeb site output
agency_cd Agency Code site_no USGS
station number dv_dt date of daily mean
streamflow dv_va daily mean streamflow
value, in cubic-feet per-second dv_cd
daily mean streamflow value qualification
code Sites in this file include USGS
02087500 NEUSE RIVER NEAR CLAYTON,
NC agency_cd site_no dv_dt dv_va dv_cd USGS 0208
7500 2003-09-01 1190 USGS 02087500 2003-09-02 649
USGS 02087500 2003-09-03 525 USGS 02087500 2003
-09-04 486 USGS 02087500 2003-09-05 733 USGS 020
87500 2003-09-06 585 USGS 02087500 2003-09-07 485
USGS 02087500 2003-09-08 463 USGS 02087500 2003
-09-09 673 USGS 02087500 2003-09-10 517 USGS 020
87500 2003-09-11 454
Time series of streamflow at a gaging station
USGS has committed to supporting
CUAHSIs GetValues function
15
Observation Stations
Map for the US
Ameriflux Towers (NASA DOE)
NOAA Automated Surface Observing System
USGS National Water Information System
NOAA Climate Reference Network
16
Water Quality Measurement Sites in EPA Storet
Substantial variation in data availability from
states
Data from Bora Beran, Drexel University
17
Water Quality Measurement Sites from Texas
Commission for Environmental Quality (TCEQ)
18
Geographic Integration of Storet and TCEQ Data in
HIS
19
Observations Catalog
Specifies what variables are measured at each
site, over what time interval, and how many
observations of each variable are available
20
CUAHSI Hydrologic Data Access System
(being built using HIS Server in collaboration
with ESRI)
NASA
NCDC
EPA
NWS
Observatory Data
USGS
A common data window for accessing, viewing and
downloading hydrologic information

21
HIS Server

• Supports data discovery, delivery and publication
• Data discovery how do I find the data I want?
• Map interface and observations catalogs
• Metadata based Search
• Data delivery how do I acquire the data I want?
• Use web services or retrieve from local database
• Data Publication how do I publish my
  observation data?
• Use Observations Data Model

22
HIS Server and Analyst
HIS Server
HIS Analyst
Implemented at San Diego Supercomputer Center and
at academic departments and research centers
Implemented by individual hydrologic scientists
using their own analysis environments
Web Services
Flexible any operating system, model,
programming language or application
Sustainable industrial strength technology
http//www.cuahsi.org/his/webservices.html
Details of HIS Analyst are here
23
Point Observations Information Model
http//www.cuahsi.org/his/webservices.html
USGS
Data Source
Network
Streamflow gages
Sites
Neuse River near Clayton, NC
Variables
Discharge, stage (Daily or instantaneous)
Values
Value, Time, Qualifier
206 cfs, 13 August 2006

• A data source operates an observation network
• A network is a set of observation sites
• A site is a point location where one or more
  variables are measured
• A variable is a property describing the flow or
  quality of water
• A value is an observation of a variable at a
  particular time
• A qualifier is a symbol that provides
  additional information about the value

24
Example Matlab use of CUAHSI Web Services
create NWIS Class and an instance of the
class createClassFromWsdl('http//water.sdsc.edu/w
ateroneflow/NWIS/DailyValues.asmx?WSDL') WS
NWISDailyValues Site Info for Site of
Interest siteid'NWIS02087500' strSiteGetSiteIn
foObject(WS,siteid,'') strSite.site.siteInfo.site
Name ans NEUSE RIVER NEAR CLAYTON, NC
latstrSite.site.siteInfo.geoLocation.geogLocatio
n.latitude longstrSite.site.siteInfo.geoLocation.
geogLocation.longitude lat 35.6472222 long
-78.4052778
25
Variable and variableTimeInterval
strSite.site.seriesCatalog(1).series().variable
ans variableCode '00065'
variableName 'Gage height, feet'
units 'international foot' ans
variableCode '00060' variableName
'Discharge, cubic feet per second'
units 'cubic feet per second'
strSite.site.seriesCatalog(1).series().variableT
imeInterval ans beginDateTime
'1927-08-01T000000' endDateTime
'2006-10-16T000000' ans beginDateTime
'1927-08-01T000000' endDateTime
'2006-10-16T000000'
26
getVariableInfo
varcode'NWIS00060' varInfoGetVariableInfoObjec
t(WS,varcode,'') varInfo variables 1x1
struct varInfo.variables.variable ans
variableCode '00060' variableName
'Discharge, cubic feet per second'
units 'cubic feet per second'
27
GetValues
GetValues to get the data siteid'NWIS02087500'
bdate'2002-09-30T000000' edate'2006-10-16T0
00000' variable'NWIS00060' valuesxmlGetValu
es(WS,siteid,variable,bdate,edate,'')
28
Parse XML and Analyze
Parse the XML into a Matlab object to work
with valuesobjxml_parseany(valuesxml) ... plot(d
ate,flowval)datetick
29
Data Sources
NASA
Storet
Ameriflux
Unidata
NCDC
Extract
NWIS
NCAR
Transform
CUAHSI Web Services
Excel
Visual Basic
ArcGIS
C/C
Load
Matlab
Fortran
Access
Java
Applications
Some operational services
http//www.cuahsi.org/his/
30
Objective

• Search multiple heterogeneous data sources
  simultaneously regardless of semantic or
  structural differences between them

What we are doing now ..
Michael Piasecki Drexel University
31
Semantic Mediator
What we would like to do ..
GetValues
GetValues
GetValues
GetValues
generic request
GetValues
GetValues
Michael Piasecki Drexel University
GetValues
GetValues
32
CUAHSI HIS Update

• Current Status of the Project
• HIS 1.0 Components
• WaterOneFlow web services
• Observations data model
• Digital Watershed

33
Hydrologic Science
It is as important to represent hydrologic
environments precisely with data as it is to
represent hydrologic processes with equations
Physical laws and principles (Mass, momentum,
energy, chemistry)
Hydrologic Process Science (Equations, simulation
models, prediction)
Hydrologic conditions (Fluxes, flows,
concentrations)
Hydrologic Information Science (Observations,
data models, visualization
Hydrologic environment (Dynamic earth)
34
Data Cube
A simple data model
Time, T
When
D
Where
Space, L
Variables, V
What
35
Continuous Space-Time Model NetCDF (Unidata)
Time, T
Coordinate dimensions X
D
Space, L
Variable dimensions Y
Variables, V
36
Discrete Space-Time Data ModelArcHydro
Time, TSDateTime
TSValue
Space, FeatureID
Variables, TSTypeID
37
Terrain Data Models
Grid


TIN
Contour and flowline
38
CUAHSI Observations Data Model

• A relational database at the single observation
  level (atomic model)
• Stores observation data made at points
• Metadata for unambiguous interpretation
• Traceable heritage from raw measurements to
  usable information

Streamflow
Groundwater levels
Precipitation Climate
Soil moisture data
Flux tower data
Water Quality
39
Hydrologic Observations Data Model
What are the basic attributes to be associated
with each single observation and how can these
best be organized?
Data Source and Network
Controlled Vocabulary Tables
Sites
Variables
Values
Metadata
e.g. mg/kg, cfs
e.g. depth
Streamflow
Depth of snow pack
Landuse, Vegetation
e.g. Non-detect,Estimated,
Windspeed, Precipitation
A data source operates an observation network
A network is a set of observation sites
A site is a point location where one or more
variables are measured
A variable is a property describing the flow or
quality of water
A value is an observation of a variable at a
particular time
Metadata provide information about the context of
the observation.
Data Delivery
Data Discovery
See http//www.cuahsi.org/his/documentation.html
Ernest To Center for Research in Water
Resources University of Texas at Austin 20061011
40
Independent of, but coupled to Geographic
Representation
Arc Hydro
HODM
1
MonitoringPoint
1
SiteID
SiteCode
SiteName
OR
Latitude
Longitude

1
1
41
Variable attributes
Cubic meters per second
L3/T
m3/s
VariableName, e.g. discharge VariableCode, e.g.
0060 SampleMedium, e.g. water Valuetype, e.g.
field observation, laboratory sample IsRegular,
e.g. Yes for regular or No for intermittent TimeSu
pport (averaging interval for observation) DataTyp
e, e.g. Continuous, Instantaneous,
Categorical GeneralCategory, e.g. Climate, Water
Quality NoDataValue, e.g. -9999
42
Stage and Streamflow Example
43
Daily Average Discharge ExampleDaily Average
Discharge Derived from 15 Minute Discharge Data
44
Water Chemistry from a profile in a lake
45
ODM and HIS in an Observatory Settinge.g.
http//www.bearriverinfo.org
46
Downloads
Uploads
HTML -XML
Data access through web services
WaterOneFlow Web Services
WSDL - SOAP
Data storage through web services
47
CUAHSI HIS Update

• Current Status of the Project
• HIS 1.0 Components
• WaterOneFlow web services
• Observations data model
• Digital Watershed

48
Digital Watershed
How can hydrologists integrate observed and
modeled data from various sources into a single
description of the environment?
49
Digital Watershed
Hydrologic Observation Data
Geospatial Data
(GIS)
(Relational database)
Digital Watershed
Weather and Climate Data
Remote Sensing Data
(NetCDF)
(EOS-HDF)
A digital watershed is a synthesis of hydrologic
observation data, geospatial data, remote
sensing data and weather and climate data into a
connected database for a hydrologic region
50
NHDPlus for Region 17E
51
NHDPlus Reach Catchments 3km2
Average reach length 2km
2.3 million reaches for continental US
About 1000 reach catchments in each 8-digit HUC
52
Reach Attributes

• Slope
• Elevation
• Mean annual flow
• Corresponding velocity
• Drainage area
• of upstream drainage area in different land
  uses
• Stream order

53
http//www.daymet.org/
54

• Project sponsored by the European Commission to
  promote integration of water models within the
  Water Framework Directive
• Software standards for model linking
• Uses model core as an engine
• http//www.openMI.org

55
OpenMI Conceptual Framework
All values are referenced in a what-where-when
framework, allowing different data resources or
models to communicate data
VALUES
An application of the data cube to integrate
simulation models
Jon Goodall, Duke University
56
HIS as OpenMI Components
To calculate storage, the model needs inflow and
outflow
NWIS Streamflow
water balance model
Trigger Calculate storage
Daymet Precipitation
To calculate storage, the model needs
precipitation
Goal Link the National HIS web services with a
simple water balance model using OpenMI as the
mediator
57
Watershed Hydrovolumes
Hydrovolume
Geovolume is the portion of a hydrovolume that
contains solid earth materials
USGS Gaging stations
58
Stream channel Hydrovolumes
Residence time distributions
Need the capacity to represent Acoustic Doppler
Current Profiler (ADCP) data (Iowa)
59
Integration of surface water and groundwater data

• Describe the relationship between surface water
  features ( e.g. streams and waterbodies) with
  groundwater features (aquifers, wells).
• Enable the connection with the surface water data
  model

Hydro network
Aquifers
In the future go to 3D...
60
Water OneFlow

• We need a Water OneFlow a common window for
  water data and models
• Advancement of water science is critically
  dependent on integration of water information