The Chesapeake Bay Interpretive Buoy System Enterprise Data Management

1
The Chesapeake Bay Interpretive Buoy System
Enterprise Data Management

• Tucker Pierce
• Tellus Applied Sciences

2
The CBIBS Enterprise Data Management System
supports consumers of Chesapeake Bay information
and data

• The CBIBS middleware enterprise system provides
  end-users with access to distributed databases
  containing a wide range of data types.
• The system is built on a set of abstracted and
  loosely coupled components to deliver an open and
  interoperable architecture with minimum barriers
  to entry.
• The system employs Web Services and XML standards
  to reduce the system specific knowledge required
  to access information.
• The technology components that currently comprise
  the CBIBS enterprise system provide the following
  benefits
• Reduction in data access application development
  time
• Reduced OM cost through ease of integration of
  new components
• Increased agility in responding to changing
  system and end-user needs
• Standards-based development approach reduces the
  impact of software obsolescence
• A platform and language independent messaging
  protocol (XML) that facilitates internal and
  external access to disparate database information
  and system services.
• The on-going CBIBS middleware development process
  continues to develop capabilities that facilitate
  end-user access to distributed information
• Incorporation of new data sources into the CBIBS
  information access framework
• Quality assurance options for new data sources
  (internal and externally sourced)
• End-user information discovery, subsetting, and
  data access
• Metadata definition, implementation and
  management
• Business logic implementing building blocks for
  tailored end-user products

3
CBIBS Enterprise Data Management Overview
Front End

• Provides for data access, and supports third
  party apps development
• Standards-based (XML) data interchange format
• Provides for syndicated data feeds such as RSS,
  Yahoo Pipes
• Developer keyed access to allow third party
  applications to retrieve CBIBS and other data on
  demand

Back End
Syndicated features, tools, data, and products

• Sensor system interface approach decouples
  framework from sensor hardware specifics
• Standards-based (XML) Web Services ingest

External Applications And Models
4
Enterprise Data Management Details
T
The system resides on a virtual server in a
24X7 offsite facility
Web Services Interfaces
DBASE
Storage
Data Retrieval Handler
DBASE
Client Data Request Handler
DBASE
. . .
DBASE
DBASE
Web Services Interfaces
External Data Sources
5
Applying the Framework to other Environments

• The framework is applicable to any environment
  that accesses multiple data sources and services
  multiple data users
• It is supported by the NOAA Chesapeake Bay Office
  primarily in support of education and constituent
  projects (e.g. Nauticus Buoy Kiosk in Norfolk)
• Attempts at leveraging the technology into
  existing IT environments has been (sometimes)
  problematic
• Personnel unfamiliar with current technology
• New technologies introduce risk that has not been
  evaluated
• The good news is that once these technologies
  are reviewed and the organizations IT staff is
  comfortable, the door is opened to a greatly
  expanded set of services and capabilities

6
Example of Constituent use of the CBIBS Data
Management Framework
T
(Nauticus Information Kiosk)
7
End

• Questions

8
Component Details.Sensor Data Acquisition and
Storage
T
The Data Archive Handler is responsible for
archiving the data received from the observing
system sensors (or other sources) to the
appropriate local CBOS/CBIBS data storage facility
(Buoys, sensors, meters, etc)
The DAH - QA/QC module performs the necessary
quality checks on the incoming data, and
generates the appropriate system messages in the
case of anomalies.
TCP/IP XML
Data Source 1
System DB
Data Info Storage
Data Source 2
System DB
Data Source 3
System DB
(To DMS Framework)

System DB
Data Source n
The DAH - Transformation module performs any data
modifications required for the newly acquired
raw data
The Metadata module contains data about the
data, e.g. sensor type, calibration history,
and other information that gives the
user confidence in the validity of the data
contained in the archive
9
Component Details..Data Retrieval Handler and
Business Logic
T
NDBC (as required)

Data Retrieval Handler

Construct, send, receive OpenDap commands and
data to and from NDBC
OpenDap Connector
Data Communications Interface
Manages access to the servers local near-term
archive to satisfy user service request
Data Storage
(from Data Archive Handler)
NT Archive Connector
Manages access and retrieval of data from
external sources, e.g. NWS, NERRS.
Services connector
XML
This function represents Push based feed from
external data or information source, e.g. NWS,
NERRS
External Data Sources
The Data Retrieval Handler locates and retrieves
data and products requested by the internet based
front end function. It also initiates business
logic processing for either internal or external
applications supporting value added data product
development for target clients.
Models, procedures, functions, performing value
added services on demand as required to fulfill
user needs
10
Component Details.Client Data Request Handler
and front-end Web Services interface
The Client Data Request Handler manages all
connections and data requests from client
(internet browser) applications.
Public Internet
Client Data Request Handler
Direct Access Mgr
XML Data Comm interface
Browser applications use API key for access to
framework data and/or services
XML
(to the Data Retrieval Handler)
SQS (as required)
The system first checks the Near Term Archive for
any data requested. If not found, system
reformats request as necessary to retrieve the
data from the appropriate source The Near Term
Archive is a queue which pushes older data out
in lieu of data which is in more frequent demand
Note Provision has been made to accommodate the
Amazon Simple Cue Service if and when the need
arises to load balance service requests from
the connected browser application(s)