Instruments and Sensors as Grid Services

1
Instruments and Sensors as Grid Services

• Donald F. (Rick) McMullen1
• Kenneth Chiu2
• John C. Huffman1
• Kia Huffman1
• Randall Bramley1
• 1Indiana University
• 2SUNY Binghamton

2
Motivations

• Instruments and sensors are not well integrated
  into grids
• Data is acquired, processed and stored before it
  hits the grid.
• Need methodology for interacting with instruments
  and sensors in real time from grid applications
• Some abstraction of sensor and instrument
  functionality is needed to make grid applications
  that use them more robust and flexible.

3
Goals

• Integrate instruments and sensors (e.g. real-time
  data sources) into a Grid computing environment
  with Grid services interfaces.
• Abstract instrument capabilities and functions to
  reduce data acquisition and analysis
  applications dependence on specialized knowledge
  about particular instruments.
• Move production of metadata as close to
  instruments as possible and facilitate the
  automatic production of metadata.
• Develop a standard, reusable methodology for
  grid enabling instruments.
• Collaborate with scientists in academia and
  industry in a broad range of disciplines who
  either develop instruments or whose work depends
  on the details of using them.

4

• Simple 2-D analysis of instrument taxonomy
• At lease five dimensions identified in more
  detailed analysis
• Project must address enough points (classes) to
  assure breadth of applicability

5
Initial Applications

• High brilliance X-ray crystallography
• Large instrument application
• Deeply integrated into bio and medical discovery
  research methodology
• Mature analysis software and large user community
• Robotic telescopes
• Small numbers of sensors CCD, environmental
  some control aspects filters, aiming, dome.
• Global coordination needed for scheduling
• Aggregation of disparate sensors into a
  compositeinstrument
• Small sensors
• Minimal memory and CPU
• Wireless connectivity
• Developing parallel project to use ad hoc/swarm
  networks for data collection for real-time
  simulation and prediction
• Updating of data flows in response to
  sensor/network reconfiguration.

6
CIMA implementation targets
Large scientific instruments
Embedded sensorsand controllers
verylargesystems, few elements
verysmallsystems, manyelements
PC104 industrialcontroller board
Synchrotron beamline(APS/ALS)
MICA Mote wireless sensor/controller board
7
MMSF Automated Telescope

• Typical remote access automated observatory
• System has 33 distinct sensors, 12 controllers
• Open/close of roof based on a Polaris
  transparency monitor and rain detector simple
  grid of wires detecting rain drops
• Telescope direction and dome control
• Filter selection and telescope focus
• Liquid nitrogen fills of the CCD dewar jars .

8
MMSF Observatory Features

• Instruments producing data without units
• Temperature, humidity cutoffs determined
  empirically as resistances, not degrees or
• Hierarchical and co-located instruments
• Single platform holds three instruments, so
  orienting one changes orientation of all
• Updates to equipment occurs frequently
• Data transferred via 28k modem line middleware
  needs to work locally, directly between
  instruments and sensors

9
Observatory Data Architecture

• Control Data
• object control
• instrument control
• Object Data (i.e., object of scientific interest)
• Full spectrum from raw unitless data to derived
  data artifacts
• Instrument package
• system package data (multiple attributes output)
• system sensor data (single attribute output)
• nonsystem sensor data (weather data from NOAA)
• calibration data
• access protocols

10
X-Ray Crystallography
Proxy Box
Portal
InstrumentManager
InstrumentServices
LAN
WAN
LAN
DataArchive
Non-grid service
Grid service
Persistent
Non-persistent
11
LTER

• Automatic updating of flows
• Provenance
• SOAP/ARTS (Antelope Real-Time System)

12
Further applications currently being explored

• Electron Microscopy
• U. Queensland EM group regional scale,
  multiple instruments
• - KISTI 2nd largest EM (after Osaka)

Robotic telescopes - Bradford robotic telescope,
Oxenhope observatory, Faulkes telescope project

• Environmental monitoring
• Water use
• contaminants

Industrial monitoring and control, e.g. Train
axles - Ore trains Km long, derailment is very
expensive to fix - Temperature sensors on axles
monitor bearing status, anticipate wheel failure
13
Architecture
14
Common Instrument Middleware Architecture (CIMA)
Elements

• Schema for instrument functionality (and ontology
  for schema attributes)
• Data model for representing instrument metrics
  and calibration
• A small, high performance, embeddable Web
  Services stack, initially in Java, including
  Proteus support for multi-protocol, multimodal
  transport
• Service implementation for accessing the
  instruments functionality and metrics via the
  Proteus-mediated interface
• Ability to dynamically insert new protocols into
  running instances
• OGSA and WS-RF compliant functions to register
  with a location service, authenticate users,
  provide access control to instrument controls and
  data, send and receive events, and co-schedule
  the instrument into a Grid computing and storage
  context.

15
Overview
16
(No Transcript)
17
Example CIMA minimal knowledge bootstrap procedure
CIMA instrument
Application
Globus
init, user
Proteus/SOAP calls
authentication, and
instrument lookup
Service

send description
Implementation (SI)
returns

RDF description
Application parses
description of itself
description for ports to
and instrument
read for calibration
and voltage

read calibration port
SI returns

calibration
stored calibration

read thermocouple port
SI calls controller

thermocouple voltage
function to read
Application reads
and return voltage
thermocouple voltage
then computes and
displays a
temperature
18
Services

• Instrument
• get
• set
• Sensor
• get
• set
• ChannelSource
• get
• set
• register
• ChannelSink
• receive
• get
• set

Instrument
ChannelSink
19
Parcels

• Wish to unify our data models, etc.
• Toolkit must be application-independent as much
  as possible.
• Attributes
• Type (string)
• Globally unique ID (string)
• Encoding
• CDATA, Binary, ASCII, Base64
• Location
• Inline, URL, Other
• Parcel Sets
• Special data used for connectivity information.

20
Technologies
21
Technologies

• Web services
• XML, SOAP, WSDL, binary XML
• Grid services
• OGSA/OGSI, WS-RF, DAIS
• Axis C gSOAP
• Proteus (SC 2002)
• XBS (HPC 2004)
• Schema-specific parsing

22
Proteus Motivation

• Web Services for scientific computing
• SOAP performs well as a lingua franca
• But suffers from performance problems for
  scientific data
• Solution establish initial communication with
  SOAP, and then switch to a faster protocol.
• Grid intermediaries

23
Proteus Overview

• Provides multiprotocol RMI system to applications
• Can wrap existing protocol implementations with
  dynamic invocation
• Facilitates use of SOAP as common language
• Switch to faster protocols if supported by both
  sides.
• Mediates between protocol providers and
  applications
• Applications use Proteus client API
• Providers use Proteus provider API
• Allows a new provider to be added (at run-time)
  without changing application
• Generic serialization/deserialization allows
  marshalling code to be reused for multiple
  protocols

24
Multiprotocol
Process 1
Process 2
Protocol A
Protocol B
Network
25
Proteus APIs
Application
Client API
Proteus
Provider API
Protocol A Provider
Protocol B Provider
26
Schema-Specific Parsing

• XML processing stages (conceptual)
• Well-formedness
• Lexical and syntactic, defined by core XML
  specification.
• Validity
• Conformance to a schema, mainly structural
• Application

27
Merging Stages
Fully articulated
Implicitly validated
Schema-specific parsing
User written
Merged
28
Compiler-Based Approach
C (fast)
XMLSchema
IR
Java
RELAXNG
C(low-pow)

• Front-end parses schema into intermediate
  representation.
• Back-end generates code from intermediate
  representation.
• Intermediate representation is a generalized
  automata.

29
Summary

• Create standards for accessing broad spectrum
  instruments and sensors.
• Incompatible components should still have some
  base level of interoperability.