DATA%20SHARING%20ISSUES,%20METADATA,%20ARCHIVES,%20AND%20COMPREHENSION

1
DATA SHARING ISSUES, METADATA, ARCHIVES, AND
COMPREHENSION

• Urgency
• NEESgrid (www.neesgrid.org) schedule
• Characterize the Earthquake Engineering community
  use of data and metadata January 2002.
• Distribute preliminary metadata standards May
  2002.
• Publish standards for data and metadata models
  and representations by September 2002
• (Prudhomme and Mish, 2001).
• Consortium Developer of NEES www.nees.org
• Working groups on data issues looking for
  interested volunteers

2
Identify/define uses of data and metadata

• To help me remember what I did last time
• To permit other researchers to duplicate test
• Real time remote PI interaction
• To allow numerical simulation
• Interactive decision making during experiment
• Years after the test
• Automated control of the experiment
• Visualization
• Research and education, sponsors
• Data search/query filter
• Artificial Intelligence, inverse/system
  identification
• Software sharing by common interface opensees

3
Use of data

• Data search/query filter
• Artificial Intelligence, inverse/system
  identification
• Software sharing by common interface opensees

4
Experience of geotech community

• CWRU database on element tests
• VELACS USC
• COSMOS and IRIS
• PEER structures data bases UCSD, UW
• UCD cgm.engr.ucdavis.edu

5
Other community examples

• Atmosphere/ocean research NCAR, NOAA, Navy
• Example of flux vector interchanged between
  programs
• User specific API to interface with black box
• CORBA Common Object Request Broker
  Architecture. A spec for an object that may be
  accessed by many platforms java, fortran, etc.
• Fluid flow
• Visualization code runs with solver
• Open GL
• Generic flux vector
• Connection of mismatched meshes (regular and
  scattered.)
• Meshing experimental data with numerical data.

6
Data use and format

• Think ahead for uses
• Needs assessment
• Format changes
• Visualization of large data sets is demanding
• What is data ?
• Format
• Access tools input and output
• Dont store twice because it is in different
  format (calibration?)

7
Formats, coding

• Oracle
• Flat ASCII
• XML

8
What are benefits of standardization?

• Knowledge of data format at one facility is
  transferable to others.
• E.g., numerical simulation of tests at CWRU, UCD.
• Training of experimenters may transferable.
• User interfaces to databases may be sharable so,
  maybe we will not have to each develop the
  interfaces independently.
• Search, query, automated IO, visualization..

9
Barriers to standardization and how to overcome
them

• Need a killer app that assumes a standard
• The gap between Civil Engineering and Information
  Technology.

10
Killer App features

• To help me remember what I did last time
  automated metadata documentation
• To permit other researchers to duplicate test
• Real time remote PI interaction-
  teleparticipation
• To allow numerical simulation
• Interactive decision making during experiment
• Years after the test
• Automated control of the experiment

11
Killer App features(2)

• Visualization
• Data search/query/access/filter
• Web portal - for all of the above?

12
Metadata Design

• Determine the structure of metadata to optimize
• Intuitive query language
• Readable to computers and humans
• Completeness without redundancy
• Flexibility and Evolution
• Curation by NEES SI and Consortium
• Write code- XML document type definitions

13
Strawman metadata structure

• Project Identifiers
• Catalog of Materials, Objects, Sensors and
  Apparatus
• Sequence of Model Test Events and Measurements
• Sensor Channel Gain Lists (1)
• Image Data
• Control Data Files

14
Discussion Items

• Philosophical issues related to culture of data
  sharing?
• Data producer should get first shot at
  publication
• How long should we allow a data generator to
  ponder before other people can have access?
• How do we publish electronic data?
• Give academic credit to data publishers,

15
XML

• ltModelTestgt
• ltCataloggt
• ltSensorsgt
• ltSensor SN"PCB3245"gt
• ltTypegtPiezoelectric Accelerometerlt/Typegt
• ltManufacturergtPCBlt/Manufacturergt
• ltModelgt352lt/Modelgt
• ltCalibrationDategt092899lt/CalibrationDategt
• ltSensitivity Unit"mV/g"gt100lt/Sensitivitygt
• ltRangegt50glt/Rangegt
• ltSensorDatagt http//www.pcb.com/pcb3245
  lt/SensorDatagt
• lt/Sensorgt
• lt/Sensorsgt
• lt/Cataloggt

16
There must be nice interfaces to complex data
structures. Automatic metadata generator should
do most of the work. TEDS (Transducer Electronic
Data Sheets), SCEDS, automated geometry
definition will make the job do-able.
17
Discussion Items

• At what metadata level do we refer to other
  archives instead of re-archiving?
• Example
• Accelerometer amplifier gain for each test event
  archive
• Accelerometer calibration in the test archive
• Date and method of calibration in facility
  archive
• Cross-axis sensitivity at manufacturers archive

18
Strawman metadata hierarchy

• Section 1 of the outline in Table 1 contains
  metadata associated with the research project.
• Section 2 is a catalog of physical objects used
  to construct or test the model. This includes
  apparatus used to test the model, passive
  materials and markers that are placed in the
  model, and sensors that are used in the model
  tests.

19
Strawman metadata hierarchy

• Section 5 describes image data. This could
  include photographs, video camera data, and/or
  engineering drawings of configuration.
• Section 6 describes the data required to control
  the experiment. This could determine the
  location of a CPT sounding, the rate of
  penetration of a penetrometer, or command files
  to control a shaker.

20
Strawman metadata hierarchy

• Section 3 describes sequencing of events. A
  sequence can be the measurement of the location
  of an object, or an event involving activation of
  an actuator or a penetrometer sounding. 
• Section 4 includes the sensor-channel-gain lists
  this documents which sensors are plugged into
  which amplifier channels, and also includes the
  sequence in which the sensor data was recorded,
  and parameters that define gains and filters.

21
CAD of geometry and instrument location numbers
Printable version of report (pdf) describing
experiment and automatically generated data time
histories
Excel spreadsheets of metadata
ASCII data files of sensor readings during about
90 simulated earthquakes (about 1 MB each)
22
Excel spread sheet describing calibration
factors, amplifier channel numbers, gains, data
file format, ...
23
Event BV, page 3 of pdf document - semiautomatic
plot generation using MathCAD program, central
vertical array of accelerometer data
24
NEES Collaboratory
Othersite 1
System Integrator
Othersite 2
Earthquake Researchers
Site A
NEESgrid
Simulation/Experimental Facilities
Educators Students
Site B
NEES Consortium
NEES Consortium Development
Site C
Other Practitioners
Site Council
Professional Engineers
25
UC Davis Research Network
Prototype OLS router
Prototype OLS router
To Sacramento, Merced
To Berkeley, SantaCruz
26
(No Transcript)