SeaDataNet

1
SeaDataNet

• A Pan-European Infrastructure for Ocean and
  Marine Data Management
• by
• Dick M.A. Schaap (MARIS)
• Technical coordinator

2
Background

• Data availability of vital importance for marine
  research
• European data set is fragmented
• Not always quality controlled
• Not always easily accessible
• In 35 countries bordering European seas
• more than 1.000 scientific laboratories from
  government organizations and private industry
  collect data
• use various sensors on board of research vessels,
  submarines, fixed and drifting platforms,
  airplanes and satellites
• measure physical, geophysical, geological,
  biological and chemical parameters, biological
  species, etc.

3
What is SeaDataNet?

• EU Sixth Framework Programme
• Integrated Infrastructure Initiative (I3)
• 1 April 2006 to 31 March 2011
• Objectives
• develop an efficient distributed pan-European
  marine data management Infrastructure for
  managing large and diverse data sets
• network existing professional data centres of 35
  countries, active in data collection
• provide integrated databases of standardised
  quality on-line

4
SeaDataNet partners

• 49 partners including
• Major oceanographic institutes of 35 countries
• National Oceanographic Data Centres (NODCs)
• Divisions of major national marine research
  institutes
• 3 satellite data centres
• 3 scientific modelling centres
• 2 SMEs experts in software development
• 3 International organisations (IOC, ICES, JRC)

5
Building on experience

• Leaders of Data Management Concerted Actions
• MEDAR/MEDATLAS II - Mediterranean Data
  Archaeology and Rescue, aiming to make available
  a comprehensive data product of temperature,
  salinity and bio-chemical data in the
  Mediterranean and Black Sea through a wide
  international cooperation
• Sea-Search The gateway to Oceanographic and
  Marine Data Information in Europe, to provide
  an effective navigation tool to oceanographic and
  marine data and information sources for European
  marine waters and to centres in Europe with
  expertise and skills in oceanographic and marine
  data information management.
• Data management of scientific projects
• MTPII-MATER, CANIGO, OMEX, MERSEA, etc.
• Bringing in new partners satellite data centres

6
SeaDataNet components
7
Infrastructure, versions planning

• Version 0
• Continuation, maintenance and further population
  of existing Sea-Search systems with minor
  modifications
• Version 1
• All data centres involved in upgraded
  meta-databases (ISO compliant XML
  metadata-profiles and web services)
• Technical Task Team partners involved in
  transparent data access, security, and monitoring
  of use trans-national access via downloading of
  datasets in standard formats (ODV ASCII NetCDF
  MedAtlas)
• Launch by March 2008
• Version 2
• All data centres involved in trans-national
  access and upgrading of data access to include
  spatial data infrastructure features (adopting
  OGC ISO and INSPIRE standards) and using XML
  data profiles
• Operational by 2010

8
SeaDataNet Data Policy

• Metadata
• free and open access, no registration required
• each data centre should provide the meta-data in
  standardized format to populate the discovery
  services (metadata directories)
• Data and products services
• in general free and open access, but mandatory
  registration
• a SeaDataNet role (partner, academic,
  commercial etc.) is attributed to each user
• Each NODC attributes the roles to the users of
  its of country
• Out of the partnership, the roles are assigned by
  SeaDataNet user-desk
• each data centre node delivers data according to
  the users role and its local regulation
• each data centre should provide freely the data
  sets necessary to develop the common products

9
Discovery services

• Marine environmental data sets (EDMED)
• Marine environmental research projects (EDMERP)
• Observing systems (EDIOS)
• Research Cruise Summary Reports (CSR)
• Common Data Index (CDI)
• Directory of marine organisations (EDMO)

10
Cruise Summary Reports

• Conceived by IOC
• ICES acted as focal point
• Currently over than 37000 cruises
• Documenting data collected on scientific research
  cruises operated by European vessels
• Cruise track charts included
• Online entry form
• Online searchable database

11
European Directory of the Ocean-observing System
(EDIOS)

• Initiative of the European Global Ocean Observing
  System (EuroGOOS)
• Regular repeated measurements
• On-line searchable directory of observing,
  measuring, and monitoring systems
• Over 10,000 observation entries
• Links to on-line real-time and archive data
• Tool to aid design of observing systems

12
European Directory of Marine Organisations (EDMO)

• Over 1.000 organisations addresses, profiles and
  relations in other directories
• Online maintenance per country by CMS
• On-line user interface with combined alphanumeric
  geographical searching
• Export in .csv files
• Also available as Web service (SOAP based)

13
Common Data Index

• Index to individual data sets
• Now gt250000 entries from 20 partners
• Mid Oct 2007 entries from 35 countries
• ISO19115 metadata standard has been adopted
• On-line searchable directory
• Direct access to on-line data sets, on-line order
  facility or e-mail request

14
Technical developments

• Streamlining of metadata formats for
  interrelationships between the directories
• Establishment and governance of common
  vocabularies
• ISO 19115 content model as basis for the XML
  formats and exchange schemas (XSD) with
  SeaDataNet XML approach
• Innovating maintenance modalities
• XML exchange using local software component
  (MIKADO) or own application
• XML validation services
• Online Content Management System (CMS)
• Introducing Web services
• Vocabularies and Directory of Marine
  Organisations (EDMO)
• Coming other metadata directories

15
Interrelationships between the directories,
using vocabularies
EDMED
XML

• CSR

EDMERP
Platf
Param
Country
EDIOS
CDI
EDMO
16
Common Vocabularies

• Controlled vocabularies containing keys, terms,
  abbreviations and essential (but often missing)
  definitions
• Lists for platforms, countries, sea areas, etc
• International content governance by SeaVox list
  server
• Web service
• On-line user client
• Export in .csv files

17
SeaDataNet standards

• Interoperability key to distributed data
  management system success
• Common understanding of terms used via common
  vocabularies
• Standard data exchange formats (ODV ASCII,
  NetCDF, MedAtlas)
• Common quality control protocols and flag scale
• Standard Web services
• Schemes for handling duplicate data
• Ocean Data View software (ODV) adapted for
  SeaDataNet

18
SeaDataNet Delivery Services V1

• Primary function
• Users will be able to download selected data sets
• CDI User Interface
• Searching and browsing of metadata of data sets

19
SeaDataNet Delivery Services V1

• Shopping basket
• Preparing a user request of multiple data sets
• Central User Register
• Validation of users and their roles
• Business matrix
• Combining user role and data access
  restriction(s) to decide on modality for
  processing each data request

20
Central User Register

• Single Sign On system required for access to
  distributed system
• Users authentication information based on
  personal login / password
• User must register in order to get login
• Web form to provide necessary information
• Online user agreement SeaDataNet General
  Licence
• Registration processed per country via online CMS
  
• After processing, login/password sent by e-mail
  (e-mail check)
• User gets Role
• SeaDataNet Data Policy
• Data access based upon combination of User role -
  Access restriction(s) as indicated in CDI record
  and business logic

21
SeaDataNet Delivery Services V1

• Request Status Manager
• Processing of all requests and data deliveries
• Enables users to follow progress
• Enables DCs to oversee transactions
• Download manager
• Local component for communication with portal and
  preparing data deliveries

22
SeaDataNet Users portal schema V1
data
metadata
Project info
Registr.
Standards
Cross search
My transact.
Shopping basket
Requestsstatus manager
CDI
EDIOS
EDMED
Software
AAA
CSR
EDMERP
EDMO
Vocabularies
Organisation data source id
User Register
CSR
EDIOS
CDI
EDMO
BODC Database
EDMED
EDMERP
BSH Database
at BODC
at BSH
at MARIS
Metadata Data catalogues
...
Download managers in data centres
23
Data products

• SeaDataNet will provided added-value products to
  be created by regional centres
• Mediterranean Sea
• Black Sea
• Baltic Sea
• Arctic and North Sea
• Atlantic and global ocean
• First products gt gridded climatologies
• based on already available data-sets (mostly
  hydrographic data)
• updated during the project life
• To make products comparable, all regional data
  centers will use a common tool for spatial
  mapping
• For this a new version of DIVA (Data Inverse
  Variational Analysis) was developed (University
  of Liege)

24
SeaDataNet wider involvement

• SeaDataNet partners are involved in many other
  European and international projects, contributing
  and safeguarding good data management. Examples
• Services for the European marine geological
  institutes
• EDIOS Services for EuroGOOS
• Partners in the MERSEA project (GMES) for MIM
• Partners in the HUMBOLDT project (GMES) for
  developing a framework for a European Spatial
  Data Infrastructure
• Partners in SIMORC for oil gas industry
  metocean data
• Partners in ECOOP for data provision and
  management
• Component of the GMES Marine Core Services

25
Marine geology data
Catalogue of gt 300.000 seafloor samples and gt2.5
million line kilometres of marine seismic and
sonar survey from the ocean basins and
continental shelves, held at European
institutions
www.eu-seased.net
26
Industry metocean datasets - SIMORC

• System of Industry Metocean data for the
  Offshore and Research Communities
• Metadatabase and Database of datasets from major
  oil gas companies

At present ca 870 data sets (Shell, BP, Total),
more underway
www.simorc.org
27
SeaDataNet - BlackSeaScene

• BSS is an Integrating Activity of EU-Research
  Infrastructures programme and runs from 1 Dec
  2005 30 Nov 2008
• BSS has 32 partners with 25 from Black Sea region
  and is coordinated by MARIS
• BSS is a Black Sea Scientific Network
• analysing the present quality of Black Sea
  datasets and applied DQC procedures
• developing a Black Sea regional portal for
  metadata and data access
• BSS is adopting the SeaDataNet standards and
  building upon the SeaDataNet basic services
• BSS engages 25 Black Sea institutes,
  strengthening and expanding the compilation of
  metadata and access to data for the Black Sea
  region

28
SeaDataNet capacity building

• The training program has two main objectives
• to ensure that the data and metadata to be
  integrated in the SeaDataNet system will be
  formatted, checked for quality and disseminated
  according to the common protocols developed
• to transfer expertise and to train IT experts of
  the data centers in the basics, installation and
  operation of the SeaDataNet technical components

29
International Conference IMDIS 2008

• Second international conference International
  Marine Data Information Systems (IMDIS)
• March 31 - April 2, 2008, Athens, Greece
• Joint organisers SeaDataNet, Hellenic Centre for
  Marine Research, IOC/IODE
• Four sessions
• Marine environmental data bases infrastructures
  and data access systems
• Data Services in ocean science standards and
  interoperability
• Visualisation tools
• Services for Users and Education

30
www.seadatanet.org