CLIVAR and Carbon Hydrographic Data Office

1
CLIVAR and Carbon Hydrographic Data Office
Data Management for CLIVAR and beyond
CLIVAR/OOPC/Argo/GOOS/CPPS South Pacific Meeting
Concepcîon, Chilé (2005.10.13)
2
The CCHDO locates, collects, inspects, modifies
to improve adherence to standards, organizes, and
makes available the CTD, hydrographic, tracer,
and ocean carbon data and associated
documentation relevant to studies of the
large-scale circulation and water masses of the
World Ocean. (Your one-stop clearinghouse for
reference-quality hydrographic data)
South Pacific Meeting Concepcîon, Chilé
(2005.10.13)
3
South Pacific Meeting Concepcîon, Chilé
(2005.10.13)
4

• Improvements for CLIVAR (Post-WOCE) Period
• Strategic Alliances
• CDIAC/Kozyr
• US Repeat Hydrography Program (Talley)
• Hydrographic ATLAS Groups (Koltermann/Orsi)
• Easier access to entire basins of data
  (OnetimeRepeat Cruises)
• Access to cruises by originator cruise name (and
  WOCE line )
• Data formats which are easy to understand and use
• Improved metadata and cruise documentation
• We serve as data detectives for CLIVAR ADCP,
  Surface Meteorology and Underway data groups
  (ARGO?)
• Provide pointers to originators websites
• Proactively contacting
• International data centers
• International institutions
• International funding agencies

South Pacific Meeting Concepcîon, Chilé
(2005.10.13)
5
Adding Datasets The old and the new
INPOC 1993 (Talley/Musgrave)
AAIWSAMFLOC Aug-Sep 2005 (Talley/Chereskin)
BEAGLE / P06 2003 (Fukasawa)
South Pacific Meeting Concepcîon, Chilé
(2005.10.13)
6

• WHAT WE NEED
• Metadata
• Data
• Cooperation of national agencies
• Reliable and friendly contacts

South Pacific Meeting Concepcîon, Chilé
(2005.10.13)
7

• GLOBAL DATA MANAGEMENT
• TASK LIST
• Collect Data (1)
• Archive Data (2)
• Manage and Disseminate Data (3?)
• What is the definition of data management?
• What is the current state of data management?
• Has data management changed and/or improved in
  the last 20 years?

South Pacific Meeting Concepcîon, Chilé
(2005.10.13)
8

• Is Data Management Effective Today?
• (How easily can you get a usable dataset?)
• Within your institution?
• Within your country?
• From other programs? (i.e. CLIVAR, GOOS, IOOS,
  GOSIC, JGOFS)
• Internationally?

South Pacific Meeting Concepcîon, Chilé
(2005.10.13)
9

• Technology makes it easier to manage data
• (doesnt it?)
• The Internet will solve all of our problems!
• Moores Law per GB cost to store and process
  data
• Size and availability of hardware and O/S
• Desktop and remote bandwidth improvements
• Standard formats protocols (ASCII, NetCDF,
  OpenDAP)
• Affordable (or free) database software
• Analytical software (MATLAB, IDL, Excel)
• Scientists have better computer skills
• LIVE data archives available on the web

South Pacific Meeting Concepcîon, Chilé
(2005.10.13)
10

• Age-Old Problems
• Lack of time
• Lack of commitment
• Lack of memory (the cruise was 10 years ago)
• Lack of personal contacts and/or trust
• Meta-what? (lack of definition for metadata)
• Lack of expertise
• Lack of support staff
• National agencies removed from the process
  (lack of cooperation)
• Version control What are FINAL data?

South Pacific Meeting Concepcîon, Chilé
(2005.10.13)
11

• Data Management Priorities
• Acquire Data
• Secure Data and Documentation
• Metadata
• Access 3-click Availability
• Easy to Use Formats (ASCII vs. NetCDF)
• Data Pedigree (are these the definitive/final
  data?)
• Know what has happened, what is happening and
  what will happen in your field of interest (and
  who to talk to).

South Pacific Meeting Concepcîon, Chilé
(2005.10.13)
12
WHERE YOU COME IN The Unknown As we know,
there are known knowns, there are things we know
we know. We also know there are known unknowns.
That is to say, we know there are some things
we do not know. But there are also unknown
unknowns, the ones we don't know we don't know.
From the collected works of D. H. Rumsfeld
Feb. 12, 2002, Department of Defense news
briefing
South Pacific Meeting Concepcîon, Chilé
(2005.10.13)