NOAAs Climate Database Modernization Program Data Rescue Activities

1
NOAAs Climate Database Modernization Program-
Data Rescue Activities
Program Manager- Tom Ross
2
Climate Database Modernization Program

• ?
• The Climate Database Modernization Program
  (CDMP) goal is to preserve and make major climate
  and environmental data available via the World
  Wide Web
• The CDMP supports NOAAs data stewardship
  through all of its activities
• 
  
  

3
Climate Data Base ModernizationOutcomes
Benefits to the Nation

• Outcomes
• Improved access and digital conversion of data
  stored on aging paper, film obsolete digital
  media
• Provides partnerships and jobs through private
  sector companies in 4 states
• Benefits to the Nation
• Human Dimension Reduced loss of life impacts
  due to environmental events
• Business Dimension Enhanced opportunities and
  socio-economic gains by considering climate
  information in decisions
• Policy Dimension More effective government
  policy by appropriately using climate information
  in decisions

Ninth year of the program (began 2000)
4
Climate Database Modernization Program

• NOAAs CDMP partners with four private sector
  contractors and has placed online over 52 million
  weather and environmental images
• CDMP supports over 77 climate and environmental
  data modernization projects
• Most of this data and information was
  underutilized and in some cases in danger of
  being lost
• Keying and Imaging the data increases data
  accessibility and data integration- Work must be
  done by CDMP contractors in KY, MD, WV

5
CDMP Proposal Process has NOAA Working Together
on Data Rescue and Recovery

Climate Database Modernization Program

• The proposals judged against the following
  criteria
• 1. Supports NOAAs Strategic Goals
• 2. Contribution to improved data access rescue
• 3. Value to climate community
• 4 General merit to overall program
• 5. Cost effectiveness
• 6. Ease of digitization by the contractors
• Call for papers- Issued Each July/August
• Data Access Workshop Held November each Year
• 60 Proposals were submitted for FY 2009, 53 in
  2006 but delayed until FY 2008 funding

6
Non-Digital Data Archive
Percent digitized (Keyed or imaged) 50 (55
million) 1.7 (2,105 reels) 1.0 (8,600
fiche)
Manuscript / Autograph 55 Million Pages stored
in 120,000 boxes Located at Asheville
additional paper records located at the
Federal Records Center in Georgia that will
be inventoried and prioritized for
digitization
35mm 16mm Film 125,129 Rolls
Microfiche 860K fiche containing 51 million
pages
7
Climate Database Modernization Program 2000-2008
Over 10 terabytes of Climate data now only a
mouse click away

• Fifty two million weather environmental images
  online
• Hundreds of million of records digitized now
  online
• International data access and rescue activities

1842 Hourly Weather Data from Washington, DC
Imaged and Digitized through the CDMP Program
North Carolina Cooperative Observer Images -Online
Missing images 1950-1980 were made available in
2007
African Records Rescue Project
8
Data Received From Many Sources
Forecast Warning Analysis NWS Coop Observers
Global Synoptic Reports NCEP Charts Ship,
Buoy Reports Rocketsonde Radiosonde Storm
Data Doppler Radar (GOES, POES, NPOESS, many
other) Satellites Aircraft Profiler ASOS
USCRN
9
Climate Database Modernization Program
10
2008 Major NOAA CDMP Data Recovery Tasks
Climate Database Modernization Program
Highlights/Past Successes
11
2008 Major NOAA CDMP Data Recovery Tasks
Climate Database Modernization Program
Highlights/Past Successes
NESDIS The following data were either scanned
or keyed or both by CDMP contractors surface,
marine, upper air, solar, ionospheric data, DMSP
film , arctic sea ice charts, and glacier
photographs
12
2008 Major NOAA CDMP Data Recovery Tasks
Climate Database Modernization Program
Highlights/Past Successes
NESDIS The following data were either scanned
or keyed or both by CDMP contractors surface,
marine, upper air, solar, ionospheric data, DMSP
film , arctic sea ice charts, and glacier
photographs
OAR image and key historic European ship
logbooks, hurricane reconnaissance information,
and various publications
13
2008 Major NOAA CDMP Data Recovery Tasks
Climate Database Modernization Program
Highlights/Past Successes
NESDIS The following data were either scanned
or keyed or both by CDMP contractors surface,
marine, upper air, solar, ionospheric data, DMSP
film , arctic sea ice charts, and glacier
photographs
OAR image and key historic European ship
logbooks, hurricane reconnaissance information,
and various publications
NOS Image nautical charts historical coast
pilots, vectorize and geo reference shoreline
charts, and image and key water level gauge
records
14
2008 Major NOAA CDMP Data Recovery Tasks
Climate Database Modernization Program
Highlights/Past Successes
NMFS Imaging and keying data on cetaceans ,
fish eggs/population, coral reefs, lightship
records and TED data,
NESDIS The following data were either scanned
or keyed or both by CDMP contractors surface,
marine, upper air, solar, ionospheric data, DMSP
film , arctic sea ice charts, and glacier
photographs
OAR image and key historic European ship
logbooks, hurricane reconnaissance information,
and various publications
NOS Image nautical charts historical coast
pilots, vectorize and geo reference shoreline
charts, and image and key water level tidalgauge
records, mammal stranding data
15
2008 Major NOAA CDMP Data Recovery Tasks
Climate Database Modernization Program
Highlights/Past Successes
NMFS Imaging and keying data on cetaceans ,
fish eggs, coral reefs, lightship records, and
magnetic tape recovery.
NESDIS The following data were either scanned
or keyed or both by CDMP contractors surface,
marine, upper air, solar, ionospheric data, DMSP
film , arctic sea ice charts, and glacier
photographs
OAR image and key historic European ship
logbooks, hurricane reconnaissance information,
and various publications
NWS Keying African Upper air Observations, and
surface observations from Uruguay and Chile
NOS Image nautical charts historical coast
pilots, vectorize and geo reference shoreline
charts, and image and key water level gauge
records
16
Climate Database Modernization Program
One of the earliest forms with detailed hourly
data
17
Selected NOAA Projects
Climate Data Base Modernization

• On the left is a photograph of Muir Glacier in
  Alaska from on August 13, 1941, by glaciologist
  William O. Field on the right, a photograph from
  the same vantage on August 31, 2004, by geologist
  Bruce F. Molnia of the United States Geological
  Survey (USGS).
• According to Molnia, between 1941 and 2004 the
  glacier retreated more than twelve kilometers
  (seven miles) and thinned by more than 800 meters
  (875 yards).
• Ocean water has filled the valley, replacing the
  ice of Muir Glacier the end of the glacier has
  retreated out of the field of view. The glaciers
  absence reveals scars where glacier ice once
  scraped high up against the hillside. In 2004,
  trees and shrubs grow thickly in the foreground,
  where in 1941 there was only bare rock.

18
Selected NOAA ProjectsClimate Database
Modernization Program

• Supports NOAAs Ecosystem and Climate goals
• Digitize 7,200 negatives of killer, minke whales
  and other mammals
• Scan 15,000 pages of notes
• Keypunch 3,000,000 characters
• May expand into capturing 750 hours of audio
  tapes collected over 30 years.
• Supports research into the decline of the
  southern killer whale population- classified as
  depleted under the MMPA and could become listed
  under the ESA

19
Selected NOAA ProjectsClimate Database
Modernization Program
Hurricane Celia 1962- Hurricane Wallet Project
20
(No Transcript)
21
EIC Sample Logbook Image with primary elements
(mainly daily data) being keyed

• Wind Direction/Force (closest to noon)

• Lat/Long (noon)
• Barometer (noon)
• Air Temperature (noon)

• State of Weather/Visibility (all available)

• State of Sea (all available none pictured)

22
DMSP Sample Film Scan
Label with file name attached by Lason.
F-4 August 4, 1979 1239 GMT Daytime
Visible Australia
Fiducial mark and number (seconds during day.
23
Solar CDMP Proposed FY07 Work
CDMP Solar FY07

• Focus on completing high resolution research
  quality scans
• Complete scanning of 67 years of daily solar
  H-alpha images on film 24,000 images
• NEW -- Scan 35 mm slides of historical NOAA Space
  Weather activities about 300 NGDC slides from
  Helen Coffey for NOAA History website, to
  complement the SEC files.

24
(No Transcript)
25
Example of a messy page
26
Climate Database Modernization Program
27
Climate Database Modernization Program

• Priorities For Projects Based on the FY 09 TOTAL
  Budget
• On-going projects receive highest priorities
• -- important to complete a task once started
• 25 of CDMP funding reserved for NOAA tasks
  outside of NCDC within NOAA in good budget years
  (not FY 07)
• Identify additional rescue tasks
• Tasks identified in November 2006 were on hold
  until FY 2008

28
Climate Database Modernization Program

• FY 09 TOTAL Budget
• Who Knows