Content and Practice: Background to the NC Geospatial Data Archiving Project Steve Morris NCSU Libraries

1
Content and Practice Background to the NC
Geospatial Data Archiving ProjectSteve Morris
NCSU Libraries
2
Project Context

• Partnership between university library (NCSU) and
  state agency (NCCGIA)
• Focus on state and local geospatial content in
  North Carolina (state demonstration)
• Addressing the NC OneMap objective Historic and
  temporal data will be maintained and available
• One of eight projects in the first NDIIPP funding
  round Building a Network of Partners

3
Targeted Content

• Resource Types
• GIS data
• Digital orthophotography
• Digital maps
• Tabular data (e.g. assessment data)
• Content Producers
• Mostly state, local, regional agencies
• Some university, not-for-profit, commercial
• Selected local federal projects

4
Value of Content

• National State Geographic Information Council
  estimate for NC 162 million (2003)
• Local content also forms basis of many
  state/federal data resources
• Value over time
• Greatest demand for current data
• Increasing (yet still small) demand for older
  versions of data
• Older versions impossible to replace once lost

5
Large scale, current, attributed vector data
6
High resolution digital orthophotography
7
County Digital Orthophotography Specifics
Source NC OneMap Data Inventory 2004
8
Value as part of cultural heritage
9
Data Quantity

• County Digital Orthophotos
• 88 counties with, estimated 154 flights by 2006
• Estimated 30 gb/flight 4.6 TB total
• County, City, COG Vector Data
• Variable mix of layers some continuous update
• 92 of 100 counties with GIS systems
• 51 municipalities with GIS systems
• State Agency Data
• 1993 and 1998 statewide orthos 800 gb
• Terabytes of vector data and other imagery
• 17-20 TB of LIDAR data

10
NC OneMap Initial Data Layers Produced by Cities
and Counties
Source NC OneMap Data Inventory 2004
11
Content Organization (tiling, etc.)

• State agency data
• Vector data statewide, river basin, quarter
  quadrangles, counties
• Imagery quarter quadrangles, local images
• County data
• Vector data county, tax map units
• Orthophotos tax map units, county mosaics
• Increasing spatial databases (SDE, PostGIS,
  etc.)
• Municipal data
• Vector data city, tax map units

12
Metadata

• FGDC CGDSM implemented by major state GIS
  agencies starting in 1994
• NC CGIA Metadata Outreach regional workshops,
  phone support, training materials
• Adoption
• Some adoption by county agencies (21 of 92
  county GIS systems as of Spring 2004)
• Some adoption by municipal agencies and COGS (13
  of 51 municipal GIS systems by Spring 2004)
• Rare adoption by private, university, NGOs

13
Local Agency Geospatial Metadata
Source NC OneMap Data Inventory 2004
14
Versioning and Updating

• Orthophotos
• County digital orthophotos reflown every 2-7
  years
• Statewide digital orthophoto plan every 5 years
  (alternating BW and color infrared)
• Vector Data
• State agency vector data some static, some
  periodically updated, relatively fewer
  continuously updated
• County/City/COG vector data many data layers
  continuously or periodically updated
• Old versions supplanted, exist on relatively
  inaccessible backups

15
Time series vector data Parcel Boundary Changes
2001-2004, North Raleigh, NC
16
Content Use

• Much sharing between agencies (all levels)
• Municipal agencies use county data (orthos,
  centerlines, parcels) as base layers
• State and federal agencies use local data to
  improve their own data (linework
  adoption/improvement or attribute conflation)
• Local agencies use LIDAR (control points, etc.)
• Problems with different data structures, naming,
  coding schemes (e.g. parcels)
• NC OneMap framework data standards in development

17
Local Applications Where GIS Is Used
Source NC OneMap Data Inventory 2004
18
Knowledge Required for Use

• Most data published in widely used formats
• Vector shapefile, coverage, geodatabase, MIF
• Image TIFF, MrSID, BIL
• LIDAR data range of early/middle/late stage
  products requiring different skills
• Semantic issues different naming schemes across
  counties need data dictionaries and ancillary
  documentation
• Early stage image products (e.g.
  pre-rectification orthos) require greater
  knowledge, but are not widely available

19
Associated Applications/Web Services

• Largely compatible with predominate commercial
  software packages
• Exception several counties using Understanding
  Systems GIS software
• NC OneMap viewer system Cascading map server
  based on drawing on local/state/federal WMS
  sources
• Geocoding function on the NC OneMap map viewer
  uses the commercial geocoding web service

20
GIS Software Used
Source NC OneMap Data Inventory 2004
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
Data/Metadata Workflow Community

• Data
• Orthophoto work contracted out to commercial
  firms
• Some vector data contracted out (notably parcels)
• Most other vector data produced in-house
• Early, middle, late, and late-late stage products
• Metadata
• Metadata published by producer, with NC Metadata
  Outreach Program support
• Metadata published to NC NSDI clearinghouse,
  Geospatial One-Stop, and NC OneMap

29
GIS Data Management
Source NC OneMap Data Inventory 2004
30
County Street Centerline Specifics
Source NC OneMap Data Inventory 2004
31
County Cadastral Specifics
32
County Surface Water Specifics
33
Data/Metadata Workflow Project

• Data identified through a mix of eight different
  existing inventory processes
• NC OneMap Inventory starting from 2004
• Data acquired through a variety of means
• Download, WAN, CD/DVD, external drives, tapes
• Will experiment with OGC technologies for
  identification and transfer of data
• Ingest existing metadata and augment
• Normalize existing FDGC and create minimal if
  none
• Wrap descriptive, adminstrative, technical
  metadata into METS records links to services

34
Data/Metadata Workflow Project

• Ingest into Dspace open source digital repository
  software
• Explore issues surrounding ingest of geospatial
  content into generic, widely available repository
  environments
• Maintain independence of preservation object from
  ingest object of a given repository environment
• Explore re-ingest into a different repository
  environment avoid imprinting on a single
  environment
• Explore a mix of federal/state/local options for
  longer-term archiving

35
Is Content Being Archived Now?

• Records retention requirements dont necessarily
  ensure preservation
• A wide range in practice in general custodian
  and consumer energies are focus on current data
• Where archiving is occurring permanent access
  is generally not supported
• A variety of different state/regional/local
  approaches (focus of project site visits)

36
Integrating with NGDA?

• Community Would need to be relatively effortless
  and inexpensive
• local GIS shops are small, strapped for staff
  time
• long-term access is a low priority
• need automation, web servicesbased archive
  development (and GeoDRM?)
• Rights issues must be cleared need to address
  concerns about disclaimer provision, liability,
  etc.
• Project Interest in exploring re-ingest into a
  separate repository environment

37
Intellectual Property Rights Issues

• Subject to Public Records Law
• Public record no privacy issues
• but records for some individuals may be
  filtered
• Disclaimer viewing important (liability)
• Restrictions on commercial reuse desire for
  downstream control of data
• Great deal of variability in access/use policy
• Trust between agencies is important
  interpersonal and interorganizational
  relationships a key part of the infrastructure

38
Obtaining Local GIS Data
Source NC OneMap Data Inventory 2004
39
NC OneMap Data Sharing Agreements

• NCCGIA working to clarify legal issues
  surrounding redistribution
• Striking MOAs with local agencies as part of NC
  OneMap framework for open access
• One of stipulations AGREE that the data
  shared under this agreement may be further
  redistributed with applicable metadata by either
  agency WITHOUT FEES in the public domain and
  without restriction, unless otherwise noted
  herein and/or unless otherwise subject to public
  laws of governing authorities
• As of 2004 MOAs distributed to 55 counties in
  draft form, 24 signing with option for
  redistribution

40
Transition Plan for Content

• NCSU/CGIA NDIIPP Project
• Project timeline 3 years, 2004-2007
• Commitment to retain data 5 years after project
  termination
• State/Regional/Local Agencies
• Variable
• NC OneMap and other initiatives evolving
• Role of State Archives?

41
Questions?
Contact Steve Morris Head of Digital Library
Initiatives NCSU Libraries Steven_Morris_at_ncsu.edu
Phone [phone]