Michael Moeller

1
Metadata
Understanding the Value and Importance of
Proper Data Documentation
Michael Moeller NOAA Coastal Services Center
2
Workshop Outline

• Metadata
• What is Metadata?
• Metadata as a Component of Data
• The Value of Metadata
• Standardized Metadata
• The FGDC Workbook
• Playing the Metadata Game
• Exercise 1
• Metadata as a Data Discovery Tool
• Exercise 2
• Exercise 3
• Writing Metadata
• Tool Time

3
First Things First
Some Terms We Will Be Using
Metadata - Documentation of geospatial data
written in a consistent manner
FGDC - Federal Geographic Data Committee
CSDGM - Content Standard for Digital Geospatial
Metadata, referred to commonly as The Standard
or The Content Standard
Clearinghouse - A distributed catalog of metadata
Geospatial - refers to a geographic location
4
What is Metadata?
5
What is Metadata?
Simply put, metadata is information about your
data.
6
This is the metadata for this.
Whats Missing?
7
This is the metadata for this.
Author(s) Boullosa, Carmen.
Title(s) They're cows, we're pigs /
by Carmen Boullosa
Place New York Grove Press, 1997.
Physical Descr viii, 180 p 22 cm.
Subject(s) Pirates Caribbean Area Fiction.
Format Fiction
While the card-catalog entry is a form of
metadata, it does not address topics such as
quality, accuracy, or scale. Well-written
geospatial metadata describes these and many more
aspects of the data.
8
a small part of
This is the metadata for this.
Identification_Information Citation
Citation_Information Originator NOAA
Coastal Services Center Publication_Date
19971131 Title Hurricane Storm Surge
Geospatial_Data_Presentation_Form Map
Publication_Information
Publication_Place Charleston, SC
Publisher NOAA Coastal Services Center
Larger_Work_Citation Citation_Information

9
What is Metadata?
We often use metadata without even
knowing it.
10
What is Metadata?
Metadata contains vital information.
Imagine being given two identical cans, one
containing cat food, the other containing
tuna. Your task is to choose one to eat. But
heres the catch. Neither can has a label. Which
would you choose?
Cat Food?
Tuna?
Without a label, how would you know which one to
eat and which one to feed to your cat?
11
What is Metadata?
Other Examples?
12
Metadata as a Component of Data
13
A Component of Data
Properly documented data provides vital
information to interested parties.
14
A Component of Data
Metadata is that component of data which
describes it.
Environmental Sensitivity Index Data
Metadata
15
A Component of Data
Its data about a data set.
16
A Component of Data
Metadata describes
CONTENT CONDITION QUALITY
Characteristics of the data
17
A Component of Data
Metadata is a critical component of a complete
data set.
18
A Component of Data
Take Home Message
Because metadata provides vital information about
a dataset, it should never be viewed or treated
as a separate entity.
Metadata is an integral part of any complete
data set.
Metadata
Spatial
Non-spatial or
attributes
19
The Value of Metadata
20
The Predominant Attitude?
It takes a lot less time, and most people wont
notice the difference until its too late.
21
The Time Value of Metadata
Properly documenting a data set is the key to
preserving its usefulness through time.
22
The Time Value of Metadata
Two similar paintings by Picasso up for auction
sold for vastly different prices.
Why?
One had metadata... ...One didnt.
23
Metadata should be updated to reflect changes in
the data.
24
The Value of Metadata
Metadata has other value associated with it.
25
The Value of Metadata
For data developers, metadata...

• Avoid duplication
• Share reliable information
• Publicize efforts
• Reduce workload

26
The Value of Metadata
For data users, metadata...

• Facilitates understanding
• Focuses on key elements
• Enables discovery inside and outside of
  organizations

27
The Value of Metadata
For organizations, metadata...

• Protect investment in data
• Create an institutional memory
• Counter personnel changes
• Allow sharing of data with
• other agencies
• Reduce costs
• Limit potential liability
• Save time and money

28
The Value of Metadata
Metadata as a data discovery tool

• If its geospatial data you need, metadata helps
• Find data of interest
• Determine the usefulness of the data
• Determine how to access the data

This saves time and money.
29
The Value of Metadata
This data discovery is all accomplished through
the use of a clearinghouse, an on-line searchable
catalog of standardized metadata records.
30
Standardized Metadata
31
Why Have a Standard?
Think for a moment how hard it would be to
bake a cake without standard units of
measurement. put gas into your car
without standard nozzle sizes.
plug a lamp into a socket without standard
electrical outlets.
32
Why Have a Standard?
Standards ensure consistency.
The standard for metadata ensures a level of
consistency in data documentation.
33
Establishing a Standard
The Metadata Standard
The Federal Geographic Data Committee (FGDC) was
organized in 1990 under the Office of Management
and Budget to promote the coordinated use,
sharing, and dissemination of geospatial data on
a national basis. The FGDC was tasked with
creating a metadata standard to meet these
objectives.
34
Establishing a Standard
The Content Standard for Digital Geospatial
Metadata (CSDGM)
"... each agency shall document all new
geospatial data it collects or produces, either
directly or indirectly, using the standard under
development by the FGDC, and make that
standardized documentation electronically
accessible to the Clearinghouse network."
Executive Order 12906, signed by President
Clinton in 1994.
35
Establishing a Standard

• This Content Standard
• serves as a uniform
• summary description
• of the data set.

36
Establishing a Standard

• The Content Standard utilizes...
• Common terms
• Common definitions
• Common language
• Common structure

37
Establishing a Standard

• The Content Standard
• Establishes names of compound elements and data
  elements.
• Defines the information about the values that are
  to be provided for the data elements.

38
Establishing a Standard

• The Content Standard helps
• the user determine...

• If a set of geospatial data is available
  and fit for a particular use.
• How to access and transfer the
• data set.

39
Establishing a Standard
Metadata written using the Content Standard
answers these important questions

• Who
• Who collected the data?
• Who processed the data?
• Who wrote the metadata?
• Who to contact for questions?
• Who to contact to order?
• Who owns the data?

Where Where were the data collected? Where were
the data processed? Where are the data located?
When When were the data collected? When were the
data processed?
What What are the data about? What project were
they collected under? What are the constraints on
their use? What is the quality? What are
appropriate uses? What parameters were
measured? What format are the data in?
How How were the data collected? How were the
data processed? How do I access the data? How do
I order the data? How much do the data cost? How
was the quality assessed?
Why Why were the data collected?
40
The FGDC Workbook
41
FGDCs Metadata Workbook
Defines the 334 metadata elements.
42
What do I use The Workbook for?

• It is the definitive resource for applying the
  FGDC Content Standard.

• It provides section and element definitions.

• It describes element domain values, which are
  valid values that can be assigned to the data
  element.

• However, it does not define
• the production rules.

43
Use the Graphical Representation for quick
access.

• It is a quick reference for production rules
  and structure.

• You will still need to use the workbook to
  find the definition of a particular element and
  its domain.

44
Playing the Metadata Game
45
Organization of the Content Standard
The Content Standard is organized using numbered
chapters called sections. There are 7 main
sections and 3 supporting sections. Each
section is organized into series of elements
that define the information content for metadata
to document a set of digital geospatial data.
46
Organization of the Content Standard
The Seven Main Sections
4
5
2
6
7
3
1
Data Quality Information
Spatial Data Organization Information
Spatial Reference Information
Entity and Attribute Information
Distribution Information
Metadata Reference Information
Identification Information
47
Organization of the Content Standard
Each section begins with the name and definition
of the section. These are followed by the
component elements of the section. Each
section provides the names and definitions of
its component elements, information about the
types of values that can be provided for the
elements, and information about the elements
that are mandatory or repeatable.
48
Interpreting the Graphical Production Rules

• The workbook uses graphics to illustrate the
  production rules of the standard. These graphics
  include most of the information provided by the
  production rules, including
• How elements are grouped.
• What is mandatory and what is not.
• What elements can repeat and how many times
  they can repeat.

49
Interpreting the Graphical Production Rules
Sections are depicted by this symbol.
50
Interpreting the Graphical Production Rules
A data element is a logically primitive item of
data. Data elements are the things that you fill
in.
The form for the definition of a data element is
Data element name -- definition. Type (choice
of integer, real, text, date, or
time) Domain (describes valid values that can
be assigned)
An example of the definition of a data element is
Abstract -- a brief narrative summary of the data
set. Type text Domain free text
Note Data element definitions are contained in
the text of the Content Standard, not in the
graphical production rules.
51
How Elements Are Grouped
Compound elements are composed of other compound
and data elements. The composition is
represented by nested boxes.
Compound Element 1
Compound Element 1.1
Data Element 1.1.1
Data Element 1.1.2
Data Element 1.2
52
How Elements Are Grouped
Compound Element 1 is composed of Compound
Element 1.1 and Data Element 1.2.
Compound Element 1
Compound Element 1.1
Data Element 1.1.1
Data Element 1.1.2
Data Element 1.2
53
Whats Mandatory? Whats Not?
Meaning
Mandatory - must be provided.
54
Repeating Elements

• If an element can be repeated
• independently from other elements, it
• will be indicated as such below the
• element name.

This group of elements would repeat.
Compound Element 1 Compound Element 1.1
Data Element 1.1.1 Data Element
1.1.2 Data Element 1.2
55
Using the Graphics to Make Decisions
Compound Element 1
Compound Element 1.1

• All elements are colored yellow, so all are
  mandatory and must be reported.

Data Element 1.1.1
Data Element 1.1.2
Data Element 1.2
56
Using the Graphics to Make Decisions

• Compound Element 1 is mandatory.
• Compound Element 1.1 is optional.
• If yes, Data Elements 1.1.1 and 1.1.2
  are mandatory.
• If no, do not report Compound Element
  1.1, Data Element 1.1.1 or 1.1.2, and skip
  to Data Element 1.2.
• Data Element 1.2 is mandatory.

57
Using the Graphics to Make Decisions

• Compound Element 1 is mandatory.
• Compound Element 1.1 is mandatory.
• Data Element 1.1.1 is mandatory.
• Data Element 1.1.2 is mandatory if
• applicable.
• Data Element 1.2 is optional.

58
Using the Graphics to Make Decisions

• Compound Element 1 is mandatory if applicable. If
  not applicable to the data set, do not report any
  elements. If applicable, it is mandatory and
• Compound Element 1.1 is mandatory.
• Data Element 1.1.1 is mandatory if
• applicable. If not applicable, do
• not report it. If applicable, it is
• mandatory.
• Data Element 1.1.2 is mandatory.
• Data Element 1.2 is optional.

59
Exercise 1 Using the Green Book
60
Metadata as a Data Discovery Tool
61
Discovering Data Through Metadata
The FGDC metadata clearinghouse is a
decentralized system of Internet servers you can
use to search for available geospatial data.
Servers housing metadata
Client
FGDC Gateway
62
Discovering Data Through Metadata
The descriptive information that fuels the FGDC
clearinghouse is metadata, which is collected in
a standard format to facilitate query and
consistent presentation across the multiple
participating sites.
63
A Brief Look at the FGDC Clearinghouse
The FGDC has six gateways to its clearinghouse
system, with access to over 250 spatial data
servers.
www.fgdc.gov/clearinghouse/clearinghouse.html
64
A Brief Look at the FGDC Clearinghouse
Searches can be performed by using the NSDI
Search Wizard, or by using a map interface with
place names, or by place names alone.
65
A Brief Look at the FGDC Clearinghouse
The new NSDI Smart Select Search Wizard bins
servers by the types of metadata they house.
66
A Brief Look at the FGDC Clearinghouse
Searches can be performed using a map interface
that allows the user to define an area of
interest.
An area of interest can be defined by dragging an
area of interest box on the map interface.
67
A Brief Look at the FGDC Clearinghouse
The selected area defines the bounding
coordinates that will be used in the search.
68
A Brief Look at the FGDC Clearinghouse
You can search all the servers listed, or you can
select only those that interest you.
69
A Brief Look at the FGDC Clearinghouse
Select individual servers of interest to your
search.
70
A Brief Look at the FGDC Clearinghouse
Search criteria can be further refined by time
period of content and keywords.
71
A Brief Look at the FGDC Clearinghouse
The status of each selected node is displayed as
the search is conducted.
72
A Brief Look at the FGDC Clearinghouse
When the search is complete, the status window
lets you know if you were successful in
discovering metadata that matched your search
criteria.
73
A Brief Look at the FGDC Clearinghouse
Select a server to see what metadata is
available.
74
A Brief Look at the FGDC Clearinghouse
Metadata discovered by the search is shown by
title.
75
A Brief Look at the FGDC Clearinghouse
Metadata record returned in HTML format.
76
A Brief Look at the FGDC Clearinghouse
77
A Brief Look at the FGDC Clearinghouse
The Coastal Information Directory (CID) at the
NOAA Coastal Services Center is similar to the
FGDC gateway interface, but the CID searches
only those spatial data servers that house
metadata of a coastal nature.
www.csc.noaa.gov/CID/
78
A Brief Look at the FGDC Clearinghouse
For more information on the clearinghouse system,
visit the FGDC Web site (www.fgdc.gov). Here
you can find information on how to establish your
own clearinghouse node using free Isite?
software. On-line tutorials provide assistance
for setting up and configuring this software.
79
Exercise 2
Searching for Metadata

• Access the FGDC Website at www.fgdc.gov.

• Click on the Clearinghouse link,
• and then click on Search for Geospatial Data.

• Choose a Clearinghouse Gateway.

• Decide how you are going to search.

• Perform a search using the keywords
• from the exercise Searching for Metadata.

80
Exercise 3
Reading a Metadata File
81
Writing Metadata
82
Its not so bad!

• First records are the hardest.
• Not all fields may need to be filled in.
• Tools are available.
• Training classes can be taken.
• Can often be produced automatically.
• Can (and should) be reviewed for updates.

83
Writing Metadata
Six Steps for Writing Quality Metadata.
84
Writing Metadata

• Before you begin writing, get organized.

• Document your data as you go.

• Write so others can understand.

• Always review your document.

85
Items required
FGDC Workbook
Metadata entry tool
Chocolate
Coffee
Sense of Humor!
86
Writing Metadata
Keep your readers in mind.

• Write simply but completely.
• Document for a general audience.
• Be consistent in style and terminology.

87
Writing Metadata
Keep your readers in mind.

• Define all acronyms.

• Avoid using jargon.

• Clearly state data limitations.

88
Writing Metadata
Write a complete title that includes

• What
• Where
• When
• Scale
• Who

89
Writing Metadata
The title is critical in helping others find your
data.
Which is better?
90
Writing Metadata
Be specific. Quantify when you can.
Vague We checked our work and it looks
complete. Specific We checked our work using 3
separate sets of check
plots reviewed by 2 different people. We
determined our work to be 95 complete based on
these visual inspections.
91
Writing Metadata

• Select your key words wisely.

• Use unambiguous words.
• Use descriptive words.
• Fully qualify geographic locations.

92
Writing Metadata
Review your final product.

• Have someone else read it.
• If youre the only reviewer, put it away and
  read it again later.
• Check for clarity and omissions.

93
Writing Metadata
When you review your work, ask

• Can a novice understand what you wrote?
• Are your data properly documented for
  posterity?

94
Writing Metadata
When you review your work, ask

• Does the documentation present all the
  information needed to use or reuse the data?
• Are any pieces missing?

95
Writing Metadata
Write so that others will understand.
96
Tool Time

• Metadata Creation
• and Validation

97
Tool Time
A sample of some of the available tools for
metadata creation, validation, and publication.

• TKME
• Text editor used for metadata entry.

• NOAA CSC ArcView Metadata Collector
• Extension for ArcView 3.x.

• Template tools
• Allows you to create a template record that can
  be used to create large numbers of similar
  records.

• CNS and MP
• Chew n spit, checks and corrects structural
  errors, and Metadata Parser, which checks for
  errors in element compliance.

98
Metadata Entry Tools
TKME - An editor for formal metadata, TKME is
intended to simplify the process of creating
metadata that conform to the standard.
99
Metadata Entry Tools
NOAA CSC ArcView Metadata Collector - The
ArcView Metadata Collection Tool was developed
by the National Oceanic and Atmospheric
Administration (NOAA) Coastal Services Center in
ArcView using the Avenue scripting language.
This tool collects and compiles Federal
Geographic Data Committee (FGDC)-compliant
metadata for ARC/INFO coverage's, shapefiles,
grids and supported image formats.
100
Tool Time
CNS (Chew n Spit) - A pre-parser for formal
metadata designed to assist metadata managers
convert records that cannot be parsed by mp into
records that can be parsed by mp.
Assumptions
The input file has metadata elements identified
by name elements always start with an element
name, which is the first alphabetic string of
characters on the line. CNS does not assume that
you have used underscores in the standard element
names, and if you have used nonstandard names for
standard elements, you can tell CNS how to
interpret them.
101
Tool Time
CNS does not require indentation of any sort. It
uses its knowledge of the standard and your
arrangement of the elements to structure the
metadata record. Because this method is not
flawless, careful inspection of the output from
CNS is the most prudent course of action.
102
Tool Time
Usage Cgt\cns options input_file Options
-v -c config_file -i info_file -a
aliases -e leftovers -o output_file
103
Tool Time
MP (Metadata Parser) - A compiler to parse formal
metadata, checking the syntax against the FGDC
Content Standard for Digital Geospatial Metadata
and generating output suitable for viewing with a
web browser or text editor.
Usage Cgt\mp options input_file
Options -c config-file -l language-code -e
error-file -t text-file -h html-file -f
faq-file -s sgml-file -x xml-file -d
dif-file -fixdoc
104
Tool Time
TKME, CNS, and MP are available as free downloads
from the United States Geological Survey (USGS)
Website. (geology.usgs.gov/tools/metadata) TKME
will run from a shortcut on the desktop, but both
MP and CNS must be run from a command line in
MS-DOS or UNIX.
105
Finally...
Remember, metadata is a legacy document that
concisely sums up your data or data set.
Without metadata, your data set is incomplete.
106
Michael Moeller Mike.Moeller_at_noaa.gov www.csc.noaa
.gov/metadata/