Dynamic Data Displays: Experiences from USDA

1
Dynamic Data Displays Experiences from USDAs
Economic Research Service

• Vince Breneman
• GIS Coordinator
• USDA-Economic Research Service
• March 5, 2007

2
Economic Research Service Overview

• 400 employees (300 social scientists)
• ERS conducts a research program to inform public
  and private decision-making on economic and
  policy issues involving
• agriculture
• trade
• food markets
• food safety, nutrition, and assistance
• natural resources
• rural America

3
Data visualization objectives

• Clients
• Policy/legislative gatekeepers
• Press/Media
• Commodity industry/Agribusiness
• Research community
• Public
• Goal is to provide our clients with robust data
  products that will allow them to find, explore,
  extract data and build custom presentation
  products with some level of consistency

4
Challenges

• Lots of products often in the wrong format to
  create dynamic displays
• 100 data products that in include 1000s of
  datasets
• Most products are file based with meta data in
  the file
• A variety of users wanting different things from
  the data
• Some want a number
• a relationship
• a table or database
• graphics
• Finding data
• How do we expose data and information that are
  contained in our automated data delivery systems
• How do users find cross tabulations and
  geographic subsets etc.

5
Challenges

• Departmental guidelines
• limited screen real-estate 767 pixel width,140
  pixel header, ¼ left nav box, ¼ right nav box
• 508 compliance must provide reasonable access
  to the visually impaired
• Popups not advised because of popup blockers
• Departmental guidelines have and will change so
  building applications with a level of flexibility
  becomes part of our requirements

6
Commercial packages we use for developing Web
data visualization applications

• Cordas PopChart graphing tool
• Generates flash files as well as jpgs, eps, pdf
• Provides some standard graph types but we have
  done a bit of customization
• To reduce development time and provide some
  consistency we have been building graphing web
  services
• Provides a d-link for visually impaired
• ESRIs ArcIMS 9.1
• for mapping of detailed geographies generally
  county level of lower
• Utilizes our SQL Server and spatial data
  repository
• Able to incorporate a host of spatial reference
  layers

7
ERS data sets page Lots of products Departmental
guidelines Finding data
8
ERS food consumption application

• Provides food availability estimates of food
  supplies moving from production through marketing
  channels for domestic consumption.
• U.S. national level annual time series 1909 -
  2005
• Includes hundreds of commodities from beef and
  pork to coffee and tea to potato chips

9

• Food Consumption data selection page
• - Application instructions
• - User defined graph-able subset
• Output options
• Challenge
• Wanted to show over 100
• variables creating a LIL
• (Long Intimidating List)

10
Solution Break list into categories Challe
nge Application description and data query
component takes up lots of screen real-estate
11
Solution Jump return page to data selection
and results Challenge Want to illustrate the
group total and commodity shares - this can
create scaling issues
12
Solution Allow user to add or delete items
from selected group
13
PopChart features - Graph dynamically scales
Y-axis for a given set of data - PopCharts
generates flash files with rollovers to get
actual values -PopCharts provides d-link for
visually impaired
14
(No Transcript)
15
Challenge Wanted graph to standalone and be
clear about what it was representing Solution In
clude extensive footnotes
16
Challenge How do we handle to many variables?
17
Solution Panel plots?
18
Profiles of America mapping applicationspatial
data

• Purpose
• Help users visualize over 100 important US county
  level demographic and socio-economic indicators
  and a number of ERS rural indicators/geographies
• Summarize demographic data by rural geographies
  and by State
• Allow the application to be extended with new
  indicators of regional geographies
• Outputs include maps, graphs and tables
• This is one of our first highly interactive data
  visualization products

19
Regional geographies/typologies
Rural-Urban Continuum Code, 2003 County Typology
Codes, 2004 Metro-Nonmetro status, 2003
Metro-Micro-Noncore status, 2003 Urban
Influence Code, 2003
20
Example Demographic and Socio-economic data
Population and Migration Age and Sex Race and
Ethnicity Educational Attainment Households and
Families Journey to Work Employment and
Unemployment Income and Poverty
Unemployment rate, 2003
21
How do unemployment rates compare between metro
and nonmetro regions?
22
(No Transcript)
23
State selection US default
24
Topic groupings
25
Rural indicators
26
Challenge How to map 3000 counties
quicklyunder 2 seconds? Solution Currently we
use ESRI ArcIMS - thin client
27
Regions view Standard map navigation
28
Issue Do you just show the state or surrounding
states? Challenge Where to display identity
tool results?
29
Solution Used pop-ups - Now posting table
results below the map
30
Challenge Dynamic legend vs. predefine
legends Rural indicators use pre-define
legends Solution Build in a and RGB lookup for
the indicators
31
Challenge How to link map regions and chart
regions Solution Used color linkages
32
Same population change variable aggregated to a
different geography
33
How to aggregate data?

• Make sure all variables were using the same
  county base geography
• Develop a data sturcture to handle regional
  aggregations
• Identified variables as totals, ratio or
  categorical
• Ratio data (rates, percentages) need to know the
  numerator and the denominator by region
• Totals sum variable by region

34
Agricultural Resource Management Survey (ARMS)
application

• ARMS is USDA's primary source of information on
  the financial condition, production practices,
  resource use, and the economic well-being of
  America's farm households
• Its a large and complicated dataset with over
  650 variables before you start summarizing the
  data with various categorical variables

35
Its a large complicated dataset with over 650
variables before you start slicing and dicing All
these variable are embedded in an application and
its quit difficult to wrap a search engine
around it Challenge How do you search this
variable list and provide useful result to the
user? Solution ???
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
Commercial farms Very large family farms,
non-family farms Intermediate farms farming
occupation/high sales farms, large family
farms Rural residence farms limited-resource,
retirement, residential/lifestyle, farming
occupation/low sales 2005 view of farm assets
per farm
40
2005 view of farm assets Percent Challenge How
do we show farm assets by farm typology over
time? How do we provide statistical significance
to the estimates?
41
Solution Panel plots Provide confidence
bounds Issue We had to calculate confidence
bounds in the application.Would prefer the
graphing software to handle it. Challenge How
do we show farm assets by region?
42
Solution Charts on a map
43
Farm assets over time by region with confidence
bound
44
Challenge There are several ways to show the
same data each with different users in mind
Solution Display data in multiple formats
45
Challenge How to provide an easy way for
comparison of multiple graphs? Solution Here
grabbed jpgs and lined up in PowerPoint
Farm household income from farming and from off
farm sources
46
Useful approaches for developing Web data
visualization applications

• Identifying a few graphing products to
  concentrate developmentCordas PopChart and ESRI
  ArcIMS
• Developing data exploration apps for in house use
  provides utility to research staff and a test
  bed for external applications
• In house usability testing really helps you
  identify the common information people are
  looking for from a dataset/application

47
Concluding thoughts

• Building dynamic data displays helps ERS and its
  clients in a number of ways
• Makes data products more accessible and usable
• Provides ways of turning data into information
• Provides better ways to find information
• Provides incentives to organize and manage our
  data in better ways supporting both our research
  and data dissemination missions