SGER Collaborative: A Testbed for eRulemaking

1
SGER Collaborative A Testbed for eRulemaking
Data EIA 0327979, 0328175, 0328914 0328618
Jamie Callan Carnegie Mellon University Eduard
Hovy USC/Information Sciences Institute Stuart
Shulman Drake University Stephen Zavestoski
University of San Francisco dg.o 2004 Project
Highlight
2
Types of Collaboration

• Cost-sharing
• DG program 100K Collaborative SGER
• eRulemaking Initiative 115K supplement
• Data sharing
• Large public comment text data sets
• Strategic planning meetings
• DOT, EPA, HUD, OSHA, NARA, USDA, USFS
• Workshops and focus groups
• Agency personnel
• Interest group representatives

3
Scientific Research Objectives

• Applied Research Objectives
• Help agencies handle the information flood
• Measure efficiency quality improvements
• Basic Research Objectives
• Advance NLP IR
• Analyze and categorize floods of text according
  to several novel dimensions stakeholders,
  opinions, arguments
• Advance social science methods for measuring the
  impact of IT on democracy
• Develop metrics for assessing the quality and
  content of the public comment
• Develop a public comment database that
  facilitates greater empirical social science
  research

4
Accomplishments Text Testbed

• USDA National Organic Program
• 20,936 public comments (ASCII text)
• USDOT Hours-of-Service Regulations
• 22,954 public comments, 356 related documents,
  and 2 docket descriptor files (OCR PDF)
• USDOT Corporate Average Fuel Economy Standards
• 18,407 public comments, 42 related documents, and
  2 docket descriptor files (OCR PDF)
• More datasets are available in the pipeline
• All available to the Digital Government research
  community
• Full-text search bi-gram counts are up
• Expanded functionality over time
• http//hartford.lti.cs.cmu.edu/ERulemaking/D
  ata.html

5
Management Structure

• Four PIs at four academic institutions
• Independent work, centrally coordinated
• Group email, conference calls DC
  meetings/workshops
• One primary government partner
• Oscar Morales
• Director, eRulemaking Initiative
• One Project Director
• Dr. Stuart Shulman
• Drake University, Environmental Science Policy
  Program
• Regular Morales-Shulman communication
• Status reports, advising, strategy sessions
• Communication with the lead IT contractor
• Some challenges talking technology transfer

6
Success Story 2-Day Workshop

• Agency Participants
• DOT
• EPA
• USFS
• USDA-APHIS
• NARA
• FDA
• NSF
• Academics
• Sociology
• Political Science
• Computer Science
• Cognitive Psychology
• Students
• Law and policy

• Interest groups
• OMB Watch
• Defenders of Wildlife
• AFL-CIO
• Electronic Frontier Foundation
• Center for Democracy Technology
• Earthjustice Legal Defense Fund
• Taxpayers for Common Sense
• National Association of Home Builders
• Center for Public Environmental Oversight
• Association of Builders Contractors
• League of Conservation Voters

7
(No Transcript)
8
(No Transcript)
9
Broader Impact

• Technical developments
• New dimensions of text classification (CS
  research)
• Results annually evaluated in user studies (SS
  research)
• Integration of social and computer sciences
• Working out difficult research design issues
• Bridging methodological gaps
• Defining new integrative research domains
• Informed cross-agency dialogue
• Building social capital at the federal level
• Linking agency personnel, academics and
  stakeholders
• Calling attention to the consequences of IT
• Avoid the unreflective roll out of applications
  and architectures that no one likes or uses
• Encouraging public involvement transparency

10
Challenges and Barriers

• Applied vs. basic research
• The NSF favors the latter
• Agency partners favor the former
• Reward structures
• NSF favors high risk/payoff
• Agencies tend to be risk averse
• Timelines and budgets
• Funding takes time
• Planning and cultivating meaningful partnerships
• Interested government partners are ready to go
• Delays and distractions make it tough to get
  traction

11
Research Value in the DG Domain

• Real data that matters
• Problems are not just academic
• The value of citizen-government interaction is at
  stake
• Valuable insights from practitioners
• Stovepipe-busting ideas innovations
• Vast uncharted terrain
• New methods, metrics, and baseline data
• Technical innovations applicable also to other
  domains

12
Recommendations for Improving DG

• Digital Government created a forum where we
  (social computer scientists) could meet it
  gave us money that allows us to work together
  and it helped give us legitimacy so that the
  agencies would talk to us.
• Jamie Callan
• If it aint broke, dont fix it!
• Perhaps fund DG at the HSD level
• Lower the cost of dg.o meetings