Data Quality Standards at the U.S. Census Bureau

1
Data Quality Standardsat the U.S. Census Bureau

• Pamela D. McGovern and John M. Bushery
• U.S. Census Bureau
• Quality Program Staff
• Washington, DC 20233
• European Conference on Quality in Official
  Statistics
• July 2008

2
Agenda

• Background
• Development Process
• Framework and Structure
• What Worked Well
• Next Steps
• Challenges

3
Background
4
Office of Management and Budget (OMB)

• Provides direction on Government-wide budget and
  management issues
• Issued Information Quality Guidelines (2002)
• Issued Standards and Guidelines for Statistical
  Surveys (2006)

5
U.S. Census Bureau

• Composed of separate program directorates
• Economic
• Demographic
• Decennial

6
Quality Program

• Initiated in 1998
• Purpose
• Advance and maintain the Census Bureau interests
  in quality of its products and processes
• Key players
• Methodology and Standards (MS) Council
• Quality Program Staff
• Reference Landman, et al., (Q2001)

7
Methodology and Standards (MS) Council

• Steering Committee for the Quality Program
• Chiefs of methodology research divisions and
  senior statisticians and methodologists
• Ensure sound statistical and methodological
  practices

8
Quality Program Staff (QPS)

• Support staff for the MS Council
• Facilitate development and implementation of
  standards and guidelines
• Evaluate implementation of standards

9
Development Process
10
Original Approach

• Informal process
• Team formed based on need
• No formal deadlines
• Detailed requirements
• Development periods lengthy
• Took from 1 to 4 years

11
New Approach

• Structured approach that uses
• Dedicated Staff
• Contractor Assistance
• Directorate Reviewers
• Focus Groups

12
Dedicated Staff (QPS)

• Two Mathematical Statisticians and one Survey
  Statistician
• Advantages
• Development times decrease
• Cohesion of content improve
• Consistency of format improve

13
Contractor Assistance

• Advantages
• Provides expertise in conducting focus groups
• Brings outside perspective
• Disadvantage
• Lacks familiarity with Census Bureau issues

14
Directorate Reviewers

• Focus group participants and program area
  managers
• Advantages
• Allows QPS to obtain directorate input early in
  process
• Promotes buy-in
• Disadvantage
• Risk exists that can lose buy-in

15
Focus Groups

• Seven to ten mid-level managers Two hours in
  length
• Advantages
• Forum for discussing important issues across
  directorates
• May increase buy-in
• Participants more attentive during the review
• Disadvantage
• Scheduling difficulties can slow down process

16
Process Steps

• QPS sends draft standard to reviewers
• QPS compiles comments and produces discussion
  issues for focus group
• Contractor conducts focus group
• QPS revises standard and conducts walk-throughs
  with the MS Council
• QPS prepares final draft standard and sends to
  reviewers along with responses
• QPS and quality assurance staff review the
  requirements to determine if they are auditable
• QPS submits set for final review and approval

17
Framework and Structureof the Standards
18
Framework

• Lists standards and their purpose
• Based on the OMB standards
• Based on our existing Census Bureau standards
  (See our Information Quality website
  http//www.census.gov/quality/)
• Consists of two types of standards
• Process Standards (18)
• Supporting Standards (5)

19
Process Standards

• Reflect the phases of the life cycle of a data
  product
• Planning and Design
• Obtaining Data
• Processing Data
• Analyzing Data
• Disseminating Products

20
Supporting Standards

• Apply to all phases of the life cycle
• Protecting Confidentiality
• Managing Data and Documents
• Ensuring Quality
• Applying for a Waiver

21
Structure of the Standards

• High-level requirements in list form
• State what is required avoid the how
• Include examples to provide guidance
• Provide links to reference documents and to
  related standards

22
What Worked Well
23
Accelerated Development

• Reduced development times due to
• Early input
• Straightforward requirements
• Individual walkthroughs with the MS Council
  members

24
Program Area Input

• Interviews with program area experts to get
  assistance in drafting requirements
• Focus group approach
• Structure the discussion based on comments
• Overlap participants on related standards
• Record the session

25
Focus Group Evaluation

• Scale
• 1 strongly disagree to 5 strongly agree

Characteristic Rating
The goal of the focus group was clear. 4.2
The participants point of view was heard. 4.5
The goal of the focus group was accomplished. 4.0
The length of the focus groups was sufficient. 4.2
The participants satisfaction with the outcome. 4.0
26
Next Steps
27
Next Steps

• Review the full set of standards for
  completeness, consistency, and cohesion
• Submit the standards to the directorates for
  final review and approval
• Expect completion by September 2009

28
Next Steps

• Implementation of the standards
• QPS assists the program directorates
• Training
• Evaluations of the implementation
• Identify aspects of the standards that need
  improvement

29
Challenges
30
Challenges

• Risk of excessive burden
• How compliance will affect costs and schedules
• High-level standards
• Avoid being too vague
• Program areas interpret standards differently
• Verification of compliance
• Audit program (Bushery et al., Q2006)

31
Thank you

• Contact Information
• Pamela D. McGovern
• 4600 Silver Hill Road
• U.S. Census Bureau
• Washington, DC 20233
• pamela.d.mcgovern_at_census.gov