Evaluation of Census Quality and Coverage

1
Evaluation of Census Qualityand Coverage

• Preston Jay WaiteAssociate Director for
  Decennial CensusU.S. Census Bureau
• Presentation Before the Joint ECE/Eurostat Work
  Session on Population and Housing Censuses
• Geneva, Switzerland
• November 23, 2004

2
Summary of the REX Program 1940-1990

• 1940 Census included research and testing prior
  to and during the census
• 1950 Census initiated REX program identified
  sources and measures of error and evaluated
  coverage
• 1960 Census included content and coverage
  evaluations and a post-enumeration survey
• 1970 Census included studies of content error,
  procedural evaluations, and an experiment to
  expand the mail census
• 1980 Census consisted of content, coverage
  improvement, and processing evaluations,
  experiments, and a post-enumeration survey
• 1990 Census similar to the 1980 program in
  content and scope

3
Overview of the Census 2000 TXE Program

• The Census 2000 TXE Program was the most
  ambitious formal test and evaluation program ever
  undertaken by the Census Bureau.
• 87 Evaluations assessed the effectiveness of
  operations, systems, and processes, including the
  national paid advertising campaign, the automated
  data capture system which included OMR and OCR,
  address list development operations, coverage
  improvement operations, and a new multiple
  category option for race reporting.
•  
• 4 Experiments were conducted during Census 2000
  because the decennial census environment provided
  the best conditions to learn about new
  methodologies. These included an alternative
  questionnaire experiment, an administrative
  records experiment, Social Security Number,
  attitudes, and privacy experiment, and a response
  mode and incentives experiment.

4
Overview of the Census 2000 TXE Program (contd)

• Operational profiles documented number of cases,
  rates, data from production files, quality
  assurance, and information collected from
  enumerator or other staff debriefings.
• Evaluations analyzed, interpreted, and
  synthesized the effectiveness of census
  components and their impact on data quality or
  census coverage.
• We also conducted sample surveys, targeted
  reinterview surveys, and customer satisfaction
  surveys.
• Some evaluations included focus group interviews
  to learn how or why respondents behave in a
  certain way.
• We also used ethnographic and social network
  methods to study the effects of mobility on
  census coverage.

5
Coverage Measurement in the 1950-1990 Censuses

• 1940 - An evaluation of the 1940 Census
  demonstrated the need for an assessment of census
  coverage. A match of draft registration records
  to the 1940 Census evaluated the coverage of
  adult males of an age eligible for the draft.
  Surprisingly, there were more males registered
  for the draft than enumerated in the census.
• 1950 and 1960 - The Census Bureau conducted its
  first post-enumeration survey after the 1950
  Census. The 1950 and 1960 post-enumeration
  surveys did not use dual system estimation to
  estimate census coverage error. Rather the idea
  was to form a much higher quality census
  interview with highly trained interviewers. The
  results of the post-enumeration survey interview
  were assumed to be the truth.
• 1970 - The Census Bureau did not evaluate the
  1970 Census using a post-enumeration survey. A
  decision was reached to use demographic analysis
  as the method for coverage evaluation in the 1970
  Census.
• 1980, 1990, and 2000 - implementations used dual
  system estimation.

6
The Census 2000 A.C.E. Survey

• Litigation and controversy surrounded the use of
  a coverage measurement survey to adjust the
  census throughout the 1980s and 1990s. In
  January 1999, the U.S. Supreme Court ruled that
  the law covering the census, Title 13, did not
  permit sampling for apportionment of the House of
  Representatives.
• The Census 2000 A.C.E. followed the basic
  methodology of post-enumeration surveys using
  dual system estimation.

7
A.C.E. and A.C.E. II Results

• March 2001 - A.C.E. estimates became available in
  time to correct the Census 2000 redistricting
  files. However, on March 1, 2001, the Census
  Bureau released the "Report of the Executive
  Steering Committee for Accuracy and Coverage
  Evaluation Policy" which reported that we were
  unable to conclude, based on the information
  available at the time, that the adjusted Census
  2000 data were more accurate for redistricting.
  Accordingly, we recommended that the unadjusted
  census data be released as the Census Bureau's
  official redistricting data.
• October 2001 After conducting further
  evaluations, the Census Bureau released the
  A.C.E. II report. After assessing considerable
  new evidence, we recommended that the unadjusted
  Census 2000 data also be used for
  non-redistricting purposes.

8
Revised A.C.E. II Results

• March 2003 - The Census Bureau decided to produce
  intercensal population estimates based on the
  official Census 2000 results rather than on a
  population base adjusted using the revised
  estimates of census coverage. The decision was
  based on the results of further work to correct
  for detected errors in an effort now known as
  A.C.E. Revision II.
• The Accuracy and Coverage Evaluation (A.C.E.)
  Revision II methodology represented a dramatic
  improvement from the previous March 2001 A.C.E.
  results. However, several technical concerns
  remained, including uncertainty about the
  adjustment for correlation bias, errors from
  synthetic estimation, and inconsistencies between
  demographic analysis estimates and the A.C.E.
  Revision II estimates of the coverage of
  children.

9
The 2010 Census Research Program Differences
from 2000

• The proposed 2010 Census program will make a
  distinction between true evaluations and
  accounting type assessments.
• The Census Bureau executive staff plans to decide
  the high-level scope and focus of research
  programs before participating divisions develop
  research proposals. This will ensure that the
  selected evaluations and experiments address the
  most relevant topics and program funding is
  optimally expended.
• Planning and design milestones for 2010 Census
  research components are slated to occur earlier
  than in the Census 2000 cycle.

10
The 2010 Census Research Program Areas of
Interest

• The Master Trace Sample database was designed to
  help answer research questions that go beyond
  those addressed by the evaluations and
  experiments. The 2010 Master Trace Sample
  database should be expanded to include data on
  Group Quarters and coverage measurement data
  associated with persons.
• We want to measure the effectiveness of
  procedures for unduplication, geocoding accuracy,
  the effectiveness of race reporting, the effect
  of residence rule strategies, and the impact of
  expanded use of automated tools by our field
  enumerators.
• We intend to continue research on the use of
  administrative records to supplement development
  of the address list and to obtain data for
  nonresponse items.
• We expect to continue improving upon efforts to
  measure census coverage, including net error as
  well as components of under- and over-enumeration.