Towards a Research Agenda for Official Statistics

1
Towards a Research Agenda for Official Statistics

• Lars Lyberg
• Statistics Sweden
• ETTS 2009 Brussels
• February 18-20, 2009
• Email lars.lyberg_at_scb.se

2
The current scenario

• An overwhelming number of initiatives
• 150 ESS RD groups (advisory, coordination,
  executive boards, expert, working, organizing
  committees, round table, steering, technical,
  task forces)
• Problems with priorities, finishing off, and
  implementation

3
Examples of initiatives

• A mixture of traditional methodological areas and
  more visionary ideas
• Traditional areas confidentiality, nonresponse,
  small area estimation, harmonization, response
  burden
• More visionary ideas model-based statistics,
  combining data from different sources, more
  efficient collaboration models such as ESSnet,
  standardization and evaluation efforts such as
  rolling reviews, Code of Practice, LEG on
  Quality, the new Regulation

4
So what is lacking?

• A concerted effort
• A project prioritization process based on
  strategic goals for the ESS
• Recognizing risks
• Involving the stakeholders even more
• Taking care of all links in the statistics
  production process (i.e. more on
  conceptualization, response process, measurement
  errors, quality control)
• Systematic implementation and evaluation of best
  practices

5
Anything more that is lacking?

• The systematic division of labour
• NSIs have similar challenges
• NIPS, CENEX, ESSnet, sharing results
• Capacity building (ESS competence development)
• Cooperation with universities and agencies
  world-wide
• Standards

6
A dogs breakfast
7
Having said that..

• Admittedly the general conditions are difficult
  and vary a lot
• The total openness about ESS problems is
  admirable
• Considerable persistence and success in some work
  areas
• Lots of work laid down on
• Subject-matter
• Methodology
• Strengthening the role of official statistics
• Quality
• Stakeholder dialogue
• Lots of recommendations provided

8
Lets go back to the ESS basics

• ESS shall provide reliable, comparable statistics
  for EU decision making, monitoring and evaluation
• Eurostat shall ensure that methods, criteria and
  practices within ESS are as harmonized as
  possible and statistics as comparable as possible
• NSIs and MSs shall provide national data and
  technical expertise and advice
• Implicit is that production should be efficient

9
How central are these dimensions in ESS RD work?

• Reliable should probably read accurate
• Small errors
• Stable processes that deliver what they are
  supposed to
• Important error sources left behind
• Comparative
• Hidden under the harmonization label
• Generic to ESS but comes out as an add on
• Efficient production
• Central in many RD projects

10
Accurate data

• Do not leave any key processes behind
• Develop and/or apply standard operating
  procedures (SOP) (quality assurance)
• Validate process performance (quality control)
• Allocate assurance and control resources with a
  risk management perspective
• Quantify total survey error (TSE)
• How should TSE be interpreted in a comparative
  setting?
• Resources outside ESS TSE workshop, other
  specialized networks and conferences, SOPs and
  guidelines

11
Comparative issues

• What could and should be standardized
  (harmonized)?
• How to achieve procedural equivalence?
• How to adjust concepts so that we get a uniform
  meaning across countries?
• How should differences in funding and methodology
  resources be handled?
• How should national pride and conflicts of
  interest be handled?
• Resources outside ESS CSDI network, guidelines
  for comparative research published by U o
  Michigan and CSDI

12
Controlling Quality
13
Hypothetical project priority matrix
14
Matrix use

• Project ideas submitted
• Each project suggestion scored by
  10x110x210x39x4..5x9
• Scoring is done by project management group
• Rolling RD plan
• Projects executed in ranking order

15
Endnote

• More search for and use of what is already
  available
• Continue the work on simplified coordination
• Collaborate globally
• The new Regulation should be operationalized such
  as Article 10 European statistics shall be
  developed, produced and disseminated on the basis
  of uniform standards and of harmonized methods
• Emphasize the comparative aspect of ESS
• Implementing best practices