Quality considerations in Statistical Surveys

1
Quality considerations inStatistical Surveys
Mukesh K Srivastava FAO Statistics Division
2
What is Quality?
3
Concepts What is Quality?

• Means different things to different people
• ease in use and comforts (beds sheets, clothes)
• reliability (long life)
• risk associated with use of product (electrical
  goods)
• value for money (substitute perfume)
• credibility (news paper)
• Association of quality with Brand names
• efforts to establish credibility (advertisements)
• Giorgio Armani vs. DOS, MOA

4
Concepts Scope of quality

• Product
• Characteristics
• Process (computerized eye testing)
• Inputs
• Mozzarella di Buffala
• Vino di Toscana
• Champagne of specific region of France
• Formal statements of quality need a framework
• an agreed set of characteristics/variables on
  which information is to be provided for
  comparison
• Total Quality Management
• INPUT- THROUGHPUT- OUTPUT

5
Quality in Official Statistics

• National Quality Assessment Frameworks
• http//unstats.un.org/unsd/dnss/QualityNQAF/nqaf.a
  spx
• A review reveals that the quality does not have
  the same meaning across the Globe, though there
  is broad consensus on its importance and key
  characteristics.
• Distinguish between
• Quality of Data
• Quality of Survey
• Quality of the Statistical system of a Country
  (Country Assessments of Global Strategy)

6
Quality of Statistics Product

• How close to Reality?
• Errors measured by distance from reality
• Sources of Errors
• Due to sampling and estimation procedure
• Non-sampling errors (applicable to both sample
  survey and complete enumeration)
• Coverage
• Measurement
• Response

7
Measures of quality of survey data

• Estimates of Sampling errors a measures of
  reliability of the estimate
• (closeness to the true parameter)
• Depends upon
• Sampling design
• The estimator (formula) e.g. mean, median, mode
  as measure of central tendency
• The values in the specific sample (bad sample)
• Statement about efforts to control non-sampling
  error
• extent of coverage
• quality of response (post-enumeration survey)

8
Process elements of survey quality

• Decision to undertake survey
• user needs analysis
• survey objective definition
• Survey design
• survey concepts
• sample design
• development and testing of measurement
  instruments
• Data collection
• data sources
• training of enumerators
• non response
• Data capture and data processing
• data capture
• editing and imputation procedures

• Data Analysis and output quality
• Relevance, accuracy, timeliness, comparability,
  coherence, data analysis, disclosure control
• Documentation and dissemination
• metadata documentation
• dissemination strategies
• data management
• Improvement cycles
• adaptability/flexibility
• expertise in relevant areas
• quality management

9
How to achieveQuality in Official Statistics?

• UNSD
• Principles governing international statistical
  activities
• Fundamental Principles of Official Statistics
  http//unstats.un.org/unsd/methods/statorg/FP-Engl
  ish.htm
• EUROSTAT
• National Methodological Report on survey (model
  Ref. doc.)
• The European Self Assessment Checklist for Survey
  Managers (Ref. doc)
• MEDSTAT workshop

10
???? ?????

• Message
• Statisticians should make an effort to develop a
  culture of quality in their domain by
  disseminating information on quality along with
  data.
• Quality information helps in brand positioning
• Quality information increases demand of the
  products and thus enhances revenues/resources.