TOOLKIT for Revsion Studies

1
TOOLKIT - for Revsion Studies
2003-4 System of National Accounts Revision
Project Statistics Canada September 2004
1
2
IN THIS PRESENTATION
2003-4 CSNA RP Toolkit

• Why study revisions
• Toolkit in revision studies
• Toolkit content
• Toolkit companion Workout
• Sample applications of the Tookit
• Future development
• Your feedback

2
3
WHY STUDY REVISIONS
2003-4 CSNA RP Toolkit

• Statistical users and providers are both
  concerned about data quality.
• Main dimensions of quality relevance,
  coherence, timeliness, accuracy and reliability.
• Preliminary estimates are provided primarily for
  timeliness, while final estimates are deemed to
  be accurate or the most accurate possible.
• As well as being timely, preliminary estimates
  should be reliable or are close approximates of
  final estimates.
• Revision studies aim to measure the reliability
  of the preliminary (and intermediate) estimates.
  The focus is on revision patterns and their
  change over time.

3
4
TOOLKIT IN REVISION STUDIES
2003-4 CSNA RP Toolkit
Previous revision studies other references
Source data
Stakeholders, advisory and review panels
Analyst
Manual
Data for revision studies
TOOLKIT Workout
Analytical report
Summary statistics and graphics
4
5
2003-4 CSNA RP Toolkit
TOOLKIT CONTENT

• Nature of revision studies
• Reliability and accuracy
• Types of revisions
• Forms of revision studies
• Analysis
• Analytical aspects data vintages, component
  series, links between subject matters, types of
  analysis
• Revision summary measures and graphics
• General
• Subject-matter specific
• Component share and impact analysis
• Regression and correlation
• Statistical inference
• Probability distributions
• Confidence intervals

5
6
2003-4 CSNA RP Toolkit
TOOLKIT COMPANION WORKOUT

• The revision studies manual comprises of the
  TOOLKIT and its companion WORKOUT
• TOOLKIT - document describing revision studies
  concepts, detailed analytical techniques and
  computations, and illustrations
• WORKOUT - spreadsheets in which TOOLKIT
  techniques and computations are applied to
  demonstrate how they work.
• The Toolkit and Workout can be used independent
  of each other but their content is linkable. For
  example, links between some of Workouts
  spreadsheets and their corresponding Toolkit
  sections are shown below.

6
7
2003-4 CSNA RP Toolkit
TYPES OF REVISION

• Revisions can be classified by reasons or by
  timing
• Reasons
• Incorporation of better source data
• Routine recalculation
• Improvements
• Correction of errors
• Timing
• Current revisions
• Annual revisions
• Historical (or comprehensive, major or
  benchmark) revisions
• More details Carson, C.S., Khawaja, S. and
  Morrison, T.K. Revisions Policy for Official
  Statistics A Matter of Governance. Sixteenth
  Meeting of the International Monetary Fund
  Committee on Balance of Payments Statistics,
  Washington D.C., December 1-5, 2003.
  BOPCOM-03/3A, pp 5-8.

7
8
2003-4 CSNA RP Toolkit
FORMS OF REVISION STUDIES

• Revision studies can take the following forms
• Study on levels or growth rates
• Comparison between ranges of time periods (e.g.
  decades)
• Component profiles and impact of sub-aggregates
• Time profiles (different vintages)
• International comparisons
• Review of source data and estimation methods
• Selection of studies based on the number of
  observations
• Combined graphics and narration suitable for
  studies with limited number of observations, for
  example in the early development of revision
  studies or during transition periods of the
  series being studied
• Calculation of summary measures - at least 24
  (20, 10) observations for monthly (quarterly,
  annual) series
• Significance testing - when applying analytical
  techniques associated with normal distributions,
  at least 30 observations are required. However
  confidence in the results can only comes with
  over 60 observations.

8
9
GENERAL SUMMARY REVISION MEASURES
2003-4 CSNA RP Toolkit

• Summary measures of bias and dispersion are
  generally applicable to all statistical series
• Central tendency mean, median and mode
• Number of positive revisions as a proportion of
  the total
• Spread standard deviation (s.d.), range, mean
  absolute revision (m(Abs)), etc
• Relativity ratio of revisions relative to the
  original series e.g. relative mean revision,
  relative standard deviation

9
10
MORE COMPLEX REVISION ANALYSIS
2003-4 CSNA RP Toolkit

• Linear relationship between statistical series
  and their revisions
• Regression
• Correlation
• Significance testing
• Probability distributions
• Confidence intervals
• Hypothesis testing
• Tests for efficiency
• Beta and Rho tests

10
11
SAMPLE USE OF THE TOOLKIT PRELIMINARY
FINAL ESTIMATES TOTAL REVISIONS
2003-4 CSNA RP Toolkit
Quarterly GDP Growth Rates and Total Revisions,
1981 to 1999 Seasonally adjusted, constant price,
quarter-on-quarter, point
11
12
RP0304
SAMPLE USE OF THE TOOLKIT INCREMENTAL
REVISIONS _at_ DIFFERENT VINTAGES
2003-4 CSNA RP Toolkit
12
13
SAMPLE USE OF THE TOOLKIT BIAS DISPERSION
2003-4 CSNA RP Toolkit
Quarterly GDP Growth Rates and Total Revisions,
1981 to 1999 Seasonally adjusted, constant price,
quarter-on-quarter, point
13
14
SAMPLE USE OF THE TOOLKIT DISTRIBUTION OF
THE REVISIONS HOW CLOSE TO BEING NORMAL?
2003-4 CSNA RP Toolkit
14
15
2003-4 CSNA RP Toolkit
SAMPLE USE OF THE TOOLKIT REGRESSION AND
CORRELATION
15
16
2003-4 CSNA RP Toolkit
SAMPLE USE OF THE TOOLKIT SIGNIFICANCE
TESTING
The correlation in the previous slide (No. 15) is
statistically significant, based on an F-test
with a test statistic of 13.473, which is in the
rejection region described by the F-value 3.983.
16
17
2003-4 CSNA RP Toolkit
SAMPLE USE OF THE TOOLKIT ECONOMIC EXPANSION
AND CONTRACTION
17
18
2003-4 CSNA RP Toolkit
SAMPLE USE OF THE TOOLKIT - IMPACT OF
SUB-AGGREGATES
18
19
FUTURE DEVELOPMENT will be made in other
2003-4 CSNA revision project modules
Module 2 Income expenditure components 1993
to 2003 Components with biggest impact SNA93
influence
Module 5 Productivity growth 1981-2003
Module 1 GDP growth rates Core module 1981
to 2003 1980s versus 1990s All
vintages Economic cycle indicator
Module 6 Input-output 1999 to
2000 Benchmarks Significant cells or cell
groups PIPES effect HST
Module 3 Component Provinces 1997 to
2002 Provincial differences PIPES effect
Module 4 Component industries 1993 to
2002 Industries with biggest impact Changes
over time
19
20
2003-4 CSNA RP Toolkit
Your feedback is most welcome. Please
write to Doris de Zilva Doris.dezilva_at_statcan.ca
System of National Accounts Statistics
Canada RHC 21A Tunneys Pasture Ottawa K1A
0T6 Ontario CANADA
20