Estimating and treating discontinuities in business surveys

1
Estimating and treating discontinuities in
business surveys
(ONS)

• Pam Davies and Ceri Underwood
• Office for National Statistics

2
Overview
(ONS)

• Introduction
• Overview of ONS business register and its use in
  typical business surveys
• Updating the business register
• Methods for estimating the size of
  discontinuities in survey estimates and for
  dealing with them
• Conclusions.

3
Introduction
(ONS)

• One of the most important characteristics of
  economic survey estimates is that they be
  consistent over time
• Some information, such as the frame, is dynamic,
  and temporary consistency is enforced by updating
  the frame on a regular basis, and holding it
  fixed otherwise
• This introduces a discontinuity in the series of
  survey estimates at the point at which the frame
  is updated

4
Introduction - 2
(ONS)

• Estimating the size of this discontinuity usually
  involves some dual running of survey results
  using different frame information.
• This talk looks at estimates that can be obtained
  in a typical business survey situation and their
  effect on the continuity of the series produced.
• Where the discontinuity is noticeable, there are
  strategies available for making an adjustment to
  the series to make the discontinuity less
  pronounced.

5
The frame
(ONS)

• The frame for ONS business surveys is the
  Inter-Departmental Business Register (IDBR)
• The IDBR is a central database of businesses
  maintained by the ONS.
• It is based on two major administrative sources
• the Value Added Tax (VAT) system operated by the
  Department of Customs and Excise
• the Pay As You Earn (PAYE) system operated by
  the Inland Revenue.

6
The frame - 2
(ONS)

• The IDBR is used to conduct surveys for ONS and
  other government departments in four main ways
• to select businesses to be included in surveys
• to mail forms
• to enable estimates to be made for businesses
  that do not respond or (in aggregate) for
  businesses who were not asked to participate in
  the survey
• to produce statistical analyses on businesses.

7
Updating the IDBR
(ONS)

• Frame turnover information is updated as it is
  received from VAT in October each year
• Frame employment information is received from
  PAYE in August, but only updates the frame at the
  end of December

8
Assumptions
(ONS)

• Analysis is based on the assumption that there
  are no actual measurement errors
• So all survey measurements and register variables
  are accurate at the time of the update
• The only errors are thus induced by the fact that
  register variables become increasingly
  out-of-date, with the errors being worst just
  before the next update.

9
The impact on survey results
(ONS)

• The way in which this error affects survey
  estimates is likely to depend on how the register
  is used. Two key uses are identified here
• at the sample design stage, as a measure of size
  for stratification
• at the estimation stage, as auxiliary
  information.

10
Impact of the turnover update on register
data
(ONS)

• There is usually some, relatively minor, change
  in frame turnover information between successive
  quarters
• The vast majority of changes occur between
  quarter 3 and quarter 4 each year, when the new
  information from VAT is taken onto the IDBR

11
Updates to register turnover between Q2 97 and Q3
97
(ONS)
12
Updates to register turnover between Q3 97 and
Q4 97
(ONS)
13
The effects of the turnover updates on survey
estimates
(ONS)

• Data used for this analysis is taken from the
  quarterly inquiry into distribution and services
  sector
• Register turnover is used as the auxiliary for
  ratio estimation.
• Study concentrated on the turnover update between
  Q3 and Q4 1997

14
The effects of the turnover updates on survey
estimates
(ONS)

• To estimate the size of the discontinuity in
  survey estimates, four sets of estimates were
  produced
• using the Q3 sample and the Q3 frame
• using the Q3 sample and the Q4 frame
• using the Q4 sample and the Q4 frame
• using the Q4 sample and the Q3 frame

15
The effects of the turnover updates on survey
estimates
(ONS)

• For around 90 of the four digit industries
  studied, using the Q4 frame produced higher
  estimates than using the Q3 frame for both the Q3
  and Q4 samples.
• The turnover update between Q3 and Q4 has an
  effect on the survey results in many industries,
  but there is no consistency in the size of the
  effect across industries.

16
Adjusting for discontinuities
(ONS)

• The options
• re-run estimates for quarters immediately before
  and after the update on the old and new frames
• re-run estimates for a longer time series on both
  the old and new frames
• use regression estimation with old and new
  registers as auxiliary information to re-run back
  series of estimates

17
Revising the Q3 estimates
(ONS)

• Two linking factors were devised. The first one
  was based on the ratio of the estimates in
  respect of quarter 3. The second one was based
  on the ratio of the estimates in respect of
  quarter 4.
• An overall link was calculated as the mean of the
  two links.
• This overall link was applied to the original Q3
  estimate to produce a revised Q3 estimate.

18
Revising the Q3 estimates
(ONS)
19
Revising the back series
(ONS)

• For those industries for which the estimate for
  quarter 3 is revised it is also necessary to
  adjust the estimates for quarters 1 and 2 1997.
• The Q2 97 and Q1 97 estimates were revised using
  2/3 and 1/3 of the change induced by the overall
  link, respectively.

20
Conclusions
(ONS)

• If the aim is to produce estimates of change,
  then discontinuities in the time series of
  estimates are to be avoided.
• Where discontinuities are unavoidable it may be
  wise to have a period where old and new systems
  are run in parallel.
• This should make it possible to measure the
  effects of the change and link the two series.
• Explanations should be provided to users.