Evaluation of Timber Committee Forecasts

1
Evaluation of Timber Committee Forecasts

• Goal Improve response rate and forecast accuracy
• What is the response rate
• What is the accuracy of the forecasts

2
Responses 2002-2004
3
Response Completeness- forecast for current year
4
Completeness of Response
5
Responses to specific item Sawn softwood and OSB
6
Forecast AccuracyConstraints

• One set of forecasts (current and next year) to
  limit initial work
• Data sufficiently far back so that forecast years
  would have more or less final data October 2002
• One product sawn softwood
• 31 replies to this item (although not every
  country forecast all 6 possible data points)

7
Summary Statistics

• Bear in mind
• More detailed statistics available in attached
  paper (excluding dropped outliers)
• This is only for one product, one set of
  forecasts
• No weighting by size of country

8
Forecast Error Distribution - Exports
9
Forecast Error Distribution - Imports
10
Forecast Error Distribution - Production
11
Significant errors

• What is a significant error?
• When the direction of the forecast is opposite to
  the actual outcome
• We used a cutoff of forecast gt1 and actual lt-1
  or forecast lt-1 and actual gt1

12
No change forecasts

• What is a no change forecast?
• Where the forecast change is between 0.5 and
  0.5
• Such a forecast could be the outcome of a careful
  process of consultation, estimation and modeling
  or simply rounding/repeating an earlier figure

13
Are the forecasts better than guessingresults of
EAD analysis

• 5 of the 6 series analysed showed a close to
  normal distribution (the exception was 2002/2001
  exports)
• The forecasts were better than random guessing in
  more than 80 of the cases (gt86 for most series)
• Forecasts appear to be more conservative than
  warranted by random outcomes
• There is still room for improvement

14
Future steps

• Analytical
• Cover earlier years (1997, 1987) to compare
  current forecast success with other periods
• Cover other products to see if trend in sawn
  softwood is representative (one roundwood, one
  panel)
• Accomplishing goals
• Reaching out to find more respondents
• Feedback to show how successful forecasts are,
  encourage forecasters to risk
• Training / networking to share knowledge and
  best practice