Estimation, Calibration, and Validation of Structural Simulation Models

1
Estimation, Calibration, and Validation of
Structural Simulation Models

• Sherman Robinson
• IFPRI
• July 2002

2
Papers

• C. Arndt, S. Robinson, and F. Tarp (2002).
  Parameter estimation for a computable general
  equilibrium model a maximum entropy approch.
  Economic Modeling, Vol 19.
• S. Robinson, A. Cattaneo, and M. El-Said (2000).
  Updating and Estimating a Social Accounting
  Matrix Using Cross Entropy Methods. Economic
  Systems Research, Vol. 13, No. 1.

3
Papers

• S. Devarajan and S. Robinson (2002). The Impact
  of Computable General Equilibrium Models on
  Policy. Presented at a conference on Frontiers
  of Applied General Equilibrium Modeling at Yale
  University, April 5-6, 2002.

4
Outline

• Model motivation and Design
• Policy focus
• Domain of applicability
• Model use
• Static/dynamic
• Dated/timeless
• Estimation, calibration, and validation
• Maximum entropy econometrics

5
Desiderata for Policy Models

• Relevance
• Link policy variables to outcomes
• Transparency
• Timeliness
• Validation and estimation
• Diversity of approaches

6
Domain of Applicability

• The domain of exogenous and endogenous variables
  for which the model is valid.
• Defined by choices in model design.
• Stylized models
• Applied models

7
Structural versus Reduced Form Models

• Desiderata argue for structural models
• Linking policy variables to outcomes
• Aggregate and structural indicators identifying
  winners and losers
• Transparency
• Black-box syndrome

8
Model Use

• Static versus Dynamic
• Comparative static
• Recursive versus forward-looking dynamic
• Timeless versus dated
• Timeless counterfactuals not intended to match
  actual data
• Dated projections that can be compared with
  actual data

9
Timeless Static Mode

• Standard usage comparative static
• Change a few exogenous variables and compare
  results to base solution.
• Counterfactual question
• What would the economy have looked like in the
  base year given the particular shocks under
  consideration?

10
Dated Static Mode

• Solve model for a second year some periods away
  from base year
• Comparative static
• Incorporate actual or forecast changes in many
  exogenous variables and parameters
• Historical or forward projection
• Results are identified with particular year

11
Timeless Dynamic Mode

• Steady-state equilibrium growth models
• Shock model and compare new steady-state path
  with initial steade-state path
• Analogous to timeless static Do not associate
  time path with historical time
• Examples
• Rational expectations dynamic CGE models
• Real business cycle models

12
Dated Dynamic Mode

• Projection models
• Recursive dynamic
• Forward-looking dynamic
• Results can be tested agains historical data
• Similar to dynamic macro-econometric models

13
Validation

• Validation by comparing explicit results with
  historical data can only be done for dated models
• Timeless models must be validated heuristically
• Rough comparision with historic episodes
• Reasonableness of specification and parameters

14
Parameter Estimation

• Two kinds of parameters
• Share parameters cost shares, expenditure
  shares, savings rates, tax rates, import and
  export shares
• Elasticity parameters describe the curvature of
  various structural functions (e.g., production,
  utility, import demand, export supply)

15
Parameter Estimation

• Share parameters can be estimated from a based
  SAM
• Benchmark calibration base data are assumed to
  be a solution of the model
• Estimating elasticities requires additional
  information
• Validation requires good estimation

16
Estimation and Validation

• In standard econometric practice, model
  estimation and validation are done together.
• Estimate parameters so that model tracks
  historical data as closely as possible
• Issue of domain of applicability
• Historical data must include information relevant
  for shocks to be analyzed

17
Estimation SAM

• SAM estimation
• Share parameters
• Timeliness
• Accounts define model structure
• Historical data versus average SAM for
  parameter estimation
• Issue of agriculture typical year

18
Estimation/Calibration

• Kehoe et al. Spanish economy
• Gehlhar backcasting with GTAP model
• Dixon et al. two-period calibration with Orani
  model
• Dervis, de Melo, and Robinson Turkish structural
  adjustment model

19
Validation Questions

• Can the model track historical flows?
• Which parameter values enable the model to track
  historical flows best?
• Trade parameters (import demand and export
  supply)
• Production functions
• Demand system

20
CGE Model

• A CGE model F(X,Z,B) 0 where
• F is a vector valued function
• X is a vector of endogenous variables
• Z is a vector of exogenous variables
• B is a vector of behavioral parameters
• number of equations number of unknowns.

21
Standard Econometrics
22
Standard Environment

• Lots of data relative to number of parameters
  being estimated.
• Strong assumptions about distribution of errors.
• Normal, mean of zero, single unknown parameter
  (variance).
• Assume no knowledge of parameter values. This is
  a moral imperative.

23
Standard Estimation

• Estimation criterion is to maximize within-sample
  prediction of various variables (Y).
• For efficient estimation, it is sufficient to
  represent the data by moments
• X'X and X'Y.
• Moments summarize all information needed for
  estimation.

24
Information Theory Approach

• Goal is to recover parameters that generated the
  data we observe. Focus is on parameter estimation
  rather than prediction.
• Assume very little information about the error
  generating process and nothing about the
  functional form of the error distribution.

25
Estimation Principle

• Use all the information you have.
• Do not use or assume any information you do not
  have.
• Zellner estimation using an efficient
  information processing rule.

26
Cross-Entropy Estimation

• View the model in the following form
• F(Xt , Zt , B) 0
• where t is a time subscript and
• Yt G(Xt , Zt , B) et
• where G(X0 , Z0 , B) e0 0,
• Yt is a vector of historical targets, and
• G(.) calculates target values from model results.

27
Cross-Entropy Estimation

• Impose most elements of Zt.
• Choose B (and some elements of Zt, particularly
  technical change) such that
• 1) The errors are small and
• 2) We are entropy close to our prior values on
  the elements of B.
• Choose the relative importance of the errors and
  our prior values.

28
Trade-off Between Error and Parameter Priors

• GDPt actual GDP tpredicted et
• B must fall within some (reasonable) bound with a
  priordistribution
• Blow lt B lt Bhigh
• The maximum entropy procedure attempts to choose
  errors close to zero and parameters, B, close to
  center of the prior distributions.
• Can choose the relative weights of the two
  criteria.

29
CGE Model Estimation
30
CGE Model Estimation
31
CGE Model Estimation
32
CGE Model Estimation
33
Cross-Entropy Measure
34
Lagrangian
35
First Order Conditions
36
Solution
37
Application to Mozambique

• Single country CGE model
• Six commodities and a commerce activity
• Two households rural and urban
• Rural and urban labor types
• Cobb-Douglas technology
• LES demand system
• Home consumption
• Armington import functions
• CET function on exports

38
Estimated Parameter Values

• Armington import elasticities by commodity.
• CET export elasticities by commodity.
• LES parameters by commodity and household.
• Technical change parameters by activity.
• Implicit subsidies to state owned enterprises
  during the period 1992-94.

39
Conclusion

• ME/CE estimation supports use of information in
  many forms and from many sources in estimating
  structural parameters
• Very powerful and flexible approach
• Particularly well suited for estimation of
  structural models

40
Conclusion

• Productive tension between policy applications
  and developments in theory, econometrics, and
  data.
• Advances in software, econometrics, and theory
  are narrowing the gaps and supporting productive
  collaboration between theorists, applied
  econometricians, and policy modelers.