Master conomie et Affaires Internationales

1
20 KEYS TO BETTER SIMULATING

• Master Économie et Affaires Internationales
• Paris Dauphine -October 2007
• Dr. Ramón Mahía
• Professor of Applied Economics Department
• www.uam.es/ramon.mahia

2
20 KEYS TO BETTER SIMULATING
STRUCTURE OF DOCUMENT AND EXPOSITION

• Not a technical document
• Our experience in credible simulation for
  complex systems in the real world
• Text focused on know-how to do useful real
  simulations rather than technical skills
• Simulation as a way of using econometrics in a
  useful way with analytical restrictions

3
20 KEYS TO BETTER SIMULATING
STRUCTURE OF DOCUMENT AND EXPOSITION

• I. What does Simulation mean?
• Concept
• Simulation Vs other topics
• Some Simulation last-names
• II. Basic elements of a Simulation Model
• III. 20 Keys for an Efficient Simulation

4
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?

• To represent something, feigning or imitating
  what it is not
• Simplified representation of a real complex
  system useful for
• Understanding the working of a real system
• Experimenting with, for evaluating different
  strategies to be developed on it.

5
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?
6
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?
Real System A lot of elements inter-related
Simulated System Few elements and selected
relations
7
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?

• Arrival of powerful computers and flexible
  programming systems generalized use of
  simulation

Modeler Use of complex techniques
User Use of simple interfaces
8
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?

• Simulation with a model is a wide spread
  exercise
• Macro economic relations trade, labor, supply
  demand,
• Financial markets treasury ship, stock exchange,
  commodity prices
• Technology innovation processes
• Business strategy games
• Business organization games

9
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?
FINANCE Wall Street Raider
10
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?
MARKETING MarktStrat
11
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?
BUSSINES ORGANISATION EIS Game
12
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?
MACRO-POLICY National Budget Simulation
13
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?

• Borders between Simulation, Forecast and
  Structural Analysis are diffuse.

• But the analytical approach, technical
  resources, and ways of use permit us to
  distinguishes it.

14
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?

• Simulations Vs. Optimization
• Optimization systems concentrates mainly on
  reaching a well predefined objective given a set
  of restrictions.
• Simulation is an open strategy that use the links
  between inputs and outputs without setting a
  priori what must be considered an optimum
  solution.
• Thats why we usually say that simulation models
  are not resolved, but runned.

15
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?

• Simulations Vs. Optimization Design car routes
  to pick up employees to the factory from distant
  locations

OPTIMISATION
SIMULATION

• Objective function
• Minimize time

• Inputs
• Cars
• time to finish
• of passengers

• Restrictions
• 3 cars
• 1 hour to finish
• 13 passengers

• Ouputs (results)
• Route design

• Results
• Route design

16
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?

• Deterministic (MKT Mix effects evaluation)

• Deterministic Inputs (controlled values)
• Advertising effort
• Price policy
• Distribution policy
• Sales Force

• Random (Agricultural Crop Yield)

• Random Inputs (not controlled values)
• Climate conditions
• Plagues

17
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?

• Deterministic Random (MKT Mix evaluation)

• Deterministic Inputs (controlled values)
• Advertising effort
• Price policy
• Distribution policy
• Sales Force

• Inputs to be randomly modeled (forecasted)
• MKT Mix of existing or new competitors
• Economic conditions of country
• Market Demand (2nd stage input)

Regression analysis
18
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?

• Static Vs. Dynamic Does passing of time
  result in a key variable (even an input) for the
  simulation system?
• Example Time as a basic input of a simulation
  system (two real examples)
• Economic impact of an immigration flow will not
  only depend on the amount of immigration, but in
  the speed of it.
• Effects on prices or trade flows of a tariff
  removal between two areas, will drastically
  change if you consider a gradual removal or not.

19
20 KEYS TO BETTER SIMULATING
I. WHAT DOES SIMULATION MEAN?

• Time as a basic input..(example from Femise)

20
20 KEYS TO BETTER SIMULATING
II. BASIC ELEMENTS OF A SIMULATION MODEL?

• (i) System to be analyzed.
• The collection of elements and its interactions
  which is trying to be analysed by means of the
  simulation.
• It is critical to identify the system (or
  sub-system) of interest and concentrate the
  effort on a suitable dimension, but..
• .without missing the links with the rest of the
  systems or other collateral sub-systems

21
20 KEYS TO BETTER SIMULATING
II. BASIC ELEMENTS OF A SIMULATION MODEL

• (i) System to be analyzed

TRADE BARRIES
TRADE FLOWS
TRADE SUPPORT
PRODUCTION STRUCTURES
POLITICAL ISSUES
MACRO ECONOMIC ENVIROMENT
22
20 KEYS TO BETTER SIMULATING
II. BASIC ELEMENTS OF A SIMULATION MODEL

• (ii) Analytical Mathematical Model (an example
  for international trade market equilibrium)

• Equilibrium reached making equal the inverse
  functions
• of imports and exports revenues

23
20 KEYS TO BETTER SIMULATING
II. BASIC ELEMENTS OF A SIMULATION MODEL

• (iii) Inputs, Outputs, (coherent design of
  inputs and outputs)

• Tariffs to be removed
• Span of time
• Productivities
• Fiscal pressure
• Exchange rate
• Saving rate

• Changes in trade flows
• Changes in trade prices
• Changes in Va by sector
• Changes in employment
• Changes in fiscal revenues

24
20 KEYS TO BETTER SIMULATING
II. BASIC ELEMENTS OF A SIMULATION MODEL

• (v) Interface

25
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
26
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
1.- Be sure that you a have a simulation problem

• Have to deal with a multivariate problem
• Can clearly identify inputs and outputs
• Input variables can vary in a wide range of
  values
• Output variables clearly respond to changes in
  inputs
• There is not a single scenario to be established

27
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
2.- Offer feasibility of its prospects and
involve the end users in the whole proposal

• Dont make the mistake of offer maximum proposals
  and bear in mind its realism.
• The final user usually prefers work proposals in
  which they feel an active part from the beginning

28
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
3.- Program enough time to study deeply the
system to be analysed

• The largest part of the technical decisions
  regarding the estimation, calibrating, scenario
  and interface design are conditioned by the
  comprehension of the elements and interrelations
  of the system to be analysed.

29
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
4.- Integrate in your team theoretical experts
been familiar with the system

• Research needs more heads than hands.
• Save time, which could be used to improve the
  technical issues for the simulation model
• Help even for taking technical decisions
• Establish a theoretical rigour to the whole of
  the analysis
• Give reality to the simulation mechanism

30
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
5.- Prioritise the wishes of users in all the
stages of the construction of the simulation
model and take their advices

• Nobody is interested in a technically refined
  tool that does not serve their interests
• The users only want the model to be adjusted to
  their demands, nor the other way round.

31
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
6.- Study in deep the work carried out by others

• Originality must never be an aim in itself
• Explore previous main sources of data -
  limitations of the exercise - different
  techniques available

32
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
7.- Dedicate time to analyze all of the available
data

• "Measure twice, and cut once".
• Use homogeneous data
• Choose carefully the samples
• Assess the data provided by the end user
• Be extremely scrupulous in the handling of data
• Agree with the user that data to be used responds
  faithfully to the reality perceived by him.

33
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
8.- Explore the analytical - mathematical
techniques that best adapt to the system

• Its adaptation towards capturing the specific
  phenomena observed in the specific system
• its feasibility in calculating
• its flexibility
• the quantity of theoretical hypothesis required
• its robustness towards eventual changes
• its simplicity
• the available resources

34
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
9.- Try to adapt the analytical technique to the
problem and not the other way round

• Guide the technical procedures by the suitability
  of the real characteristics observed in the
  system.
• The technique used is only valid if it works,
  independently of the objective scientific
  considerations

35
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
10.- Do not complicate the technical models if
it does not lead to clear benefits

• "If your intention is to discover the truth, do
  it with simplicity and leave the elegance for the
  tailors."

36
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
11.- Take care about the forecast power of the
model

• Try to anticipate the needs with regard to the
  prediction at the time of choosing the variables
• Evaluate with the focus on cross validation its
  vulnerability to eventual prediction errors in
  the key variables
• Avoid using single results as regards to the
  prediction use always intervals of variation or
  alternative values with probabilities of
  occurrence.

37
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
12.- If a prediction exercise is necessary try to
involve the user criteria in the interface

• "If you have to forecast, forecast often."

38
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
13.- If inferential statistics are used, check
the sensitivity of the system to changes in the
estimations

• When the analytical procedure means the use of
  statistical inference, the system can depend more
  or less critically on these estimations
• Check the sensitivity of the results of the
  system against variations in the estimated
  coeficients
• Check the robustness of the estimations with
  question such as changes in the data sample

39
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
14.- Do not underestimate the political or
qualitative aspects of which experts advise you

• The systems are not capable of being modelled
  using purely quantitative structures.
• It is not a question of choosing between a
  quantitative or qualitative approximation, rather
  than knowing how to combine both.

40
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
15.- Let simplicity guide the design of the
interface in all of its score

• "The majority of the fundamental ideas in science
  are essentially simple and, as a general rule,
  they can be expressed in language understandable
  to everyone."

41
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
16.- Call for software professionals into the
design of the interface
42
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
17.- Establish different levels of user for the
handling of the interface

• Directors, politicians, media technicians,
  technical experts, etc.
• There is no inept user, only badly designed
  systems.

43
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
18.- Instruct the users on the correct use of the
system
44
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
19.- Limit the use of the model to real scenarios

• The greatest part of the simulation systems can
  be labelled as rubbish in / rubbish out.
• Design an interface which stops, or at least
  warns the user, of possible errors in the design
  of scenarios.

45
20 KEYS TO BETTER SIMULATING
III. 20 KEYS
20.- Ensure the perfect display of the results

• Pretest Before giving the final ok to designs in
  this area, carry out several tests among your
  work colleagues to make sure that the results are
  understandable and do not oppose the proposals
  suggested to improve it.