On the Autonomy of Experiments in Economics

1
On the Autonomy of Experiments in Economics

• Martin Jones
• School of Social Sciences,
• University of Dundee

2
Autonomous Research Programmes

• Will argue that experiments can be used to
  independently investigate phenomena.
• There is no requirement to test a particular
  theory.
• There is no requirement to externally validate
  using field studies.

3
Preliminaries

• Aim of experimentation is to discover causal
  relationships.
• Use of controls to isolate causes.
• Causation not explicit in economics but implicit
  in e.g. consumer theory, rational choice theory
  etc.
• Causes in experiments often linked to subject
  attitudes.

4
Taxonomy of Experiments

• Theory testing experiments e.g. tests of EU,
  Cumulative Prospect Theory etc.
• Domesticated System experiments (Harré 2003) e.g.
  market based tests of Winners Curse
• Causal Control Experiments (CCEs) e.g. event-
  splitting effects (Starmer 2000)
• Note latter two not devoid of theoretical
  concerns- simply not theory testing.

5
Instruments

• Instruments in an experiment enhance and
  constrain knowledge gained (c.f. Hacking 1983).
• Not necessarily physical- can be experimental
  procedures (e.g. binary lottery mechanism)
• Problem with instruments- an experimental effect
  is produced jointly by behaviour and instruments.
  An effect may be an artefact of the instruments.

6
Manipulation

• Reality of phenomenon ensured by manipulation.
• Biology- microscopes (Hacking 1983) Economics-
  equitable division in Ultimatum games (Camerer
  2003).
• Manipulation involving different instruments and
  tests of instruments ensures independence of
  phenomenon.
• With multiple treatments and experiments one can
  ensure the reality and independence of phenomena-
  especially CCEs.

7
External Validity

• Guala (2005) Experiments act as a screening
  device for theories.
• External validity assured by Field Studies.
• This obviously works for Domesticated System
  type experiments. May work for theory- testing
  experiments

8
External Validity

• Gualas framework does not work for CCEs
• CCEs disrupt natural systems
• CCEs may produce effects that do not exist in the
  external world.
• Therefore CCEs cannot be made externally valid
  through Field Studies.

9
How can CCEs be Externally Valid?

• CCEs are counterfactual but have many elements
  in common with external world (human subjects,
  money etc.).
• When causal mechanisms correspond a CCE is
  externally valid.
• If additional causes postulated in external world
  then (in principle) can be tested in laboratory.
• Example- money payments higher in external world?
  (or arbitrage etc.)

10
How can CCEs be Externally Valid?

• Similarly if too many causes in lab.
• Aim of CCEs not to reproduce phenomena but to
  find explanations. Does not have to correspond
  completely- otherwise counterfactual experiments
  not possible.
• Source of knowledge for additional causes could
  come from previous experiments, theoretical
  knowledge, previous field studies or a
  combination of these.

11
Conclusions

• CCEs are useful as autonomous research tools.
  Independent of both theory testing and necessity
  for field studies.
• CCEs useful as part of a programme of experiments
  which establish the reality, independence and
  external validity of the phenomenon being
  studied.
• Repetition, Pilot studies, practice runs
  essential- eliminate artefacts from experiment.

12
Conclusions

• Is this restricted to CCEs?
• No- same arguments apply to other experiments as
  well. All investigate causal connections.
• Arguments for theory testing and field studies
  are additional arguments for specific types of
  experiment.