An Experimental Test of Information

1
An Experimental Test of Information Decision
Markets
25 minute presentation!

• Robin Hanson, Takashi Ishikida and John Ledyard
• Caltech
• 2/4/2005

2
Information Markets

• Standard Information Markets seem to work.
• Small but complete set of securities
• Many informationally small and unbiased traders.
• Theory and evidence from experiments and
  applications are all positive.
• But all assume, require, use, ..
• Straight-forward behavior
• Price taking, honest revelation, etc.
• Complete set of state dependent contracts
• Common knowledge of all priors,
• Even then we see failures to fully aggregate
  information
• Incomplete Bayesian updating
• Incompletely revealing Rational Expectations
  Equilibrium

3
Decision vs Prediction

• A policy maker does not just want to know the
  probability that terrorist attacks in the US
  will increase in 2005.
• They want to know the probability that terrorist
  attacks in the US will increase in 2005 if US
  troops remain in Iraq for 2005.
• With N events (attacks, troop size, ) and S
  outcomes for each (increase from 10-20,
  decrease, ), a complete set of state dependent
  contracts requires NS - 1 contracts.
• S 2, N 8 gt 255 contracts

4
Remember PAM?
Goal Collect accurate predictive information on
political and economic stability in the middle
east.
5
Remember PAM?

• Every nationquarter
• Political stability
• Military activity
• Economic growth
• US aid
• US military activity
• all combinations
• 8 nations, 5 indices,
• 4 quarters

6
Remember PAM?

• Every nationquarter
• Political stability
• Military activity
• Economic growth
• US aid
• US military activity
• all combinations
• 8 nations, 5 indices,
• 4 quarters (N 180)

• Even if we only use up-down questions,
• completeness requires 2180 1.5(1054)
  contracts.

7
Using Conditional Contracts

• The good news
• There may more overall trading.
• Traders may know more about and be more willing
  to trade on the relatively more precise event
  terrorism up troops up as opposed to the
  less precise terrorism up.
• The bad news
• There may be thinner trading per security.
• Too many markets to pay attention to.
• Thinner trading gt bad price discovery and
  incomplete arbitrage gt prices do not aggregate
  information.

8
Decision Markets

• Markets for Decision Analysis will be thin.
• Large and possibly incomplete set of securities
• Few informationally large and biased traders
• Theory is unlikely to be a good predictor of
  behavior.
• Current applications and experiments may not be
  applicable to the thinner situation.
• How can we know what will actually work?

9
Experimental Test Beds

• Create an environment thatcaptures as much of
  the problem as possible (the econ wind tunnel)
• Three traders, three events with 2 outcome each
  (8 states)
• Common prior with asymmetric information
• 10 draws from one urn of 6 equally likely gt
  (1,0,1), (1,1,0),..
• Each trader sees only two entries of each draw
  (1,0,x), (1,1,x),
• Run different mechanisms and market designs
• Measure performance
• How close are final prices to the fully informed
  posterior?

10
Theory Benchmarks - 3 events
priors
uniform
posteriors
11
Individual Scoring Rule
priors
uniform
posteriors
12
Standard Markets
priors
uniform
posteriors
13
Design Matters

• Asking is not enough.
• Let there be markets is not enough.
• Conjecture An IM will work better in thin
  situations, if we use (to thicken trading)
• Conditional contracts and
• a Combinatoric (package bid) Call Market
• Includes no arbitrage pricing but is
  intermittent
• Does not directly address monopolistic agents

14
Combinatoric Call Market
uniform
posteriors
priors
15
Design Matters

• We are not yet at complete aggregation.
• Conjecture An IM will work even better in thin
  situations, if we use (to thicken trading)
• Conditional contracts and
• A Combinatoric Sequentially Shared (Market)
  Scoring Rule
• Is continuous and directly addresses report
  manipulation
• But it involves a subsidy to traders.

16
Shared Scoring Rule - w/CC
posteriors
uniform
priors
17
Tentative Conclusions

• Standard markets and surveys do not work will in
  thin situations.
• Using conditional contracts and assuming some
  self - selection, either combinatoric call
  markets or combinatoric sequentially shared
  scoring rules significantly improve performance
  over standard markets.

18
Open Questions

• There are many others we did not test
• Pari-mutuel mechanisms
• Economides, Lange, and Longitude (some
  combinatorics)
• Pennock - Dynamic Pari-mutel Market
• Plott - Auction then Pari-mutuel

19
Open Questions

• There are many others we did not test
• Pari-mutuel mechanisms
• Economides, Lange, and Longitude (some
  combinatorics)
• Pennock - Dynamic Pari-mutel Market
• Plott - Auction then Pari-mutuel
• Others
• HP -
• .

20
Some Open Questions

• There are many other mechanisms we did not test.
• Pari-mutuel mechanisms
• Economides, Lange, and Longitude (some
  combinatorics)
• Pennock - Dynamic Pari-mutel Market
• Plott - Auction then Pari-mutuel
• Others
• HP -
• .
• There are many other environments we did not test
  in.
• Information monopolist
• External incentives to manipulate internally
• And for PAM -- do these results survive in an
  ultra-thin world?

21
A Force 12 Storm

• Create an environment that really stress-tests
  the mechanisms
• Six traders, 8 events w/ two outcomes each (256
  states)
• Common prior with asymmetric information
• 10 draws from one urn of 8! equally likely
• (1,0,1,0,1,1,0,0), (1,1,0,0,0,0,0,0),..
• Each trader sees only 4 different entries
• (1,0,x,0,x,1,x,x), (1,0,x,0,x,1,x,x),

22
posteriors
priors
uniform
23
Summary of Testing

• Thin 3 traders, 3 events
• 7 independent prices from 3 people in 12 minutes
• Markets lt Individual Scoring Rule lt Call lt SSSR
• SSSR Call given that the group beats the
  prior
• With selection, SSSR and Call Market do best.
• Ultra-Thin 6 traders, 8 events
• 255 independent prices from 6 people in 12 min.
• Markets Individual Scoring Rule Call lt SSSR
• SSSR beats the priors at the top (60)
• Nothing else even beats the priors
• SSSR is only one with any aggregation

24
Final Thoughts

• Information Markets are possible and desirable.
• Can improve our ability to identify and deal with
  uncertainty.
• Many policy applications will be in thin
  situations.
• Traditional market designs do not work in thin
  situations.
• Information monopolies, adverse decisions,
  partial updating
• The SSSR (w/conditionals) definitely sharpens the
  signal/noise ratio in thin and ultra-thin markets
  over traditional markets.
• Can we do better? Undoubtedly.

25
(No Transcript)