Prediction Markets

1
Prediction Markets
Leighton Vaughan Williams Professor of Economics
and Finance Nottingham Business School Nottingham
Trent University Leighton.Vaughan-Williams_at_ntu.ac.
uk
2
How Well Do Markets Aggregate Information?

• How wise is the crowd?

3
Galtons Ox

• In 1906, Sir Francis Galton (1822-1911), the
  English explorer, anthropologist and scientist,
  visited the West of England Fat Stock and Poultry
  Exhibition, where he came across a competition in
  which visitors could, for sixpence, guess the
  weight of an ox.
• Those who guessed closest would receive prizes.
• 800 people entered.

4
Galtons crowd

• Many non-experts competed, like those clerks
  and others who have no knowledge of horses, but
  who bet on races, guided by newspapers, friends,
  and their own (Brief paper by Galton in
  Nature, March 1907).
• Reference
• F. Galton, Vox Populi, Nature, 75,
• March 7, 1907.

5
Galtons findings

• Galton added the contestants estimates and
  calculated the average of the estimates.
• Using the mean, the crowd had guessed that the ox
  (slaughtered and dressed) would weigh 1,197
  pounds. In fact, the ox weighed 1,198 pounds.
• The median estimate was 1,207 pounds, not as
  close but within 1 of the correct weight.

6
Treynors Jelly Beans Experiment

• Jack Treynor, in a classic experiment, asked his
  class of 56 students to guess the number of jelly
  beans in a jar. The mean guess was 871.
• The actual number was 850. Only one student
  guessed closer.
• Reference Jack Treynor (1987), Market
  Efficiency and the Bean Jar Experiment,
  Financial Analysts Journal, 43, 50-53.
• See also Kate Gordons seminal study of 200
  students estimating the weights of items. The
  group (average) result was 94.5 correct only 5
  students were better than this.
• Kate H. Gordon (1921), Group Judgements in the
  Field of Lifted Weights, Psychological Review,
  28 (6), November, 398-424.

7
Webinar on Forecasting Excellence and Prediction
Markets, Sept. 15, 2007.

• Joe Miles, a mathematician employed at
  eyepharma (a company offering services to the
  pharmaceutical industry) gave a presentation,
  with the following key points.
• 1. He relayed the results of a MMs in a jar
  experiment he had conducted with a large group of
  conference delegates at a pharmaceutical
  forecasting conference earlier that year. The
  estimates ranged from 381 to over 40,000! The
  median estimate was 1,789. The actual number was
  1,747, just 2.4 off. The middle estimate was
  closer than any individual estimate.
• 2. He relayed the results of an experiment
  conducted by eyetravel, a sister company, at a
  hotel industry conference. Delegates were asked
  to estimate the average price of a hotel room in
  Amsterdam that day. Estimates ranged by a factor
  of three, but the average estimate was just 0.5
  off (Mean estimate 117.8 Euro Actual price
  118.4 Euro.

8
What destroyed the space shuttle Challenger?

• On January 28, 1986, the space shuttle Challenger
  lifted off from its launch pad at Cape Canaveral.
  Seventy-four seconds later, it blew up. Within
  minutes, investors started dumping the stocks of
  the four major contractors who had participated
  in the Challenger launch Rockwell International,
  which built the shuttle and its main engines
  Lockheed, which managed ground support Martin
  Marietta, which manufactured the ship's external
  fuel tank and Morton Thiokol, which built the
  solid-fuel booster rocket. Within minutes,
  trading in Thiokol was suspended and by the end
  of the day, Thiokol's stock was down nearly 12
  percent. By contrast, the stocks of the three
  other firms each fell a little but soon started
  to creep back up, and by the end of the day had
  fallen only around 3 percent. The market was
  right. Six months later and after an extensive
  investigation, Thiokol was held liable for the
  accident. The other companies were exonerated

9
How do you find a missing submarine?

• On the afternoon of May 27, 1968, the submarine
  USS Scorpion was declared missing with all 99 men
  aboard. It was known that she must be lost at
  some point below the surface of the Atlantic
  Ocean within a circle 20 miles wide. This
  information was of some help, of course, but not
  enough to determine even five months later where
  she could actually be found.
• The Navy had all but given up hope of finding the
  submarine when John Craven, who was their top
  deep-water scientist, came up with a plan which
  pre-dated the explosion of interest in prediction
  markets by decades. He simply turned to a group
  of submarine and salvage experts and asked them
  to bet on the probabilities of what could have
  happened. Taking an average of their responses,
  he was able to identify the location of the
  missing vessel to within a furlong (220 yards) of
  its actual location. The sub was found!

10

• What are Prediction Markets?

11
Betting on the outcome

• Betting markets aggregate all available
  information to produce best estimate, not least
  because those who know, and are best able to
  process the information, bet the most. Based on
  the Efficient Markets Hypothesis, the idea that
  markets accurately incorporate all relevant
  information.

12
Prediction markets v. Betting markets

• The essential difference between prediction and
  betting markets is not an issue of structure.
• Rather, prediction markets, as usually termed,
  are distinct from betting markets in the purpose
  to which they are put.
• For example, when betting markets are used
  explicitly to forecast the outcome of any event,
  whether it is the World Cup or a rowing regatta,
  they are essentially acting as prediction
  markets.
• Even so, the term prediction markets often
  implies that the markets are being used to
  produce information externalities that can inform
  business and policy decisions.

13
The Hayek Question

• How does one effectively aggregate disparate
  pieces of information that are spread among many
  different individuals, information that in its
  totality is needed to solve a problem?
• Hayeks answer was that market prices are the
  means by which those disparate pieces of
  information are aggregated.
• The mere fact that there is one price for any
  commodity ... brings about the solution which ...
  might have been arrived at by one single mind
  possessing all the information which is, in fact,
  dispersed among all the people involved in the
  process.
• Source F.A. Hayek, The Use of Knowledge in
  Society, American Economic Review, 35, 4, Sept.
  1945 520.

14
(No Transcript)
15
Speed of the market in processing new information

• Obama price spiked one day in August, 2008,
  despite the only obvious news being a relatively
  poor opinion poll.
• Why?

16
Warp Speed Market Saddam capture or neutralize

• Date 13 December, 2003
• Market moves from about 20 to 100.
• Next day News of Saddam capture announced by US.

17
(No Transcript)
18
Predicting the outcome of rowing regattas

• Jed Christiansen (2007) reports on markets set up
  to predict the outcome of rowing regattas in the
  UK. Despite the small number of participants, and
  the absence of any incentives other than the
  challenge of getting it right, the predictions
  of the rowing events were highly accurate.
• Christiansen puts the success of the experiment
  down to the effects of community and uniqueness,
  which encouraged motivated participation.
• Source Christiansen, J.J. (2007), Prediction
  markets Practical experiments in small markets
  and behaviours observed, Journal of Prediction
  Markets, 1, 17-41.

19
Polls or markets?

• Predicting the winner of an election!

20
Early prediction markets

• The earliest data we have from prediction markets
  are those from organized markets for betting on
  the US Presidential election between 1868 and
  1940.
• Although there are reports that these markets
  date back to the election of George Washington,
  and even before, the market in 1868 seems to be
  the first we would call a prediction market in
  that its data was used to inform the public about
  the likelihood of a particular candidate winning
  and may have been used by financial asset
  traders.
• As an example, the New York Times reported that
  between 500,000 and 1 million was wagered on
  the Curb Exchange (the fore-runner to the AMEX)
  in one day on the 1916 election and that oil
  stocks were almost forgotten. The total amount
  wagered in these markets in 1916 was 165 million
  (at 2002 prices).
• In this period between 1868 and 1940, the market
  failed to predict the winner on just one
  occasion.

21
AN EARLY BRITISH PREDICTION MARKET

• Brecon and Radnor By-Election, 1985.
• Mori v. Ladbrokes

22
ELECTION EVE

• MORI Labour to win by 18.
• LADBROKES
• Liberal candidate 4 to 7
• Labour candidate 5 to 4

23
WINNERS

• The Liberal candidate.
• Those who ignored MORI and backed the market
  favourite.

24
Bush v. Gore, 2000IG Index v. Rasmussen
25
Outcome forecasts

• IG Index
• 265-275 Bush
• 265-275 Gore
• Rasmussen Bush by 9

26
Opinion Polls v. markets

• Opinion polls, like all market research, provide
  a valuable source of information, but they are
  ONLY ONE source of information.
• Other information includes
• 1. Local canvass returns
• 2. On-the-ground inside information
• 3. Forecasting models
• 4. Opinions of professional pundits (experts)
• 5. Focus groups
• Betting markets aggregate all the available
  information

27
Producing an optimal forecast

• Because those who know the most, and are best
  able to process that information, tend to bet the
  most, this drives the market to produce an
  optimal forecast at any point in time.
• Moreover, unlike polls, which are snapshots of
  opinion, betting markets are all about
  forecasting the eventual outcome.
• Since the advent of zero-tax low-margin betting
  exchanges, the accuracy of these markets have
  improved yet further.

28
US Presidential Election 2004

• INTRADE state-by-state predictions 50 out of 50.
  

29
British General Election, 2005

• Predicted Labour majority to within a handful of
  seats.

30
US Senate 2006

• Intrade All correct

31
US Presidential Election 2008

• Prediction markets
• INTRADE state-by-state predictions 49 out of 50
  (called Missouri wrong).
• BETFAIR state-by-state predictions 49 out of 50
  (called Indiana wrong).
• Statistical Modelling Using Weighted Polling Data
• FIVETHIRTYEIGHT predictions 48 out of 50 (called
  Missouri and North Carolina wrong).

32
Indiana

• Polls closed at 11.30 pm (UK time) in Indiana, a
  key state which McCain almost certainly needs to
  win to secure the Presidency.
• McCain favourite to win Indiana on Betfair.
• 11.45 pm Obama becomes favourite to win Indiana,
  attracting significant sums to win from traders.
• By this time CNN was calling just 1 of precincts
  in Indiana.
• So what caused the shift to Obama on Betfair?
• In retrospect, it seems that professional traders
  had latched on to the detail in the few published
  results.
• Importantly, this shows the power of prediction
  markets in assimilating and processing new
  information very rapidly.

33
Early Precinct Results

• Stueben Kerry 34, Obama 42
• De Kalb Kerry 31, Obama 38
• Knox Kerry 36, Obama 54
• Marshall Kerry 31, Obama 50
• Only the most well-informed had accessed these
  results by 11.45, and knew what they meant, i.e.
  a big swing from Republican to Democrat since
  2004, but Betfair traders were among them.
  Minutes later, the swing was confirmed in Vigo
  County. By 12.20, Obama was shorter than 1 to 2
  on Betfair.

34
The election was called at 4am, but Betfair
watchers knew before midnight!

• At 4am, California was declared, giving Obama the
  final few electoral votes required to win the
  Presidency.
• At 2.30am Ohio was called by most news networks.
• Before midnight, the knowledge that Indiana was
  going to Obama, or at least that McCain would at
  best claim a small win there, was enough to
  indicate to Betfair watchers that the election
  was all but over. At 12.23 am, McCain was
  available at 25 to 1.
• Meanwhile, Fox News declared that Indiana was
  over-polling for Obama because it shares a border
  with his home state of Illinois!
• It was well past 3am when Fox News called the
  election for Obama.
• Betfair 1, Fox News 0.

35
2010 UK General Election

• It was the debates that lost it for the
  Conservatives!!!
• Before the first debate, the markets all
  predicted a Conservative overall majority.
• After the first debate, none of the markets ever
  predicted anything other than a hung parliament!
• While the polls swung all over the place, the
  markets barely flickered after that first debate
  in predicting a hung parliament with the
  Conservatives the largest party with somewhere
  between 300 and 320 seats.

36
Could prediction markets have prevented 9/11?

• The 9/11 Commission Report stated the problem
  like this "The biggest impediment to all-source
  analysis - to a greater likelihood of connecting
  the dots - is the human or systemic resistance to
  sharing information.
• "What was missing in the intelligence community
  ... was any real means of aggregating not just
  information but also judgements. In other words,
  there was no mechanism to tap into the collective
  wisdom of National Security nerds, CIA spooks,
  and FBI agents. There was decentralization but
  not aggregation ... (James Surowiecki, The
  Wisdom of Crowds)
• Can the market can help achieve this? Some people
  within the US Department of Defence had been
  working on just such an idea for several months
  when al Qaeda struck. Indeed, in May 2001 the
  Defense Advanced Research Projects Agency (DARPA)
  had issued a call for proposals under the heading
  of 'Electronics Market-Based Decision Support'
  (later 'Future Markets Applied to Prediction'
  (FutureMAP).

37
FutureMAP (cont.)

• The remit prescribed for FutureMAP was to create
  market-based techniques for avoiding surprise and
  predicting future events. It was not long,
  however, before the US media and key members of
  the Congress began to train their guns on the
  idea of such a market. After all, it isn't
  difficult to portray the market as no more than a
  forum for eager traders to profit from death and
  destruction. The populist arguments won the day
  and DARPA was forced to cancel the project.
• While most of the arguments against the market
  were emotional rather than intellectual, there
  was nevertheless some genuine intellectual
  concern as to how effective it would be likely to
  be.

38
Was Stiglitz right?

• In particular, Prof. Joseph Stiglitz argued in an
  article published in the Los Angeles Times on 31
  July 2003 ('Terrorism There's No Futures in
  It'), that the market would be too "thin" (i.e.
  there would be too little money traded in the
  market) for it to be a useful tool for predicting
  events meaningfully. His argument was based on
  work he had previously published showing that
  markets can never be perfectly efficient when
  information is costly to obtain. The cost of
  obtaining and processing this information is, by
  implication, likely to act as a significant
  disincentive particularly in the context of a
  thin market (and hence low rewards).
• But is it obviously the case that a properly
  constructed market, populated by suitably
  motivated (and perhaps screened) players can be
  viewed in this way?

39
Can prediction markets be used to study climate
change?

• 1) A properly constructed market might encourage
  climate change analysts to become more specific
  in their forecasts, and would encourage the
  development of new modelling techniques. 2) The
  markets could help to provide an assessment of
  the tangible impact upon climate change of
  various policies under consideration by
  governmental and international bodies.3) The
  market could potentially help to establish a
  price for carbon. 4) The markets could help to
  price in new information more quickly. 5) The
  market would help businesses and governments to
  hedge against both the dangers of climate change,
  and against costs of addressing it.
• There could be a series of contracts and perhaps
  options on, for example, temperature, CO2
  emissions, precipitation, and tropical storms
  which expired at various intervals.

40
Can prediction markets help us make flight plans?

• Volcanic ash cloud
• BA Strike
• Can we construct a prediction market which can
  amass the collective wisdom of the informed crowd
  to help us plan our future schedule?

41
Prediction Markets in Public Authorities

• A notable feature of public policy in the UK over
  the past decade has been the imposition by
  central governments of performance targets as a
  means of evaluating the performance of local
  public organisations.
• Targets cover a huge range of activities ranging
  from those specific to health or education to
  those relating to more general local authority
  performance.
• Targets are used as a means of evaluating
  performance, improving standards and allocating
  resources. The significance of achieving or not
  achieving particular targets can be very high for
  local politicians as well as senior managers in
  local authorities and health organisations in
  terms of both resources, public image. At the
  same time, it is extremely difficult for
  politicians and central managers to be aware of,
  let alone to process, the complex streams of
  information that are available

42
PMs in public authorities (cont.)

• Within this context, prediction markets offer a
  potentially valuable tool that may be used to
  synthesize the specific knowledge of those
  directly involved with implementing policy at a
  lower level. The specific nature of targets
  relating to, e.g., waiting list times,
  educational outcomes, are both specific and
  quantifiable and, hence, ideal candidates for
  operating a trading market. Taking the example
  of health care targets, the numbers of people
  involved from nurses, doctors to administrators
  further suggest that the operation of markets in
  this context is feasible.
• The value of the information provided by
  prediction markets will come primarily from the
  advance warning that politicians and managers
  will be given of weak performance in particular
  areas. This has the potential to improve
  resource allocation to make it more likely that
  key targets are met.

43
PMs for Public Policy Decisions

• Example Should policy A or policy B be
  undertaken to reduce waiting lists?
• Current waiting list for an appointment at the
  eye clinic 30 days.
• Contract pays 1 for the length of the waiting
  list in days. And currently trades at 30 pounds.
• Participants in the market can BUY the contract
  at 30 if they think the waiting list will
  increase and SELL if they think it will decrease.
• E.g. If they SELL at 30 and the waiting list
  decreases to 25 days, they will 5 (30-25). But
  if the waiting list increases to 35 days, they
  lose 5.
• By comparing the Waiting list with policy A
  contract with the Waiting list with policy B
  contract, the policy maker has gained information
  on what the market thinks about the relative
  impacts of introducing policy A and policy B on
  the length of the waiting list.
• If a policy is not implemented, the contract is
  declared void.

44
Using the power of prediction markets for disease
surveillance

• http//iehm.uiowa.edu/iehm/index.html
• Reporting speed is one of the most import
  aspects of any surveillance program for seasonal
  influenza even two weeks advance notice can have
  dramatic results on the effectiveness of
  vaccinations.
• Although there are many existing strategies for
  gathering opinions about the future trends of
  infectious diseases, the resulting data are often
  difficult to interpret using standard
  epidemiological methods. Prediction markets, on
  the other hand, are well known for their ability
  to quickly collect and summarize information.
• The Iowa Electronic Health Markets is a research
  project at the University of Iowa exploring the
  use of prediction markets as a tool for disease
  surveillance. By combining the strengths of
  prediction markets with the knowledge of our
  trading community from around the world, our hope
  is that these markets will report future
  infectious disease activity quickly enough to be
  clinically useful.

45
Limitations of crowd wisdom

• Can the crowd predict the lottery numbers?
• If not, why not?
• Because lottery numbers are drawn randomly, no
  model or individual or crowd or other means of
  aggregating information can predict them because
  random numbers are by definition unpredictable.
• If the lottery numbers were, for whatever reason,
  not drawn randomly, however, we have a different
  issue.

46
Is Manipulation Bad for Prediction Markets?

• Robin Hanson and Ryan Oprea, of George Mason
  University and the University of California,
  Santa Cruz respectively, co-authored a paper
  title, 'A Manipulator Can Aid Prediction Market
  Accuracy. A perspective on its basic message is
  offered by Alex Tabarrok at Marginal Revolution.
  Tabarrok was considering the impact of the clear
  attempt by at least one determined trader to
  manipulate one of the US election betting markets
  in favour of Senator John McCain. In particular,
  the John McCain contract was bought in the
  markets systematically every morning by one
  US-based trader for sizeable sums. In
  consequence, it was possible to arbitrage between
  McCain (on Intrade) and Obama (on Betfair) for a
  few weeks in the run-up to Election 2008.
• How much of a danger, Tabarrock asks, does this
  sort of activity pose for the whole concept of
  prediction markets? Not much, he argues, instead
  offering support for Hanson and Oprea's finding
  that manipulation can actually improve prediction
  markets, for the simple reason that manipulation
  offers informed investors a free lunch.

47
Manipulation (cont.)

• "In a stock market", Tabarrok writes, "... when
  you buy (thinking the price will rise) someone
  else is selling (presumably thinking the price
  will fall) so if you do not have inside
  information you should not expect an above normal
  profit from your trade. But a manipulator sells
  and buys based on reasons other than expectations
  and so offers other investors a greater than
  normal return. The more manipulation, therefore,
  the greater the expected profit from betting
  according to rational expectations.
• For this reason, investors should soon move to
  take advantage of any price discrepancies thus
  created within and between markets, as well as to
  take advantage of any perceived mispricing
  relative to fundamentals. Thus the expected value
  of the trading is a loss for the manipulator and
  a profit for the investors who exploit the
  mispricing. Moreover, the incentive the activity
  of the manipulator gives for others to become
  informed, and to trade on the basis of this
  information, is valuable in itself in improving
  the efficiency of the market.

48
Worth manipulating?

• Tabarrok offers the additional observation that,
  considerations of predictive accuracy aside,
  there is one even more important lesson to be
  learned from the activities of the manipulators
  "...that prediction markets have truly arrived
  when people think they are worth manipulating".
• But have they? What does the corporate sector
  think?

49
HOW CAN COMPANIES USE PREDICTION MARKETS?

• To take an example, a manufacturer of aero
  engines will seek good forecasts of future orders
  from plane manufacturers, which in turn will be
  contingent upon orders from airlines. Forecasts
  of future airline orders will be greatly assisted
  by the collation of a range of information from
  those involved in each of these sectors.
• It is important that the questions posed are in a
  form which is unambiguous and which can
  ultimately be quantified. This requires an
  assessment of who should be involved in
  responding, and ensuring that each of these
  contributors has an equivalent understanding of
  the meaning of what is being asked, and that
  these answers can usefully be pooled. The set-up
  will vary depending on the diversity of
  contributors, both geographically and
  functionally. There is also the issue of
  incentives and the number of markets to run, as
  well as the length of these markets and how often
  new markets should be introduced.
• But in principle markets should be able to help
  aggregate information.

50
But whats the evidence? Can prediction markets
actually help internal company forecasting?

• There is in fact plenty of published research
  showing how internal prediction markets have
  helped improve the ability of commercial
  organisations to structure and implement internal
  prediction markets to assist in forecasting.
• to predict key business variables
• e.g. when will a product launch, what will be the
  unit sales?
• broader-based prediction markets are a useful
  mechanism for predicting market-wide outcomes,
  e.g. box office receipts for a new film, success
  of a new video game, property prices.

51
Commercial examples

• Eli Lilly ran an experiment in which managers
  traded through an internal market the future
  monthly sales figures for three drugs. The market
  brought together all the information, from
  toxicology reports to clinical results, and
  produced more accurate forecasts than the
  official forecasts.
• Google have set up a market, in which any Google
  staff member could bet on the chances of an event
  coming true. The markets were used to forecast
  such things as product launch dates and new
  office openings. The results - based on the
  aggregated bets of thousands of Google staff
  members - were strong predictors of the actual
  outcomes.

52
How do these markets operate?

• Essentially, participants in the market exchange
  offers and counter-offers until they agree on a
  contract price.
• Trades are executed when two prices match.
• In describing how experimental internal-prediction
  market run by Eli Lilly (the pharmaceutical
  company) work, VP for Lilly Research Laboratories
  Alpheus Beingham noted
• When we start trading in the drug and I try
  buying your stock cheaper and cheaper, it forces
  us to a way of agreeing that never really occurs
  in any other kind of conversation.

53
Corporate Applications of Prediction Markets
Special Issue of the JPM

• The Journal of Prediction Markets (2009)
• www.thejpm.com
• Guest editor Prof. Koleman Strumpf, member of
  the Editorial Board of the Journal of Prediction
  Markets, and Koch Professor of Economics at the
  University of Kansas School of Business.
• Based on presentations at the Conference on
  Corporate Applications of Prediction/Information
  Markets, held at Kansas Citys Kauffman
  Foundation on 1 November, 2007.

54
What are Corporate Prediction Markets? Editorial
Introduction

• Prediction Markets use the knowledge of a pool
  of individuals to help forecast questions of
  importance to companies, such as whether a sales
  target will be reached or whether a project will
  be completed in a timely manner. A more recent
  development is the use of such markets to
  generate and evaluate new ideas, such as new
  products or cost saving procedures.
• Since first being applied within the corporate
  sector over a decade ago, over a hundred
  companies have run internal markets. These range
  in size from some of the largest in the world to
  those with only a handful of employees, and cover
  a broad range of sectors, from those whose
  products are abstract ideas to others which
  manufacture very low-tech products.

55
Why Have Such a Broad Range of Firms Become
Interested in Prediction Markets?

• The answer lies in a common problem facing
  firms, namely the isolation of executives from
  the views and insights of the companys
  workforce.
• Such seclusion is no accident but instead
  reflects one of the reasons companies are
  structured as they are in the first place, i.e.
  to avoid information overload for busy
  executives.
• To reach this goal firms developed a hierarchy
  structure, and assigned to middle management the
  task of deciding how much and what information
  was transmitted from employees to higher-level
  decision-makers. But the system has its costs,
  as potentially useful information may be filtered
  out if it reflects poorly on those who control
  the information flow. At the same time,
  lower-level employees have little incentive to
  make reports which conflict with their managers.
  The net result is that executives may only
  receive one-sided information, and flawed
  decisions may result.

56
This is where Prediction Markets Come In!

• Suppose the CEO must decide whether to continue
  funding a project, but is concerned that he has
  been receiving overly optimistic reports on its
  prospects from managers who will benefit from the
  project continuing.
• A market on the projects prospects would allow
  front-line employees to convey more realistic
  information, and they could do so without fear of
  reprisal if the trading is anonymous.
• Prediction markets may also function better than
  other approaches currently in use. For example,
  group meetings are less likely to have frank
  discussions while suggestions boxes do not scale
  well - prediction markets tend to perform better
  when there are more participants.
• And while most workers may dread the thought of
  meetings, markets are often considered a fun
  activity and often do not require much in the way
  of incentives to generate active employee
  involvement.

57
A Case Study of GE (JPM, 2009), by Brian Spears
(Risk Manager, GE-Hitachi Nuclear Energy) et al.

• 1. Internal markets used to aggregate opinions
  are consistent with opinions collected via web
  surveys (Chan et al, 2002). Markets may in fact
  improve upon traditional survey methods by
  encouraging greater honesty from the
  participants, providing participants with
  valuable feedback from other participants, and
  offering participants the joy of competitive
  play.
• 2. GEs markets are designed to help answer
  business questions such as What new technology
  ideas should we be investing in? What new
  products should we be developing?
• 3. Market participants can submit their own ideas
  for entry into the market, and they can buy and
  sell shares of any idea in the market.

58
GE Case Study (cont.)

• GEs interest in idea markets stem from our
  belief that innovative new product and service
  ideas can come from anywhere within an
  organization.
• Similar to most companies, GE uses a variety of
  methods to generate and down-select new ideas.
  Traditional means include suggestion boxes and
  brainstorming sessions.
• However, suggestion boxes often go unused because
  contributors receive little or no feedback about
  their idea or visibility into others ideas.
• Brainstorming sessions are often infeasible for
  soliciting ideas from large, globally distributed
  teams with potentially thousands of contributors.
  
• Hence the idea of a prediction market, which they
  call an Imagination Market was born.

59
Conclusions of Case Study

• Overall, the GE Energy business was extremely
  pleased with the results of the Imagination
  Market.
• Funding was immediately provided to kick-start
  the two ideas tied for the top, and the business
  has decided to file patents for several others.
• GE Energy plans to continue use of markets in the
  future.
• The volume and quality of ideas compared
  favourably to brainstorming sessions, on-line
  suggestion boxes, and on-line discussion forums.
  
• NB The Spears paper provides a wealth of
  information and detail about the GE markets,
  including a very detailed description of the
  mechanics of the markets, such as the incentives
  given to traders and to creators of new ideas.

60
Improving Forecasting Accuracy in Corporate
Prediction Markets A Case Study in the Austrian
Mobile Communication Industry JPM, 3, 2009, by
Martin Waitz and Andreas Mild

• ABSTRACT
• Corporate prediction markets forecast business
  issues like market shares, sales volumes or the
  success rates of new product developments.
• The improvement of its accuracy is a major topic
  in prediction market research ...
• We propose a method that aggregates the data
  provided by such a prediction market in a
  different way by only accounting for the most
  knowledgeable market participants.
• We demonstrate its predictive ability with a real
  world experiment.

61
Why companies are well positioned to utilize the
information generated from PMs

• 1. Company divisions often serve as standalone
  silos, and markets can be a means of integrating
  the pockets of information contained in each.
• 2. Executives may be interested not just in
  market aggregates, such as prices, but also the
  trades of particular groups of employees. For
  example, one could examine whether members of
  certain divisions are less prone to making biased
  forecasts.
• 3. Companies need real-time information about the
  many uncertain events surrounding their
  decision-makers.
• 4. Firms can internalize the informational
  benefits of the market. A company can profit from
  the information generated from prices, since the
  market can be kept private and outside of the
  purview of competitors.
• The last point is particularly important. Since
  the benefits of the markets largely accrue to the
  company, we might expect many prediction market
  innovations to first arise in a corporate
  setting.

62
Key challenges facing PMs

• 1. Operators must overcome investor reluctance to
  a project with upfront costs and possibly delayed
  benefits.
• 2. There are impacts on employees, both
  detrimental (markets may distract staff away from
  their main responsibilities) and beneficial
  (there is often a gain in morale, as workers feel
  empowered because their market-mediated
  suggestions are impacting corporate decisions.
• 3. The markets may overwhelm executives with too
  much information.
• 4. Market organizers must allay concerns of
  middle management and those whose current role in
  the company is threatened by the market.
• 5. There may be systematic biases in some markets.

63
Optimism bias

• Optimism bias is the systematic tendency for
  people to be over-optimistic about the outcome of
  planned actions. This includes over-estimating
  the likelihood of positive events and
  under-estimating the likelihood of negative
  events".
• David Armor and Shelley Taylor highlight a number
  of examples of what they consider to be optimism
  bias in an interesting paper called 'When
  Predictions Fail The Dilemma of Unrealistic
  Optimism', published in 2002.
• Examples include students' estimates of the
  likely starting salary of their first job in the
  graduate market and newlyweds' thoughts on how
  long their marriage will last. It is interesting,
  therefore, that evidence of the existence of this
  very same bias has been identified in 'internal'
  company prediction markets, notably in a 2008
  paper co-authored by Bo Cowgill, of Google,
  Justin Wolfers of the Wharton School and Eric
  Zitzewitz, based at Dartmouth College.

64
Optimism bias (cont.)

• Cowgill, Wolfers and Zitzewitz examine the
  results generated by what they call the Google
  corporate prediction market experiment. The
  primary goal of these markets is, as they put it,
  to generate predictions that efficiently
  aggregate many employees' information and augment
  existing forecasting methods.
• In support of previous investigations into the
  value of internal prediction markets, they were
  able to confirm that prices in the Google markets
  closely approximated event probabilities, i.e.
  that the markets were reasonably efficient. Even
  so, they were not perfect, and one notable reason
  was an apparent 'optimism bias' which, according
  to their findings, "was more pronounced for
  subjects under the control of Google employees,
  such as whether a project would be completed on
  time or whether a particular office would be
  opened."

65
Optimism bias (cont.)

• Optimism bias was also found to be more evident
  in new employees and in the immediate few days
  following a good news day for the Google stock
  price. Still, what is a cost in terms of
  unadjusted predictive efficiency may be a benefit
  in terms of motivation and entrepreneurial zeal,
  a feedback mechanism the value of which it is
  perhaps easy to under-estimate.
• In any case, if we are able to identify and
  measure the source and extent of the bias, it
  should be possible to adjust and compensate for
  this particular inefficiency in generating the
  objective forecasts.

66
The Favourite-Longshot Bias

• Let the probability of an event occurring be 20.
• Standard approach Probability 0.2
• Bookmakers approach Odds 4 to 1. This means
  you win 4 (net) from the bookmaker if your bet
  wins for every 1 staked (risked) with the
  bookmaker.
• Which yields the better expected return, a stake
  of 10 on a horse with odds of 2 to 1 or a stake
  of 10 on a horse with odds of 20 to 1?
• i.e. if Mr. A and Mr. B both start with 1,000.
  Now Mr. A places a level 10 stake on 100 horses
  quoted at 2 to 1 and Mr. B places a level 10
  stake on 100 horses quoted at 5 to 1. Who is
  likely to end up richer?

67
Odds Backing Ladbrokes Pocket Companion, Flat
Edition, 1990, pp. 242-243

• Not one out of 35 favourites sent off at 1/8 or
  shorter (as short as 1/25) lost between 1985 and
  1989. This means a return of between 4 and 12.5
  in a couple of minutes, which is an astronomical
  rate of interest... The point being made is that
  broadly speaking the shorter the odds, the better
  the return. More broadly, the group of white
  hot favourites (odds between 1 to 5 and 1 to 25)
  won 88 out of 96 races for a 6.5 profit. The
  following table looks at other odds groupings.
• Odds Wins Runs Profit
• 1/5-1/2 249 344 1.80 0.52
• 4/7-5/4 881 1780 -82.60 -4.64
• 6/4 -3/1 2187 7774 -629 -8.09
• 7/2-6/1 3464 21681 -2237 -10.32
• 8/1-20/1 2566 53741 -19823 -36.89
• 25/1 -100/1 441 43426 -29424 -67.76

68
An Election Super-Bias?

• The 2004 US Presidential state-by-state markets
  gave the equivalent of 50 successive winning
  favourites at the racetrack.
• In 2008, the Betfair favourites won 49/50 states.
• The Intrade favourites won 49/50.

69
Biases

• Do biases differ between different prediction
  market formats?
• Can we compensate for the biases to yield more
  accurate forecasts?
• Do some formats yield more volatile forecasts
  than others?

70
Some further examples of PMs

• Best Buy, the electronics retailer. Experimented
  with prediction markets on everything from demand
  for digital set-top boxes to store opening-dates.
• E.g. in Autumn 2006, the price in one of their
  PMs on whether a new store in Shanghai would open
  on time several weeks ahead dropped sharply from
  80 a share to about 45. Players made yes-no
  bets, and the virtual dollar drop reflected
  increasing doubt that the store would open on
  time. The store opened a month late.
• Jeffrey Severts, a VP who oversees PMs at Best
  Buy, quoted in NY Times (April 9, 2008 Betting
  to Improve the Odds)
• The potential is that prediction markets may be
  the thing that enables a big company to act more
  like a small, nimble company again.
• It helps on two fronts, the speed and accuracy
  of information, so that management can move
  faster to deal with problems or exploit
  opportunities.

71
Best Buy (cont.)

• Severts invited several hundred employees to
  submit an estimate for sales for a one-month
  period. To help them calibrate their estimate, he
  provided monthly data from the past twelve
  months. 192 employees responded, including those
  on the store floor. The estimates were given
  equal weight and averaged. He found that the
  employees collective wisdom had an error of only
  0.5 compared to an error of 5 by the
  traditional forecasting method.
• Severts went on to experiment with total sales
  over the 14 week holiday period. He provided last
  years sales figure from the holiday period and
  revenue growth for the first three months of the
  current fiscal year compared to previous years.
  The original 350 respondents predicted sales
  during the fourteen-week holiday period that was
  99.9 accurate. The merchants themselves who were
  traditionally responsible for forecasting were
  93 accurate.
• (Hamel, 2007, The Future of Management, Boston
  Harvard Business School Press).

72
PREDICTION MARKETS AS A MEDICALFORECASTING TOOL
DEMAND FOR HOSPITALSERVICES David Rajakovich
and Vladimir Vladimirov (JPM, 2009)

• This paper presents the outcome of a study
  conducted at the Royal Devon and Exeter Hospital
  in which a prediction market was established in
  order to forecast demand for services.
• The study was conducted over a period of one
  week, and involved 65 participants. Each was
  asked to provide an estimate for demand for
  services at the Royal Devon and Exeter Hospital.
  In each survey, each employee was asked to
  estimate the number of patients that would be
  admitted to each directorate, which meant that
  employees within each directorate were estimating
  the number of patients admitted to directorates
  other than their own.

73
Findings

• The overall results confirmed the effectiveness
  of prediction markets.
• The prediction for admittances was 1157.51 while
  the actual number of admittances was 1154, which
  is an error of only 0.3.
• Market participants were almost exactly right in
  the Medicine directorate, predicting 353.38,
  while the actual was 353.
• Specialist Surgerys prediction was almost as
  accurate with an actual number of admittances of
  106 and an estimate of 107.75.
• However, the prediction market was less
  successful in predicting demand for services for
  each department, which the paper attributes to
  the small sample size and lack of diversity of
  participants.

74
Further examples

• In 2007, a group in the purchasing unit at
  Hewlett Packard began prediction markets on the
  price of computer memory chips three and six
  months ahead.
• Bernardo A. Huberman, director of the social
  computing lab at Hewlett-Packard The prediction
  markets were up to 70 more accurate than the
  companys traditional forecasting model ... The
  more accurate predictions can be used to finesse
  purchasing, marketing and product pricing
  decisions.

75
The Classic Study

• Information Aggregation Mechanisms Concept,
  Design and Implementation for a Sales Forecasting
  Problem, by Charles Plott (CalTech) and Kay-Yut
  Chen (Hewlett-Packard Laboratories), 2002.
• Many business examples share the following
  characteristic small bits and pieces of relevant
  information exists in the opinions and intuition
  of individuals who are close to an activity. Some
  examples are supply chain management issues,
  demand forecasting, new product introduction, and
  supply uncertainties. In many instances, no
  systematic methods of collecting such information
  exist. In these cases very little is known by any
  single individual but the aggregation of the bits
  and pieces of information might be considerable.
  For instance, it is extremely difficult to
  combine subjective information such as the
  knowledge of a competitors move with objective
  information such as historical data. In a perfect
  world, with unlimited time and resources, a user
  of such information could personally interview
  everyone that might have a
• relevant insight but such luxury does not exist.

76
Chen and Plott (cont.)

• Gathering the bits and pieces by traditional
  means, such as business meetings, is highly
  inefficient because of a host of practical
  problems related to location, incentives, the
  insignificant amounts of information in any one
  place, and even the absence of a methodology for
  gathering it. Furthermore, business practices
  such a quotas and budget settings create
  incentives for individuals not to reveal their
  information. The principles of economics together
  with new technologies that exist for creating
  markets and related mechanisms suggest that in
  might be possible to develop a new approach that
  avoids many of the practical problems.

77
Some details of the experiment

• The experiments were conducted with three
  different HP divisions. Typically, around 20-30
  people signed up for the experiments. Trading was
  done through at a web server located at Caltech.
  The subjects were geographically dispersed in
  California.
• Typically, the prediction was for monthly sales
  for a month three months in the future. The
  market mechanism employed to support the markets
  was the web based markets of the Marketscape
  software, which was
• developed at the Laboratory of Economics and
  Political Science at Caltech. All the markets for
  an event were organized on a single web page for
  easy access.
• A participant could enter a buy offer, a sell
  offer or acceptance of an offer through the web
  form on the page. Orders were compared to the
  other side immediately. The best offers were
  listed on the main market web page. The whole
  book of offers was available for each market at
  the click of a button.

78
(No Transcript)
79
Results

• Market predictions based on IAM (Information
  Aggregation Mechanism) prices outperformed
  official HP forecasts.
• In events for which official forecasts were
  available the IAM predictions were closer to the
  actual outcome than the official forecast 75 of
  the time. The absolute errors of the official
  forecasts were also significantly higher than
  that of the IAM predictions.

80
Other general applications of PMs

• Examples of the applications of prediction
  markets range from traditional finance
  forecasting such as sales and costs, to product
  development support such as forecasting on-time
  project delivery or the likelihood of regulatory
  approval for new drugs, to innovative decision
  support such as evaluating the impact of
  switching advertising agencies or forecasting the
  market receptivity of new software releases
• Etc
• Etc
• Etc.

81
Its not just about forecasting!

• A prediction-market pilot at Microsoft in 2005
  was designed to forecast the probability of
  on-time release for several products.
• To managements surprise, the stock price
  representing on-time release dropped to zero,
  despite the staffs prior assurance that on-time
  release was likely.
• The ensuing conversation uncovered the true
  beliefs of the programmers, a result perhaps even
  more valuable than knowing whether the release
  would be missed.
• Source Todd Proebsting, of Microsoft Research,
  in his presentation Tee Time with Admiral
  Poindexter, delivered at the DIMACS Workshop on
  Market as Predictive Devices (Information
  Markets), February, 2005, Rutgers University.

82
Conclusion

• Prediction markets offer major unexploited
  opportunities to aggregate information in a rapid
  and efficient manner. There are significant
  examples of success where they have been tried,
  although in some cases they have been tried and
  then discarded.
• In an interview with DIRECTOR magazine, I said
  this
• The real problem where these markets have not
  worked as well as management expected is that the
  companies have simply bought the technology and
  more or less expected it to take care of itself.
  I dont want to stretch the point but this can be
  compared to buying a car and expecting it to
  drive itself. Of course youd be disappointed
  with performance.
• And yes cars have design and performance
  glitches, as do PMs.
• But cars can be useful and wonderful things.
• The same, may I say it, goes for Prediction
  Markets.

83
Leighton.Vaughan-Williams_at_ntu.ac.uk