The Kelly criterion and its variants: theory and practice in sports, lottery, futures

1
The Kelly criterion and its variants theory and
practicein sports, lottery, futures options
trading
Professor William T ZiembaAlumni Professor of
Financial Modeling and Stochastic Optimization
(Emeritus), UBCICMA Financial Markets Centre,
University of ReadingVisiting Professor
Mathematical Institute, Oxford University
andPresident, William T Ziemba Investment
Management Inc Kaist Lecture Program August 2011
2
Who I Am
RAB/Canyon 2005-07
PIWM 2007 to present
3
The two trading routes available for cash, equity
and equity Futures hedge funds

• The search for positive ? and the generation of
  returns from ? risk in smart index funds and
  active equity portfolio management

?Iexcess mean return ?ileveraging factor for
long market exposure

• The search for absolute returns using research on
  market imperfections, security market biases,
  mispriced derivative securities, arbitrage, risk
  arbitrage, and superior investment criteria

• The strategy is to win in all markets - up, down
  and even, and to achieve a smooth wealth path
  with few drawdowns.

4
To win consistently you must

• Get the mean right that is the direction of the
  market - adjusted for various types of hedging
  and positioning. This must consider your risk
  tolerance.
• You must diversify so that regardless of the
  market path (scenario) the positions are not
  overbet.
• You must bet (portfolio allocate) well.
• I argue that getting the mean right is the most
  important ingredient in winning strategies.
• This was especially crucial in the equity markets
  the past 10 years the mean was frequently
  negative.

5
The Importance of getting the mean right. The
mean dominates if the two distributions cross
only once.

• Thm Hanoch and Levy (1969)
• If XF( ) and YG( ) have CDFs that cross only
  once, but are otherwise arbitrary, then F
  dominates G for all concave u.
• The mean of F must be at least as large as the
  mean of G to have dominance.
• Variance and other moments are unimportant. Only
  the means count.
• With normal distributions X and Y will cross only
  once iff the variance of X does not exceed that
  of Y
• Thats the basic equivalence of Mean-Variance
  analysis and Expected Utility Analysis via second
  order (concave, non-decreasing) stochastic
  dominance.

6
Errors in Means, Variances and Covariances
7
Mean Percentage Cash Equivalent Loss Due to
Errors in Inputs
Risk tolerance is the reciprocal of risk
aversion. When RA is very low such as with log
u, then the errors in means become 100 times as
important. Conclusion spend your money getting
good mean estimates and use historical variances
and covariances
8
Average turnover percentage of portfolio sold
(or bought) relative to preceding allocation

• Moving to (or staying at) a near-optimal
  portfolio may be preferable to incurring the
  transaction costs of moving to the optimal
  portfolio
• High-turnover strategies are justified only by
  dramatically different forecasts
• There are a large number of near-optimal
  portfolios
• Portfolios with similar risk and return
  characteristics can be very different in
  composition
• In practice (Frank Russell for example) only
  change portfolio weights when they change
  considerably 10, 20 or 30.
• Tests show that leads to superior performance,
  see Turner-Hensel paper in ZM (1998).

9
Log Utility Bernoulli (1732)

• In the theory of optimal investment over time, it
  is not quadratic (the utility function behind the
  Sharpe ratio) but log that yields the most long
  term growth.
• But the elegant results on the Kelly (1956)
  criterion, as it is known in the gambling
  literature and the capital growth theory as it is
  known in the investments literature, see the
  survey by Hakansson and Ziemba (1995) and MacLean
  and Ziemba (2006), that were proved rigorously by
  Breiman (1961) and generalized by Algoet and
  Cover (1988) are long run asymptotic results.
• However, the Arrow-Pratt absolute risk aversion
  of the log utility criterion is essentially
  zero, where u is the utility function of wealth
  w,, and primes denote differentiation.
• The Arrow-Pratt risk aversion index.

• is essentially zero, where u is the utility
  function of wealth w, and primes denote
  differentiation.
• Hence, in the short run, log can be an
  exceedingly risky utility function with wide
  swings in wealth values.

10
Long run exponential growth is equivalent to
maximizing the expected log of one periods
returns
11

• Thus the criterion of maximizing the long run
  exponential rate of asset growth is equivalent to
  maximizing the one period expected logarithm of
  wealth. So an optimal policy is myopic.
• Max G(f) p log (1f) q log (1-f) ? f
  p-q
• The optimal fraction to bet is the edge p-q

12
Slew O Gold, 1984 Breeders Cup Classic f64
for place/show suggests fractional Kelly.
13
Maximizing long run growth

• Thus the criterion of maximizing the long run
  exponential rate of asset growth is equivalent to
  maximizing the one period expected logarithm of
  wealth.
• So an optimal policy is myopic - future and past
  do not affect current optimal decisions
• Max G(f) p log (1f) q log (1-f) ? f
  p-q
• The optimal fraction to bet is the edge p-q (the
  mean)
• So if the edge is large, the bet is larger
• p .99, q .01 ? f 98 of wealth

14
What does the theory tell us about long term
hedge fund trading and overbetting?
Kelly and fractional Kelly - explaining the
overbetting that leads to hedge fund disasters
you cannot ever bet more than full Kelly and
usually you should bet less
15
Mohnish Pabrai, investing in Stewart Enterprises
- Thorp (2010) in our World Scientific book
Hedge fund manager won bidding for 2008 lunch
with Warren Buffett for 600K Stewart
Enterprises, Payoff 24 months Prob Net
Return 0.8 gt100 0.19 zero 0.01 lose all
investment Pabrai bet 10 of his fund Whats the
full Kelly bet? f 0.975 half Kelly 0.3875
quarter Kelly0.24375
16
Pabrai bet issues should he have bet more?
Other opportunities must compute against all
options (nonlinear or stochastic optimization)
for the available wealth Risk tolerance what
fractional Kelly to use? Black Swans we call
them bad scenarios Long vs short run planning
17
Kelly betting at PIMCO

• During an interview in the Wall Street Journal
  (March 22-23, 2008) Bill Gross and Ed Thorp
  discussed turbulence in the markets, hedge funds
  and risk management.
• Bill considered the question of risk management
  after he read Beat the Dealer in 1966.
• That summer he was off to Las Vegas to beat
  blackjack.
• Just as Ed did some years earlier, he sized his
  bets in proportion to his advantage, following
  the Kelly Criterion as described in Beat the
  Dealer, and ran his 200 bankroll up to 10,000
  over the summer.
• Bill has gone from managing risk for his tiny
  bankroll to managing risk for Pacific Investment
  Management Companys (PIMCO) investment pool of
  almost 1 trillion.
• He still applies lessons he learned from the
  Kelly Criterion.
• As Bill said, Here at PIMCO it doesnt matter
  how much you have, whether its 200 or 1
  trillion Professional blackjack is being played
  in this trading room from the standpoint of risk
  management and thats a big part of our success..

18
Various trading records for bettors who behave
like Kelly investors
19
Top 10 equity holdings of Soros Fund Management
and Berkshire Hathaway, Sept 30, 2008 (SEC
filings)
20
The wealth levels from December 1985 to April
2000 for the Windsor Fund of George Neff, the
Ford Foundation, the Tiger Fund of Julian
Robertson, the Quantum Fund of George Soros and
Berkshire Hathaway, the fund run by Warren
Buffett, as well as the SP500 total return
index.
21
Berkshire Hathaway versus Ford Foundation,
monthly returns distribution, January 1977 to
April 2000
22
Return distributions of all the funds, quarterly
returns distribution, December 1985 to March 2000
23
(No Transcript)
24
Classic Breiman Results
25
(No Transcript)
26
Kelly and half Kelly medium time simulations
Ziemba-Hausch (1986)
These were independent
27
The good, the bad and the ugly
166 times out of 1000 the wealth is more than 100
times the initial wealth with full Kelly but only
once with half Kelly does the investor gain this
much But probability of being ahead is higher
with half Kelly, 87 vs 95.4 Min wealth is 18
and only 145 with half Kelly 700 bets all
independent with a 14 edge the result, you
can still lose over 98 of your fortune with bad
scenarios With half Kelly, lose half of wealth
only 1 of the time but is is 8.40 with full
Kelly So even after 700 plays, the strategy is
still risky See  Simulation paper  for more on
this and two more examples
28

• Kentucky Derby 1934-1998
• Use inefficient market system in Hausch, Ziemba,
  Rubinstein (1981) and Ziemba-Hausch books
• Place/show wagers made when prices off
  sufficiently and EX 1.10
• w0 2500 63 years 72 wagers with 45 (62.5)
  successful

29
Typical wealth level histories with one scenario
(the actual results) from place and show betting
(Dr Z system) on the Kentucky Derby, 1934-1994
with Kelly, half Kelly and betting on the
favorite strategies
30
Overbetting
Probability of doubling and quadrupling before
halving and relative growth rates versus fraction
of wealth wagered for Blackjack (2 advantage,
p0.51 and q0.49 Should you ever be above 0.02
that is positive power utility like It is
growth-security dominated.
Betting more than the Kelly bet is non-optimal as
risk increases and growth decreases betting
double the Kelly leads to a growth rate of zero
plus the riskfree asset. LTCM was at this level
or more, see AIMR, 2003. Several similar blowouts
are discussed in Ziemba and Ziemba (2007)
including Amaranth and Niederhoffer.
31
Growth Rates Versus Probability of Doubling
Before Halving for Blackjack
32
Fractional Kelly and negative power utility

• u(w) -w? ?lt0
• ? 0 u ? log
• f1/(1- ?) fraction (Kelly) in log optimal
  portfolio, rest in cash
• ?0 f1 full Kelly
• ?-1 f1/2 1/2 Kelly
• ?--3 f1/4 1/4 Kelly futures trading down here
• This is exact with log normality and approximate
  otherwise but it can be way off.

33
Samuelsons critique of Kelly betting
Correspondence Nov 16, 2005 to Elwyn
Berlekamp Dec 13, 2006 to WTZ Samuelson
postulated three investors, all risk averse and
concave WTZ adds two more Ida (the most risk
averse) and Victor (the most risk accepting) Cash
return zero, Stock 50-50 4 or 0.25 for 1 bet
(for every period)
34
The Investors

• Tom, I believe, is overbetting and dominated and
  will go bankrupt
• Harriet has a limited degree of risk tolerance,
  fits well with lots of empirical Wall St equity
  premium data

35
Some tests
36
(No Transcript)
37
(No Transcript)
38
and in Thorp (2006)
39
(No Transcript)
40
Aside

• Victor Niederhoff always had high returns of
  blowouts leveraging the SP500
• Too close to the money for proper risk control
• See Chapter 12 of Ziemba and Ziemba (2007)
• There are more blowouts since then!

41
(No Transcript)
42
(No Transcript)
43
Response

• This is correct. I have no disagreement.
• The fully Kelly strategy gives very high final
  wealth most of the time
• But it is possible to have low final wealth with
  no leveraging (-98) and many times W0 lost with
  leveraging
• See examples in Simulation Section

44
(No Transcript)
45
Horseracing

• market in miniature
•  
• fundamental and technical systems
•  
• returns and odds are determined by
• 1) participants -- like stock market, unlike
  roulette
• 2) transaction costs -- track take (17),
  breakage
• rebates now plus Betfair (long short)
•  
• bet to
• 1) win -- must be 1st
• 2) place -- must be 1st or 2nd
• 3) show -- must be 1st, 2nd or 3rd

46
Place market in horseracing
Inefficiencies are possible since   1) more
complex wager 2) prob(horse places) gt prob(horse
wins) gt favorites may be good bets
To investigate place bets we need   1) determine
place payoffs  2) their likelihood  3) expected
place payoffs 4) betting strategy, if expected
payoffs are positive
Bettors do not like place and show bets.
47
The Idea

1. Use data in a simple market (win) to generate
   probabilities of outcomes
2. Then use those in a complex market (place and
   show) to find positive expectation bets
3. Then bet on them following the capital growth
   theory to maximize long run wealth

48
Effect of transactions costs, calculation of
optimal place and show Kelly bets
Non concave program but it seems to converge. In
practice, adjust qs to replicate biases. Victor
Lo research on this in his thesis and Hausch, Lo,
Ziema 1994, 2008 books
49
(No Transcript)
50
Use in a calculator
What we do in the system is to reduce the
non-convex log optimization problem down to four
numbers Wi,, W, and Si, S or Pi, P,
Thousands of race results regress the expected
value and the optimal Kelly bet as a function of
these four variables. Hence, you just find horses
where the relative amount bet to place or show is
below the bet in the win pool. The calculator
tells you when the expected value is say 1.10 or
better and calculates the optimal Kelly bet. So
this can be done in say 15 seconds.
51
Exhibition Park, 1978, typical returns.
52
Aqueduct, 1981-82
53
Expected value approximation equations

• Expected value (and optimal wager) are functions
  of only four numbers - the totals and the horse
  in question.
• These equations approximate the full optimized
  optimal growth model.
• Solving the complex NLP too much work and too
  much data for most people.
• This is used in the calculators, and
  Hausch-Ziemba (1985, Management Science),
  differing track take, etc.

54
1983 Kentucky Derby
55
1991 Breeders Cup Race 5
56
Simulations in 2004-5
Real results April 2005-March 2006 Up 36,000
2 on bets 1.5 M, System -7, rebate 9, edge
2 We keep doing this searching for good bets
at 80 tracks
57
Back to Samuelson
58
(No Transcript)
59
(No Transcript)
60
Before going more into Samuelson and the
simulations, lets look at when the Kelly bets
are very large or very small
61
Commodity Trading Turn of Year Effect
Small cap stocks have outperformed large cap
stocks in January on a regular basis since 1926
Average excess returns of smallest minus largest
decile of US stocks, 1926-93, Source Ibbotson
Associates
62
January effect historical and recent, move to
December
January effect, 1926-1995. January size premium
R(10th)-R(1st).
63
sell
buy
Futures play with anticipation, mid December to
mid January, this is a typical year in the mid
90s, Value Line versus SP, 1992-3
64
Turn of the year effect using past data to get
the probability distributons of returns
Probability of reaching 10 million before ruin
for Kelly, half Kelly and quarter Kelly strategies
Relative growth rate and probability of doubling,
tripling or tenfolding before halving for various
Kelly strategies
65
Turn of the year effect, recent developments
Futures markets - much more violent Russell 2000
- has more volume than Value Line Effect moved
into December Textbooks and finance experts say
effect is not there Graphs in Hensel-Ziemba paper
in Keim-Ziemba (2000) Worldwide security market
imperfections, Cambridge University Press. Doing
this trade is like driving a dynamite truck
smoking a cigar. You do it carefully. Rendon-Ziem
ba (2007) update to 2005 turn of the year Value
Line/SP500 and Russell 2000/SP500 spread
trades Ziemba (2011) new paper updating to 2011,
showing the trade still worked in 2009/10 and
2010/11
66
My experiences trading the TOY

• I started trading the January turn-of-the-year
  effect for the 1982/83 TOY and for the next 13
  TOYs
• During that time the cash small stock advantage
  was in the first half of January
• But with futures anticipation, the effect covered
  the mid-December to mid-January period
• A rule we devised in 82/83 was to go long the
  Value Line short the SP500 during this period
• However, the effect moved to be fully in the
  second half of December
• I managed to make gains in all 14 of these years
  and be 7 of the market
• But for two reasons stopped trading it
• Low Value Line volume, and
• Teaching the trade to Morgan Stanley as a
  consultant to Peter Mullers group
• I continued to do research and write papers on
  this and returned to trade the effect the past
  two TOYs with the Russell2000 replacing the Value
  Line

67
TOY graphs
The paper Investing in the Turn-of-the-Year
hasgraphs and tables of the trades for various
years The next slide has four years of VL/SP
trades and the following slide has four years of
the R2000/SP trades Each year was a little
different but it was possible to win each
year. Finally, the third set of graphs after the
monthly R2000-SP500 futures spread, 1993-2009
has 2009/2010 and 2010/11 where we entered on the
dots and exited on the squares as the market
turned.
68
Various turns of the year
69
Various turns of the year (contd)
70
Russell2000 - SP500 Futures Spread Average
Returns during the MOY, 1993-2010
Observe December!
71
Russell2000 - SP500 spread with our entries and
exits
72
Unpopular numbers in the Canadian 6/49, 1984,
1986, and 1996 Lotto
73
Lotto games, experimental data Very small
Kelly bets when the probability of losing is high
74
Probability of doubling, quadrupling and
tenfolding before halving, Lotto 6/49
Case A
Case B
75
Probability of reaching the goal of 10 million
before falling to 25,000 with various initial
wealth levels for Kelly, 1/2 Kelly and 1/4 Kelly
wagering strategies
The downside of the analysis is that the expected
time to win a lot is in the millions of years.
76
Calculating the optimal Kelly fraction staying
above an exogenous wealth path
To stay above a wealth path using a Kelly
strategy is very difficult
Kelly fractions and path achievement

• the more attractive the investment opportunity,
• the larger the bet size and
• hence the larger is the chance of falling below
  the path.

MSZZ (2004) using a continuous time lognormally
distributed asset model calculate that function
to stay above a path at various points in time to
stay with a high exogenously specified value at
risk probability. Convex case like Geyer-Ziemba
(2008) Siemens Vienna pension model - can do on a
computer in MacLean, Zhao, Ziemba (2009)
77
The planning horizon is T3, with 64 scenarios
each with probability 1/64
78
With initial wealth W(1)1, the value at risk is
a. The optimal investment decisions and optimal
growth rate for a, the secured average annual
growth rate and 1-a, the security level are shown
in the table.
79
(No Transcript)
80
(No Transcript)
81
(No Transcript)
82
(No Transcript)
83
Brief Guide to Capital Growth Theory and Kelly
Criterion Literature

Bernoulli (1732) translated in 1954 original
idea of log utility marginal utility, current
wealth proportion to St Petersburg
Paradox MacLean, Thorp, Ziemba (2009) book with
main articles reprinted plus new ones
84
Some properties of the Capital Growth Theory

85
Some properties of the Capital Growth Theory
(contd)

86
Some properties of the Capital Growth Theory
(contd)

87
Some properties of the Capital Growth Theory
(contd)

88
Some properties of the Capital Growth Theory
(contd)

89
Some properties of the Capital Growth Theory
(contd)

See Ziemba and Hausch (1996), Aucamp (1993),
Browne (1997) and MacLean, Thorp, Ziemba (2010)
for more on this.
90
References
Essentially all of the material in this talk is
in the following books plus the papers listed
above Ziemba, The Stochastic Programming Approach
to Asset Liability Management, AIMR,
2003 Ziemba-Hausch, Dr Zs Beat the Racetrack,
William Morrow, 1987 (has UK betting
system) Hausch-Lo-Ziemba, Efficiency of Racetrack
Betting Systems, Academic Press, 1994. Classic
new and reprinted articles, bible for Hong Kong
professional betting teams. Originals sell for
huge prices as high as 12,000 I am told, I sold
one for 1400. 2008 2nd edition from World
Scientific in Singapore at a low
price. Ziemba-Vickson, Stochastic Optimization
Models in Finance, Academic Press, 1975. Classic
articles, new articles, huge collection of
portfolio theory, problems.Reprinted by World
Scientific, Singapore, 2006. Ziemba et al, 6/49
Lotto Guidebook, 1986 Ziemba-Hausch, Betting at
the Racetrack, 1986, exotic bet pricing Ziemba
and Ziemba (2007) Scenarios for Risk Management
and Global Invetment Strategies, Wiley MacLean,
L.C., E. O. Thorp, Ziemba, W.T., Eds., The Kelly
Capital Growth Criterion Theory and Practice,
World Scientific Books all available,
wtzimi_at_mac.com for information. Amazon has them
at low prices.