Random Number Generators

1
Random Number Generators
2
Why do we need random variables?

• random components in simulation
• ? need for a method which generates numbers that
  are random
• examples
• interarrival times
• service times
• demand sizes

3
terminology

• Random Numbers
• stochastic variable that meets certain conditions
• numbers randomly drawn from some known
  distribution
• It is impossible to generate them with the help
  of a digital computer.
• Pseudo-random Numbers
• numbers which seem to be randomly drawn from some
  known distribution
• may be generated with the help of a digital
  computer

4
techniques for generating random variables

• ten-sided die (each throw generates a decimal)
• throwing a coin n times
• get a binary number between 0 and 2n-1
• other physical devices
• observe substances undergoing atomic decay
• e.g., Caesium-137, Krypton-85,. . .

5
techniques for generating pseudo-random variables

• numbers are generated using a recursive formula
• ri is a function of ri-1, ri-2,
• relation is deterministic no random numbers
• if the mathematical relation not known and well
  chosen it is practically impossible to predict
  the next number
• In most cases we want the generated random
  numbers to simulate a uniform distribution over
  (0, 1), that is a U(0, 1)-distribution
• therere many simple techniques to transform
  uniform (U(a, b)) samples into samples from other
  well-known distributions

6
terminology

• seed
• random number generators are usually initialized
  with a starting number which is called the seed
• different seeds generates different streams of
  pseudorandom numbers
• the same seed results in the same stream
• cycle length
• length of the stream of pseudorandom numbers
  without repetition

7
methods for generating pseudo-random variables

• all methods usually generate uniformly
  distributed pseudorandom numbers in the interval
  0,1
• midsquare method (outdated)
• congruential method (popular)
• shift register (Tausworthe generator)

8
midsquare method

• arithmetic generator
• proposed by von Neumann and Metropolis in 1940s
• algorithm
• start with an initial number Z0 with m digits
  (seed) m even
• square it and get a number with 2m digits (add a
  zero at the beginning if the number has only 2m-1
  digits)
• obtain Z1 by taking the middle m digits
• to be in the interval 0,1) divide the Zi by 10m
• etc.

9
midsquare method example (Z0 7182, m 4)
i 0 1 2 3 4 5 6 7
Zi 7182 5811 7677 9363 6657 3156 9603 2176

Ui 0.7182 0.5811 0.7677 0.9363 0.6657 0.3156 0.960
3 0.2176
Zi2 51581124 33767721 58936329 87665769
44315649 9960336 92217609 4734976
10
midsquare method (cont.)

• advantages
• fairly simple
• disadvantage
• tends to degenerate fairly rapidly to zero
• example try Z0 1009
• not random as predictable

11
Linear Congruential Generator (LCG)

• LCGs
• first proposed by Lehmer (1951).
• produce pseudorandom numbers such that each
  single number determines its successor by means
  of a linear function followed by a modular
  reduction
• to be in the interval 0,1) divide the Zi by m
• Z0 seed a multiplier (integer)
• c increment (integer) m modulus (integer)
• Variations are possible
• combinations of previous numbers instead of using
  only the last value

12
LCG example
a 5 c 3 m 16 Z0 7
i 0 1 2 3 4 5 6 7
Zi 7 6 1 8 11 10 5 12
Ui 0.4375 0.375 0.0625 0.5 0.6875 0.625
0.3125 0.75
Z1 (5Z0 3) mod 16 38 mod 16 6 Z2 (5Z1
3) mod 16 33 mod 16 1 Z3 (5Z2 3) mod
16 8 mod 16 8
13
properties of LCGs

• parameters need to be chosen carefully
• nonnegative
• 0 lt m a lt m c lt m Z0 lt m
• disadvantages
• not random
• if a and m are properly chosen, the Uis will
  look like they are randomly and uniformly
  distributed between 0 and 1.
• can only take rational values 0, 1/m, 2/m,
  (m-1)/m
• looping

14
LCG (cont.)

• looping
• whenever Zi takes on a value it has had
  previously, exactly the same sequence of values
  is generated, and this cycle repeats itself
  endlessly.
• length of sequence period of generator
• period can be at most m
• if so LCG has full period
• LCG has full period iff
• the only positive integer that divides both m and
  c is 1
• if q is a prime number that divides m then q
  divides a-1
• if 4 divides m, then 4 divides a-1

15
Linear Feedback Shift Register Generators (LFSR)

• developed by Tausworthe (1965)
• related to cryptographic methods
• operate directly on bits to form random numbers
• linear combination of the last q bits
• ci constants (equal to 0 or 1, cq 1)
• in most applications only two of the cj
  coefficients are nonzero
• execution can be sped up
• modulo 2 equivalent to exclusive-or

16
LFSR (cont.)

• form a sequence of binary integers W1, W2,
• string together l consecutive bits
• consider them as numbers in base 2
• transform them into Uis
• maximum period 2q -1
• if l is relatively prime to 2q-1 LFSR has full
  period

17
LFSR example

• r 3 q 5 b1 b2 b3 b4 b5 1 l 4

I 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 17 18
bi 1 1 1 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1
i 1 2 3
Wi (base 2) 1111 1000 1101
Wi (base 10) 15 8 13
Ui 0.9375 0.5 0.8125
18
Testing RNGs

• Empirical Tests

19
random number generators

• methods presented so far
• completely deterministic
• Uis appear as if they were U(0,1)
• test their actual quality
• how well do the generated Uis resemble values of
  true IID U(0,1)
• different kinds of tests
• empirical tests
• based on actual Uis produced by RNG
• theoretical tests

20
Chi-Square test (with all parameters known)

• checks whether the Uis appear to be IID U(0,1)
• divide 0,1 into k subintervals of equal length
  (k gt 100)
• generate n random variables U1, U2, .. Un
• calculate fj (number of Uis that fall into jth
  subinterval)
• calculate test statistic
• for large n ?2 will have an approximate
  chi-square distribution with k-1 df under the
  null hypothesis that the Uis are IID U(0,1)
• reject null hypothesis at level if

21
Generating Random Variates
22
Generating Random Variates

• General Approach
• Inverse Transformation
• Composition
• Convolution
• Generating Continuous Random Variates
• Uniform U(a,b)
• Exponential
• m-Erlang
• Gamma, Weibull
• Normal
• etc

23
Inverse Transformation

• generate continuous random variate X
• distribution function F (continuous, strictly
  increasing)
• 0 lt F(x) lt 1
• if x1 lt x2 then 0 lt F(x1) F(x2) lt 1
• inverse of F F-1
• algorithm
• generate U U(0,1)
• return X F-1(U)

24
Inverse Transformation (cont.)
F(x)

• returned value X has desired distribution F

1
U1
U2
x
0
X2
X1
25
Inverse Transformation (cont.)

• create X according to exponential distribution
  with mean
• in order to find F-1 set

take natural logarithm (base e)
26
Inverse Transformation for discrete variates

• distribution function
• probability mass function
• algorithm
• generate U U(0,1)
• determine smallest possible integer i such that U
  F(xi)
• return X xi

27
Composition

• applies then the distribution function F can be
  expressed as a convex combination of other
  distribution functions F1, F2, ..
• we hope to be able to sample from the Fjs more
  easily than from the original F
• if X has a density it can be written as
• algorithm
• generate positive random integer J such that P(J
  j) pj
• Return X with distribution function FJ

28
Composition (example)

• double-exponential distribution (laplace
  distribution)
• can be rewritten as
• where IA(x) is the indicator function of set A
• f(x) is a convex combination of
• f1(x) ex I(-1, 0) and f2(x) e-x I0, 1)
• p1 p2 0.5

29
Convolution

• desired random variable X can be expressed as sum
  of other random variables that are IID and can be
  generated more readily then X directly
• algorithm
• generate Y1, Y2, Ym IID each with distribution
  function G
• return X Y1 Y2 ? Ym
• example m-Erlang random variable X with mean
• sum of m IID exponential random variables with
  common mean /m

30
Generate Continuous Variates

• Uniform distribution X(a,b)
• X a (b-a)U
• Exponential (mean )
• X - ln (1-U) or X - ln U
• m-Erlang