Random Number Generators

1
Random Number Generators
2
Applications

• Random Number Generators (RNGs) used very widely
  in computer simulations of all sorts of
  systems
• Basis for doing numerical experiments in
• Weather and climate
• Social studies
• Traffic analysis
• All sorts of Monte Carlo simulations
• Network systems

3
What is Random ?

• Uncorrelated withwhatever it is you are
  interested in
• We believe quantum systems are truly random
• Some (practical and commercial) attempts to make
  random number generators from radioactive sources
• Actually for simulations we often want
  repeatable random sequences

4
Pseudo Random

• Computer generated random numbers are obviously
  no longer truly random
• But they can be made random enough
• We refer to them as pseudo-random
• Usually implemented by some algorithm that
  generates a very long sequence of random
  deviates given a particular starting point

5
Desireable Properties

• A very, very long sequence - ideally that does
  not recur in any practicable compute time - eg
  lifetime of the known universe
• But this is a moving target as computers get
  faster and faster

6
Well-understood bit properties

• Ideally want a generator that is made up of
  completely random bits
• That way we can make whatever sort of numbers
  we like from them
• In practice algorithms often generate either
  integers or real numbers
• We have to be very careful how we decompose
  them into bits - not all the bits will be
  random

7
Decorrelated Number Sequence

• Want successive deviates to be decorrelated
• Simple tests might look for an equal number
  of each numerical outcome
• eg coin tosses - expect 50/50
• 16 bit integers practical to histogram and
  test that for a large sample size we do get
  roughly equal occurrences of all combinations
• Scatter plot tests show no obvious patterns
• Lots of other good statistical tests

8
Repeatability

• Generator needs some means whereby the user
  can get the same sequence again if they want
• (Need to verify/reproduce an experiment)
• Alternative is often to seed the generator
  using the system clock, if we do not care about
  repeatability
• Also want to be able to obtain same results on
  a different platform (architecture and
  Operating system..)

9
Speed and Performance

• For numerical simulations the size of system
  of the quality of our experiment may depend on
  how fast our generator runs
• Want the cost of generating a good quality
  deviate to approach one per machine cycle

10
Generator Algorithms

• Computer Systems often come with an RNG
  built in
• eg rand() or random()
• Unfortunately they are not always very good
• They may not be very random and may not have
  very long repeat sequences, may exhibit
  discernible correlation patterns and in short
  usually plain suck!

11
Linear Congruential Generator

• A sequence of random values X_n whereby
• X_n1 ( a X_n c ) mod m
• with well-chosen values of a, c, m
• Period is at most m (and can be shorter for
  bad choices..)
• Quite fast however, and if we combine several
  such generators to do shuffles etc we might
  get a good one.
• Often formulated to generate integers

12
Lagged Fibonacci

• Using the Fibonacci sequence definition
• S_n S_n-1 S_n-2
• We can exploit as
• S_n (S_n-j OP S_n-k ) mod m, for 0 lt j lt k
• Where OP is some binary operator such as ,
  - or XOR
• And we need to keep k past states
• eg Marsaglia generator generates good quality
  24-bit floating point deviates

13
Mersenne Twistor

• http//www.math.sci.hiroshima-u.ac.jp/m-mat/MT/em
  t.html
• Based on a matrix linear recurrence over a finite
  binary field
• Generates 64-bit integers

14
Combining Generators

• We can use one generator to shuffle a table
  of deviates from another
• This can improve the decorrelation a lot
• Can also use to lengthen the repeat cycle of the
  sequence

15
Software Practicalities

• Generate Integers or Real numbers?
• Rarely we get a generator that does bits
  directly
• Can make uniform deviates from integers?
• Uniform most useful define d to be a flat
  distribution between 0 inclusive and 1
  exclusive
• Written as 0,1) and very useful for probability
  calculations etc

16
What Data Primitives?

• 32 bit or 64 bit or longer ints available?
• Signed or unsigned?
• 32 or 64 bit floats?
• What does your CPU chip support?
• What about Virtual Machines or Interpreters?

17
Memory Considerations

• Simple generators have state of one machine
  word - eg a seed integer
• Some restrictions on sensible initial values
  eg non-zero, and with some maximum value
• More sophisticated generator need to store a
  longer state table
• How to save/reload the state table?
• Serialise it? Binary data files?

18
Other Distributions

• Various transforms to obtain other
  distributions apart from Uniform.
• Gaussian or Normal Distribution is common
• Use Box-Muller transformation (circle embedded
  in square) to transform pairs of uniform
  deviates into a Gaussian pair.
• Other transforms let us generate Poissonian
  deviates, Binomial distributions,

19
Parallel RNG Issues

• May have several processors or auxilliary
  units generating and consuming deviates
• Need to make sure parallel processors working
  on the same application have different
  sequences to work on or are at least well
  separate in the same sequence if using the same
  generator algorithm
• Saving/restarting can be expensive

20
Further Information

• See the Web - plenty on Wikipedia on RNGs
• See Numerical Recipes the Art of Scientific
  Computing, by Press et al, Chapter 7 gives good
  review of Random Number Generators
• Numerous articles by G.Marsaglia, P. LEcuyer,
  and others
• The Art of Computer Programming Vol 2.
  Seminumerical Algorithms D. Knuth