Random Number Generation

Dr. Akram Ibrahim Aly

Copyright Akram I. Aly 2012
Lecture (8)
Random Numbers
What is a random number?
A single number is not random. Only an infinite
sequence can be described as random.
Random means the absence of order.
All subsequences are equally distributed.

Applications of Random Numbers
Random numbers have a lot of real world
applications; e.g.:
sampling
Simulation: generate event times and random
variables in simulation
statistical inference
random search optimization
lotteries, gambling

Properties of random numbers
A sequence of random numbers R1, R2, , Rn
must have 2 important statistical properties:
Uniformity
Independence
Uniformity
Each random number is an independent sample
drawn from a continuous uniform distribution
between 0 and 1: R
i
[0,1]

s s
=
otherwise , 0
1 0 , 1
) (
x
x f
2
1
2
) (
1
0
1
0
2
= = =
}
x
xdx R E
12
1
4
1
3
1
2
1
3
) (
2
1
0
1
0
3
2
= =
|
.
|

\
|
= =
}
x
dx x R V
x
f
(
x
)

0
1
PDF:
Independence
Independence: the probability of observing a
value in a particular interval is independent of
the previous value drawn.
Generation of Pseudo-Random
Numbers
Pseudo means fake
Pseudo, because we utilize known methods to
generate numbers which removes the potential for
true randomness.
Pseudo-random numbers: Deterministic sequence of
numbers with a repeat period but with the appearance
of randomness (if you dont know the algorithm).

Generation of Random Numbers
Goal: To produce a sequence of numbers in [0,1]
that simulates, or imitates, the ideal properties of
random numbers (RN).
Important considerations in RN routines:
Fast
Portable to different computers
Have sufficiently long cycle
Replicable
Closely approximate the ideal statistical properties of
uniformity and independence.

Techniques for Generating Random
Numbers
Mid-square method
Linear Congruential Method
Combined Linear Congruential Method
Tausworthe generators (self study)
Extended Fibonacci generators (self study)
Mid-Square Method
Proposed by Von Neumann and Metropolis in
the 1940s
1. Start with an initial d-digit integer seed X
0
2. Square X
0
3. Take the middle d-digits as the next four-digit
number, X
1
4. Place a decimal point at the left of X
1
to get first
random number R
1
5. Square X
1
and continue process

SEED
Random number seed-initial random number
used to generate the next random number
which is in turn transformed into the new seed
value.
Example
Let X
0
= 5497
X1= (5497)
2
= 30217009 X
1
= 2170 R
1
= 0.2170
X2= (2170)
2
= 04708900 X
2
= 7089 R
2
= 0.7089
X3= (7089)
2
= 50253921 X
3
= 2539 R
3
= 0.2539
...and so on...

Example 2: Poor Mid-Square
Generator
Let X
0
= 5197
X1= (5197)
2
= 27008809 X1= 0088R1= 0.0088
X2= (0088)
2
= 00007744X2= 0077R2= 0.0077
X3= (0077)
2
= 00005929X3= 0059R3= 0.0059
X4= (0059)
2
= 00003481X4= 0034R4= 0.0034
Once zeros appear, they are carried in subsequent
numbers.
Example 3: Poor Mid-Square
Generator
Let X
0
= 6500
X
1
= (6500)
2
= 42250000X1= 2500R1= 0.2500
X2= (2500)
2
= 06250000X2= 2500R2= 0.25000
Cannot choose a seed that guarantees that the
sequence will not degenerate and will have a long
period.
Advantages and disadvantages of
the MID-SQUARE method
Advantages of the mid-square method
Rather simple to implement
Disadvantages of the mid-square method
Difficult to choose initial seed that will give good
sequence
Strong tendency to degenerate fairly rapidly to
zero

More Examples on the mid-square
Method
17
The Mid-product Method
Improved Variation of Mid-square method
Example: Start with two arbitrary 3-digit numbers
Z
0
= 123 Z
1
= 456
Z
0
Z
1
= (123)(456) = 56088 U
1
= 0.608 Z
2
= 608
Z
1
Z
2
= (456)(608) = 277248 U
2
= 0.772 Z
3
= 772
Z
2
Z
3
= (608)(772) = 469376 U
3
= 0.693 Z
4
= 693

Linear Congruential Method
by D. H. Lehmer, 1951
To produce a sequence of integers, X
1
, X
2
, between 0 and
(m-1) by following a recursive relationship:




If c0, the form is called the mixed congruential
method.
If c=0, the form is known as the multiplicative
congruential method

Linear Congruential Method
The random integers are being generated in the
interval [0, m-1], and to convert the integers to
random numbers:
For [0,1[


For [0,1]

,... 2 , 1 = = i
m
X
R
i
i
,... 2 , 1
1
=

= i
m
X
R
i
i
Example
Use X0= 27, a = 17, c = 43, and m = 100.
The Xi and Ri values are:
X1= (17*27+43) mod 100 = 502 mod 100 = 2,R1= 0.02;
X2= (17*2+32) mod 100 = 77, R2= 0.77;
X3= (17*77+32) mod 100 = 52, R3= 0.52;

21
Example

For example,


Starting with x
0
=5:


The first 32 numbers obtained by the above procedure
10, 3, 0, 1, 6, 15, 12, 13, 2, 11, 8, 9, 14, 7, 4, 5;
10, 3, 0, 1, 6, 15, 12, 13, 2, 11, 8, 9, 14, 7, 4, 5.
By dividing x's by 16:
0.6250, 0.1875, 0.0000, 0.0625, 0.3750, 0.9375, 0.7500, 0.8125, 0.1250,
0.6875, 0.5000, 0.5625, 0.8750, 0.4375, 0.2500, 0.3125;
0.6250, 0.1875, 0.0000, 0.0625, 0.3750, 0.9375, 0.7500, 0.8125, 0.1250,
0.6875, 0.5000, 0.5625, 0.8750, 0.4375, 0.2500, 0.3125.
Selection of LCG Parameters
The selection of the values for a, c, m, and X
0

affects the statistical properties and the cycle
length.
Most natural choice for m is one that equals to
the capacity of a computer word.
m = 2
b
(binary machine), where b is the number of
bits in the computer word.
m = 10
d
(decimal machine), where d is the number
of digits in the computer word.
23
Selection of LCG Parameters
The modulus m should be large
All generated numbers, X
i
, are between 0 and m-1,
the period can never be more than m
For mod m computation to be efficient, m
must be a power of 2 Mod m can be
obtained by dropping the leftmost binary
digits in X
i+1
and using only the b rightmost
binary digits
24
Example
Use a 4-bit computer
Let m = 2
4
= 16, a = 5, c = 3, Z
0
= 7
Use Z
i
= (5 Z
i-1
+ 3) (mod 16)
Z
1
= (57+3) (mod 16) = 38 (mod 16) = 6
Z
2
= (56+3) (mod 16) = 33 (mod 16) = 1
Z
3
= (51+3) (mod 16) = 8 (mod 16) = 8
Z
1
= 38 (mod 16) binary (38) = 1 0 0 1 10 = 6
+
Integer overflow
Z
2
= 33 (mod 16) binary (33) = 1 0 0 0 0 1 = 1
Z
3
= 8 (mod 16) binary (8) = 0 0 1 0 0 0 = 8
No need to perform division!

Example
Using the multiplicative congruential method, find
the period of the generator for a = 13, m = 2
6
, and X
0

= 1, 2, 3, and 4. The solution is given in next slide.
When the seed is 1 and 3, the sequence has period 16.
However, a period of length eight is achieved when
the seed is 2 and a period of length four occurs when
the seed is 4.
Random Numbers 26
Solution
Period Determination Using Various seeds

i X
i
X
i
X
i
X
i



0 1 2 3 4

1 13 26 39 52

2 41 18 59 36

3 21 42 63 20

4 17 34 51 4

5 29 58 23

6 57 50 43

7 37 10 47

8 33 2 35

9 45 7
10 9 27

11 53 31

12 49 19

13 61 55

14 25 11

15 5 15

16 1 3
27
Selection of LCG Parameters for
Maximum Period
If c 0, the maximum possible period m
is obtained if and only if:
Integers m and c are relatively prime, that is, have no
common factors other than 1
Every prime number that is a factor of m is also a
factor of (a-1)
If integer m is a multiple of 4, a-1 should be a
multiple of 4
Notice that all of these conditions are met if m=2
b
,
a = 4k + 1, and c is odd.
Selection of LCG Parameters for
Maximum Period
For m a power of 2, say m = 2
b
, and c = 0, the
longest possible period is P = m / 4 = 2
b-2
, which is
achieved provided that the seed X
0
is odd and the
multiplier, a, is given by a = 3 + 8k or a = 5 + 8k, for
some k = 0, 1,...
29
Multiplicative LCG with m=2
k
m = 2
b

Maximum period achieved if:
multiplier a is of the form 8k3
the initial seed is an odd integer
Maximum possible period 2
b-2
the maximum possible period = One-fourth m,
may not be too small
30
Example

Using a seed of x
0
=1
5, 25, 29, 17, 21, 9, 13, 1, 5,
Period = 8 = 32/4
With x
0
=2, the sequence is
10, 18, 26, 2, 10,
Here, the period is only 4

31
Example
Multiplier not of the form 8i 3:


Using a seed of x
0
=1, we get the sequence
7, 17, 23, 1, 7,
The period is only 4
Selection of LCG Parameters for
Maximum Period
For m a prime number and c = 0, the longest possible
period is P = m - 1, which is achieved provided that
the multiplier, a, has the property that the smallest
integer k such that a
k
- 1 is divisible by m is k = m - 1
33
Multiplicative LCG with m=2
k
Modulus m = prime number
With a proper multiplier a, period = m-1
Maximum possible period = m
If and only if the multiplier a is a primitive root of the modulus m
a is a primitive root of m if and only if a
n
mod m =1
for n = 1, 2, , m-2
34
Example

Starting with a seed of x
0
=1
1, 3, 9, 27, 19, 26, 16, 17, 20, 29, 25, 13, 8, 24, 10, 30, 28, 22, 4, 12, 5,
15, 14, 11, 2, 6, 18, 23, 7, 21, 1,
The period is 30
3 is a primitive root of 31
With a multiplier of a = 5
1, 5, 25, 1,
The period is only 3 5 is not a primitive root of 31


Primitive roots of 31= 3, 11, 12, 13, 17, 21, 22, and 24
35
Full period generators
A generator that has the maximum possible period, m,
is called a full-period generator
Examples:
Combined Linear Congruential
Generators,
LEcuyer[1988]
Reason: Longer period generator is needed
because of the increasing complexity of
stimulated systems.
Approach: Combine two or more
multiplicative congruential generators.
Combined Linear Congruential
Generators
Suggested form:






Combined Linear Congruential
Generators
The maximum possible period is:

Example
Example: For 32-bit computers, LEcuyer [1988]
suggested combining k = 2 generators:
m
1
= 2,147,483,563, a
1
= 40,014
m
2
= 2,147,483,399, a
2
= 40,692.
The algorithm
Step 1: Select seeds
X
1,0
in the range [1, 2147483562] for the 1
st
generator
X
2,0
in the range [1, 2147483398] for the 2
nd
generator.
Step 2: For each individual generator:
X
1,j+1
= 40,014X
1,j
mod 2,147,483,563
X
2,j+1
= 40,692X
1,j
mod 2,147,483,399
Step 3: X
j+1
= (X
1,j+1
- X
2,j+1
) mod 2,147,483,562.
Step 4:Return

Example
The algorithm (contd)
Step 4:Return





Step 5: Set j = j+1, go back to step 2
Combined generator has period: (m11)(m21)/2 ~
2x10
18


Random-Numbers Streams
The seed for a linear congruential random-number
generator:
Is the integer value X0 that initializes the random-
number sequence.
Any value in the sequence can be used to seedthe
generator.
A random-number stream:
Refers to a starting seed taken from the sequence X0,
X1, , XP.
If the streams are b values apart, then stream i could
defined by starting seed:
Random-Numbers Streams
A single random-number generator with k
streams can act like k distinct virtual random-
number generators
To compare two or more alternative systems it
is advantageous to dedicate portions of the
pseudo-random number sequence to the same
purpose in each of the simulated systems.