Using Transformations

Generating from a Normal distribution (Box-M uller Method): We need two

sequences of standard uniform variables. Let U
1
and U
2
be two independent
standard uniform variables.
Dene
Z
1
:= [2 ln U
1
]
1/2
cos(2U
2
)
Z
2
:= [2 ln U
1
]
1/2
sin(2U
2
)
Then both Z
1
and Z
2
have a standard normal distribution and are
independent.
To get X
i
N(,
2
) transform using X
i
:= + Z
i
, i = 1, 2
To get a sequence of n random numbers from a Normal N(,
2
)
distribution, we use a U(0, 1) sequence u
i
, i = 1, . . . , n, convert them
to z
i
, i = 1, . . . , n and use them to compute x
i
, i = 1, . . . , n.
1
Inverse CDF Method for Continuous Densities
Main Idea: Consider generating a random variable X from a distribution
with cdf F and suppose that F is continuous and strictly increasing and
F
1
(u) is well-dened for 0 u 1.
If U is a random variable from U(0, 1), then it can be shown that
X = F
1
(U) has distribution F.
x
F
X
x
o
2
Applications of Inverse CDF Method
This result can be used to obtain closed form transformations only if F can
be inverted analytically i.e., the equation u = F(x) can be solved for x in
closed form.
However, if F can be accurately and eciently inverted numerically, this
method can be very useful.
For e.g., numerical inversion of the Standard Normal cdf is used as the basis
for several functions available in libraries for generating from the Standard
Normal distribution.
Generating from Exp()
The cdf of Exp() is Exp

(x) = 1 e
x
for x 0
To nd the inverse, set u = Exp

(x) where u be a positive real number:

3
Generating from Exp() (contd)
Set u = 1 e
x
1 u = e
x
ln(1 u) = x
x =
1

ln(1 u) =: F
1
X
(u)
So to sample X from Exp() distribution, generate u from U(0, 1) and
plug it in the above transformation.
Algorithm:
1. Generate u from U(0, 1).
2. Set x = log(1 u).
3. Set X = x.
In fact, since 1 U is uniform, if U is uniform, we could also have used
X :=
1

ln U
4
Generating from Erlang distribution
The Erlang distribution with parameters k and was discussed in the
previous chapter.
By denition, an Erlang random variable is the sum of k independent
Exp() random variables.
is called the rate parameter and k is called the stage parameter
When = 1, the distribution is called the Standard Gamma.
A Standard Gamma variable Y can be easily converted to a general Gamma
variable X by the transformation X =
1

Y .
Thus Erlang(k, ) variable is easily generated by summing k independent
Exponential variates each with parameter = 1 and multiplying the sum
by
1

.
5
Generating from Erlang distribution (contd)
This resiults in the following algorithm for generating X Erlang(k, )
variates:
1. Generate u
1
, u
2
, . . . , u
k
independent U(0, 1) variates.
2. Set y
i
= log(1 u
i
) for i = 1, 2, . . . , k.
3. Set s =

i=1
y
i
and x =
1

s.
4. Set X = x.
Algorithms for generating random variables by using transformations
obtained from denition such as above and Box-M uller for Normal random
variables are not ecient methods.
More ecient and fast algorithms can be obtained bt specialized techniques.
We shall discuss one such method next.
6
Acceptance/Rejection Method
We want to generate a random variate from a distribution with pdf f.
Let g be another pdf dened in the support of f such that
f(x) c g(x),
holds for all x in the support and c > 1 is a known constant.
Suppose that a method to generate random variates from g is available.
Rejection Algorithm:
1. Generate y from g.
2. Generate u from U(0, 1)
3. If u f(y)/c g(y) set X = y
else return to Step 1.
It can be shown that X is a random variate from the distribution with
density f.
7
Acceptance/Rejection Method (contd)
The function q(x) = c g(x) is called an envelope or a majorizing function.
In many applications, it is easier to rst nd a q(x) and then obtain g by
nding the normalizing constant.
The closer the enveloping function to f, the better the acceptance rate.
For this method to be ecient, the constant c must be selected so that the
rejection rate is low.
It is easy to show that the expected number of trials for an acceptance is c
Thus the smaller c is, the lower the rejection rate, beeter the algorithm.
A method to choose an optimum c is by setting c = max
x
f(x)/g(x)
8
Acceptance/Rejection Method: Example
Example: Generate a random variable from a distribution with pdf f
x
(x) =
3x
2
, 0 < x < 1 (First check: is f a density?)
Let us take g to be g(x) = 1 for 0 x 1; that is the desity of U(0, 1).
Thus
max
x
f(x)
g(x)
= 3 = c
So well use
f(x)
cg(x)
= x
2
in the above algorithm
Algorithm:
1) Generate two uniform random variables u
1
and u
2
from U(0, 1).
2) If u
2
u
2
1
accept u
1
as the random variable from f
else go to step 1).
By looking at the shapes of f and g we can guess that this is not a terribly
ecent algorithm.
9
Acceptance/Rejection Method: Example (contd)
Let us try to nd a better majorizing function for f
0.0 0.2 0.4 0.6 0.8 1.0
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
2
.
5
3
.
0
x
d
e
n
s
i
t
y

f
Instead of using q(x) = 3 (the blue line), use q(x) = 3x (the magenta line)
10
Acceptance/Rejection Method: Example (contd)
To nd g we need to nd the normalizing constant k such that k

1
0
3xdx = 1
Easy to show k = 2/3 so that g(x) = 2x for 0 x 1 and
max
x
f(x)
g(x)
= 3/2 = c
This majorizing function reduces the rejection rate by 50%
To implement the rejection method we need to generate from g
Note that the cdf of is easy to derive. (Show F(t) = t
2
); thus to generate
from this distribution using the inverse method is simple: Generate u from
U(0, 1) and set X = +

u.
New Algorithm:
1) Generate two uniform random variables u
1
and u
2
from U(0, 1).
2) If u
2

u
1
2
accept u
1
as the random variable from f
else go to step 1).
11