Sei sulla pagina 1di 4

Theory of Statistics xx-xx-xxxx

Notes 1: Basics of Monte Carlo Approximation


Professor: Ernest Fokoué xx-xx-xxxx

Monte Carlo Approximation of Integrals


Let f (x) be the density function of a random variable X with nonzero support on some interval
[a, b]. Assume that realizations of X can be drawn independently from the interval [a, b] using
the cdf F (x) of X. Then the integral
∫ b
I= g(x)f (x)dx
a

can be approximated by
1∑
n
I˜ = g(xi ),
n i=1
where x1 , · · · , xn are drawn independently according to the density f (x) in the interval [a, b].
The above formula is generic and very useful. However, in practice we need to compute integrals
given in the form ∫ b
I= g(x)dx
a
In other words, the density from which to draw the empirical observations for approximation
is missing. One way around it is reformulate the integral with the weight function from which
realizations can be drawn. That’s called the importance function.
∫ b ∫ b( )
g(x)
I= g(x)dx = w(x)dx,
a a w(x)

which is then approximated using


( )
1∑
n
g(xi )
I˜ = ,
n i=1 w(xi )

where x1 , · · · , xn are drawn independently according to the density w(x) in the interval [a, b].
Let X be a continuous random variable with pdf fX (x). Then,
∫ b ∫ b ( ) ∫ b
1
I = Pr[a ≤ X ≤ b] = fX (x)dx = (b − a) fX (x) dx = (b − a) fX (x)u(x)dx
a a b−a a

can be approximated by Monte Carlo, using


( ) n
˜ b−a ∑
I= fX (xi )
n i=1

where where x1 , · · · , xn are drawn independently from a uniform distribution in the interval
iid
[a, b] or more succinctly, Xi ∼ Unif[a, b].

1-1
If one can sample directly according to fX (x), then one can use the indicator function IA (x)
to compute the probability as an expectation. Recall, that
{
1 if x ∈ A
IA (x) =
0 if x ∈ /A
Now, the probability of an event can be reformulated as an expected value as follows,
∫ b ∫
I = Pr[a ≤ X ≤ b] = fX (x)dx = I[a,b] (x)fX (x)dx = E[I[a,b] (X)],
a RX

which can then approximated by


1∑
n
I˜ = I[a,b] (xi ) = Proportion of samples in [a,b],
n i=1

where x1 , · · · , xn are drawn independently according to the cdf FX (x) over the whole range of
the random variable X.

Example: Let X ∼ N (µ = 9, σ 2 = 4). Compute Pr[7 ≤ X ≤ 11].


∫ 2
1
√ e− 2 z dz
1 2
Pr[7 ≤ X ≤ 11] = Φ(2) − Φ(−2) =
−2 2π
1. Probability as expectation
∫ 2
1∑
n
1 − 1 z2
√ e 2 dz = E[I[−2,2] (Z)] ≈ I[−2,2] (zi )
−2 2π n i=1

where the z1 , · · · , zn are drawn independently from the standard normal distribution.

n <- 10000
z <- rnorm(n)
proba <- length(which(abs(z)<2))/n
print(proba)

2. Probability using the uniform density as the weighting/importance function


∫ 2
4 ∑ 1 − 1 z2i
n
1 − 1 z2
√ e 2 dz ≈ √ e 2
−2 2π n i=1

where the z1 , · · · , zn are drawn independently from the uniform distribution in [−2, 2].

n <- 10000
u <- runif(n,-2,2)
proba <- (4/n)*sum(dnorm(u))
print(proba)

3. Theoretical value through R


Pr[7 ≤ X ≤ 11] = Φ(2) − Φ(−2) = pnorm(2) − pnorm(−2) = 0.9545

Exercise: Compute Pr[ 21 ≤ X ≤ 34 ] where X ∼ FX (x) with fX (x) = 3x2 in [0, 1].

1-2
Monte Carlo Approximations in High Dimensions
Consider now two random variables X1 and X2 with joint density function f (x1 , x2 ) defined on
X1 × X2 . We now show how to compute probabilities in such a context.
∫ a12 ∫ a12
I = Pr[a11 ≤ X1 ≤ a12 ∩ a21 ≤ X2 ≤ a22 ] = f (x1 , x2 )dx1 dx2
a11 a11

The so-called Multivariate Uniform distribution is defined in 2-dimension as


{ 1
(a12 −a11 )(a22 −a21 )
if a11 ≤ x1 ≤ a12 ∩ a21 ≤ x2 ≤ a22
u(x1 , x2 ) =
0 otherwise
Using the Multivariate Uniform Density as an importance function, a Monte Carlo approxima-
tion of the above joint probability is

I = Pr[a11 ≤ X1 ≤ a12 ∩ a21 ≤ X2 ≤ a22 ]


∫ a12 ∫ a12
= f (x1 , x2 )dx1 dx2
a11 a11
∫ a12 ∫ a12 ( )
1
= (a12 − a11 )(a22 − a21 ) f (x1 , x2 ) dx1 dx2
a11 a11 (a12 − a11 )(a22 − a21 )
(a12 − a11 )(a22 − a21 ) ∑
n
≈ f (xi1 , xi2 )
n i=1

where the sample {(xi1 , xi2 ), i = 1, · · · , n} is drawn iid from the multivariate uniform distribu-
tion in [a11 , a12 ] × [a21 , a22 ], namely (Xi1 , Xi2 ) ∼ MVUnif([a11 , a12 ] × [a21 , a22 ]).
Now, if X1 ∈ [a, b] and X1 ∈ [c, d], then one could also approximation I using

I = Pr[a11 ≤ X1 ≤ a12 ∩ a21 ≤ X2 ≤ a22 ]


∫ a12 ∫ a12
= f (x1 , x2 )dx1 dx2
a11 a11
∫ b∫ d ( )
1
= (b − a)(d − c) Ia11 ≤x1 ≤a12 ∩a21 ≤x2 ≤a22 f (x1 , x2 ) dx1 dx2
a c (b − a)(d − c)
(b − a)(d − c) ∑
n
≈ Ia11 ≤x1 ≤a12 ∩a21 ≤x2 ≤a22 f (xi1 , xi2 )
n i=1

where the sample {(xi1 , xi2 ), i = 1, · · · , n} is drawn iid from the multivariate uniform distribu-
tion in [a, b] × [c, d], namely (Xi1 , Xi2 ) ∼ MVUnif([a, b] × [c, d]).

Example: X1 and X2 are defined on [0, 1] × [0, 1] with density f (x1 , x2 ) = 4x1 x2 . Compute

I = Pr[1/3 ≤ X1 ≤ 1/2 ∩ 1/2 ≤ X2 ≤ 3/4]

using both theory and Monte Carlo. Clearly

( 1 − 1 )( 3 − 1 ) ∑
n
I˜MC = 2 3 4 2 4xi1 xi2
n i=1

where Xi1 ∼ Unif( 13 , 12 ) and Xi2 ∼ Unif( 12 , 34 ).

1-3
Multivariate Monte Carlo in R

f <- function(x1,x2){return(4*x1*x2)}

n <- 100000
a11 <- 1/3; a12 <- 1/2; a21 <- 1/2; a22<-3/4
x1 <- runif(n, a11,a12)
x2 <- runif(n, a21,a22)

# Monte Carlo
probMC <- ((a12-a11)*(a22-a21)/n)*sum(f(x1,x2))

# Monte Carlo: approach 2 -> Probability as expectation

x1 <- runif(n, 0,1)


x2 <- runif(n, 0,1)
fraction <- which(x1<a12 & x1>a11 & x2<a22 & x2>a21)
probMC.2 <- ((1-0)*(1-0)/n)*sum((f(x1,x2))[fraction])

print(probMC.2)

probTheo <- (a12^2-a11^2)*(a22^2-a21^2)

cat(’\n The theoretical value is’, probTheo,’\n’)


cat(’\n The Monte Carlo approx is’, probMC,’\n’)

1-4

Potrebbero piacerti anche