Moment Generating Functions

Moment Generating Functions (I)

Considerhe
t seriesexpansion
of ex :

xi
x2 x3
e 1 x ...
2 6
i0 i!
Notethatby taking
derivative
s withrespect tox, weget:
x

dex
2x 3x2
x2
0 1
... 1 x ... ex
dx
2! 3!
2!
d 2ex
2x
0 1 ...
2
dx
2!
Now,Replacing
x withtY, weget:

i
2
3
(
tY
)
(
tY
)
(
tY
)
etY
1 tY

...
i!
2
6
i0

t 2Y 2 t 3Y 3
1 tY

...
2
6

Moment Generating Functions (II)

Taking
derivative
s with
respect
tand
evaluating
to
at
t0:
tY
2
2 3
de
2
tY
3
t2Y3
2 tY
0
Y

... YtY

... Y00
...
Y
dtt0
2
!
3
!
2
!
t
0
t
0

d2etY
2
3
2
2

tY

...

...

Y
2
t
0
dt
t
0
Taking
the
expected
value
of
etY,and
labelling
function
as
M
(t):
i
ty

M
(t)Ee
ep
(y)
(y)
p
!
all
y
all
y
i
0 i

tY

ty

(k)

M
'(t)t0E
(Y
), M
''(t)t0EY2, ... M
(t) EYK
t
0

M(t) is called the moment-generating function for Y, and cam be used to

derive any non-central moments of the random variable (assuming it exists in
a neighborhood around t=0).
Also, useful in determining the distributions of functions of random variables

Binomial Distribution MGF & PGF

M(t) E e

tY

n y

n y

e p (1 p)
y0
y

ty

n
n
t y
n y
t

pe
(
1

p
)

pe

(
1

p
)

y0 y

M' (t) n pet (1 p)

n1

M' ' (t) np (n 1) pet (1 p)

n2

n1

n1 t

pet et pet (1 p)

E(Y) M' (0) np p(1) (1 p)

pet np pet (1 p)
(1) np

E Y 2 M' ' (0) np (n 1) p(1) (1 p)

n2

n1

np(n 1) p 1 n2 p2 np2 np n2 p2 np(1 p)

V (Y) E Y 2 E(Y) n2 p2 np(1 p) (np)2 np(1 p)

2

np(1 p)

P(t) E t

y0

n y

n y

p
(
1

p
)

n
y
n
n y

pt
(
1

p
)

pt

(
1

p
)

y0 y
n

p(1) (1) p(1) (1 p) [1]

n1

Geometric Distribution MGF & PGF

p
p
tY
ty y1
ty y
t y
M(t)Ee
e q p
e q qe
q y1
q y1
y1
t

pqe

qe

q y1

t y1

pe
pe

t
1qe 1(1p)et

p
p
tqy
P(t)EtY
tyqy1p
tyqy
q y1
q y1
y1

ptq
pt
pt
y1
tq

q y1
1tq 1(1p)t

Poisson Distribution MGF & PGF

M(t) E e

tY

y0

et

ty

e
e e
e

y!
y0
y! y0

t y

y!

e e

t y

et 1

e
e t
P(t) E t t

y!
y0
y! y0

y0

y!

eet e(t1)

Exponential Distribution - MGF

1 yt
tY
ty 1 y
M(t) E e e e dy e
dy
0
0

1t

where
1t
1 1 y *
*
*
1
1
M(t)
e

(
0

1
)

(
1

t
)

0
1*

1t

dy

y *

dy

M'(t) 1(1t)2() (1t)2

M''(t) 2(1t)3() 22(1t)3
E(Y) M'(0)
V(Y) M''(0)M'(0) 22 2 2
2

Gamma Distribution - MGF

M (t ) E e

tY

1
1 y
dy
ety
y
e

( )

1
y t
1

1t
y

1
1
y
e
dy

y e
dy
0
0
( )
( )

1 y *
*

y
e
dy
where

( ) 0
1 t
1
*

M (t )

(
1

t
)
( )

M ' (t ) (1 t ) 1 ( ) (1 t ) 1
M ' ' (t ) ( 1)(1 t ) 2 ( ) ( 1) 2 (1 t ) 2
E(Y ) M ' (0)
V (Y ) M ' ' (0) M ' (0) ( 1) 2 ()2 2
2

Normal Distribution - MGF

2
2
1
1 y2

1
y
y

dy
M(t) E e e
exp

exp

ty

dy
2
2
2
2

2
22

2
2 2

y2 y(t2) 2
1

exp
2 dy
2
2
2
2

2
2

tY

ty

Completing
thesquare
: (t2)2 2 2t2 t2

2
2

2t2 t2
y2 y(t2) 2 2t2 t2

M(t)
exp

dy
2
2
2
2
2
2
2

2
2

1
2t2 t2
y2 y(t2) 2 2t2 t2

exp
2

dy
2
2
2

2
2

2 2
2
2 2

1
y

)
2

exp

dy
2
2

2
2 2
2 2

)
2t t
1
1

exp
exp

dy
2
2

Thelastintegral
being
1,since
itisintegratin
gover the
density
ofa normal
R.V.
: Y ~ N t2,2

2
2 2

t22
2t t

M(t) exp

exp
t

2
2

Distribution of Z2 (Z~N(0,1))
Z ~N(0,1) fZ(z)

1 z2/2
e
z
2
212t

212t

1
1 z2/2
1 z 2
MZ2 (t) e
e dz
e
dz2
e 2 dz (symmetric
about
0)

0
2
2
2
dz 1
Letuz2 z u

0.5u1/2 dz0.5u1/2du
du 2 u

tz2

212t

1 z
e
2

1
2(12t)1/2 (12t)1/2
2
Z2 ~Gamma
(1/ 2, 2) 12

Notes
:

1/2

1 1/2 u/12t
1 1/21 u/12t
1
2
dz
u
e
du

u
e
du

(
1
/
2
)

2 0
2 0
2
12t

y1ey/dy()

(1/ 2)
n

Z1,...,
Zn mutually
independen
t Zi2 ~Gamma
(n/ 2, 2) n2
i1