Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Analisis
Bayesiano de Series Temporales
Definitions
Analisis
Bayesiano de Series Temporales
BASIC DEFINITIONS AND INFERENCE
Raquel Prado
Universidad de California, Santa Cruz
Julio, 2013
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Definitions
Definitions
100
200
-400
-300
-200
-100
1000
2000
3000
time
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Definitions
Definitions
France
1950
1960
1970
1980
10
0.04
20
0.08
30
Austria
1950
1960
1980
1950
1960
Greece
1970
1980
Example: TAR(1)
Italy
yt =
20
1.0
40
2.0
60
10 14 18
80
Germany
1970
1950
1960
1970
1980
1950
1960
1980
1950
1960
UK
1970
1980
(1)
where t
USA
(1)
N(0, v2 ).
N(0, v1 ) and t
(M1 )
(M2 ),
1.0
20
1.4
30
1.8
Sweden
1970
1950
1960
1970
1980
1950
1960
1970
1980
1950
1960
1970
1980
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Definitions
Definitions
500
1000
1500
time
(a)
1.8
1.0
1.4
I
0
500
1000
1500
time
(b)
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Definitions
Definitions
Stationarity
Other goals
Clustering
Stationarity
Definition
{yt , t T } is completely or strongly stationary if, for any
sequence of times t1 , . . . , tn and any lag h, the distribution of
(yt1 , . . . , ytn )0 is identical to the distribution of (yt1 +h , . . . , ytn +h )0 .
Definition
{yt , t T } is weakly or second order stationary if for any
sequence t1 , . . . , tn , and any h, the first and second joint
moments of (yt1 , . . . , ytn )0 and those of (yt1 +h , . . . , ytn +h )0 exist
and are identical.
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Definitions
Definitions
Stationarity
Stationarity
Stationarity
Stationarity
-300
0
500
1000
1500
2000
200
100
0
-300
-200
-100
100
200
time
-100
-200
-300
-200
-100
100
200
50
100
time
150
200
50
100
time
150
200
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Definitions
Definitions
The ACF
The ACF
t=1
where y =
PT
t=1 yt /T
Definition
The sample autocorrelation is given by (h) =
(h)
(0) .
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Definitions
Definitions
The ACF
The ACF
ACF of AR(1)
0.5
v
,
(1 2 )
= 0.9
= 0.9
= 0.3
0.5
0.0
1.0
1.0
10
20
30
h
40
50
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Definitions
The ACF
= 0.3
1.0
likelihood
0.8
p(|y1:T ) =
ACF
0.4
ACF
p(y1:T ) =
0.2
0.0
0.4
0.0
10
15
20
Lag
(a)
10
Lag
(b)
15
20
p(y1:T |)p()d.
1.0
0.0
0.5
0.2
ACF
with
prior
z }| { z}|{
p(y1:T |) p()
,
p(y1:T )
| {z }
predictive
0.6
0.6
0.5
0.8
1.0
1.0
= 0.9
10
15
20
likelihood
Lag
(c)
Analisis
Bayesiano de Series Temporales
predictive
Analisis
Bayesiano de Series Temporales
Example
AR(1): yt = yt1 + t , t N(0, v ), and so = (, v ).
I
I
T
Y
p(|y1:T ) p() p(y1 |)
p(yt |yt1 , ) .
|{z}
t=2
prior |
{z
}
Then,
p
(1 2 )
Q ()
exp
p(|y1:T ) p()
,
2v
(2v )T /2
likelihood
with
Q () =
y12 (1
T
X
)+
(yt yt1 )2
2
|t=2
{z
Q()
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Estimation
I
Example
AR(1) (cont.): We can also use the conditional likelihood
p(y2:T |, y1 ) as an approximation to the full likelihood and
obtain the posterior
Q()
(T 1)/2
p(|y1:T ) p()v
exp
2v
I
I
= MLE that
Maximum likelihood estimation (MLE): Find
maximizes p(y1:T |).
= MAP
Maximum a posteriori estimation (MAP): Find
that maximizes p(|y1:T ).
Least squares estimation (LSE): Write the model as
y = F0 + , N(0, v I).
with dim(y) = n and dim() = p so that
p(y|F0 , , v ) = (2v )n/2 exp(Q(y, )/2v ) ,
that minimizes Q(y, ).
and find
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Bayesian Estimation
Bayesian Estimation
n/2
exp(Q(y, )/2v )
where
0 (FF0 )( )
+ R,
Q(, y) = (y F0 )0 (y F0 ) = ( )
= (FF0 )1 Fy and R = (y F0 )
0 (y F0 ).
with
I The MLE of is ;
I
I
I
n
on/2
0 FF0 ( )/(n
p)s2
p(|y, F) |FF0 |1/2 1 + ( )
s2 (FF0 )1 ).
When n p(|y, F) N(|,
(np)s2
p(v |y) = IG (np)
.
2 ,
2
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Bayesian Estimation
Conjugate Prior:
e
I
I
2v
m = m0 + C0 F[F0 C0 F + In ]1 (y F0 m0 )
C = C0 C0 F[F0 C0 F + In ]1 F0 C0 ,
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
80
100
120
140
Example
ML, MAP, and LS estimators for the AR(1) model.
yt = yt1 + t , with t N(0, 1). In this case = .
60
Estimation
0.6
0.7
0.8
0.9
1.0
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
40
60
80
120
140
140
120
100
80
100
60
40
0.6
0.7
0.8
0.9
1.0
0.6
0.7
0.8
0.9
1.0
I
I
Analisis
Bayesiano de Series Temporales
Pn
t=2 yt1 yt /
Pn1
t=1
yt2 .
Analisis
Bayesiano de Series Temporales
AR(1) with full likelihood: The prior p(, v ) 1/v does not
lead to a closed form posterior distribution when the full
likelihood is used. We obtain
Q ()
(n/2+1)
2 1/2
p(, v |y1:n ) v
(1 ) exp
.
2v
1200
1000
Frequency
600
800
1000
800
400
200
Frequency
600
400
200
0
0.84
0.88
0.92
(a)
0.96
90 100
120
v
(b)
140
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
p(
(m1)
|y1:n )/J(
(m1)
| )
(c) Set
(m)
(m1)
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
AR(p) Models
2.0
v
0.0
0.2
0.0
0
200
400
600
800
1000
200
600
800
1000
80
60
20
(c)
0.94
T
Y
t=p+1
p(yt |y(tp):(t1) ) =
T
Y
t=p+1
0
0.90
j ytj + t ,
40
Frequency
60
40
20
0
Frequency
400
iteration
(b)
80
iteration
(a)
0.86
p
X
j=1
0.5
0.4
yt =
1.0
0.6
1.5
0.8
1.0
1.00
1.05
v
(d)
1.10
1.15
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Autoregressions
Autoregressions
Definition
An AR(p) process yt is causal if it can be written as
yt = (B)t =
j tj ,
j=0
with
P B the backshift operator Bt = t1 , 0 = 1 and
j=0 |j | < .
Definition
The AR characteristic polynomial is defined as:
(u) = 1
p
X
yt is causal only when (u) has all its roots outside the unit
circle (or the reciprocal roots inside the unit circle). In other
words, yt is causal if (u) = 0 only when |u| > 1.
Causality Stationarity.
j u j .
j=1
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Autoregressions
Autoregressions
= F0 xt
xt
= Gxt1 + t ,
1 2 3 p1 p
1 0 0
0
0
0 1 0
0
0
G=
.
..
..
..
.
. 0
.
0 0 1
0
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Autoregressions
Autoregressions
AR Models: ACF
The autocorrelation structure of an AR(p) process is given in
terms of the solution of the homogeneous difference equation
(h) 1 (h 1) . . . p (h p) = 0, h p.
If 1 , . . . , r are the reciprocalPcharacteristic roots with
multiplicities m1 , . . . , mp and pi=1 mi = p, the general solution
of the equation is
h
(h) = 1h p1 (h) + . . . + m
pm (h), h p,
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Autoregressions
Autoregressions
1 = 2 = real.
ft (h) = p(h)h ,
with p(h) polynomial of degree one, i.e., p(h) = d + eh.
I
1 , 2 complex conjugates.
h
1 = 2 = real.
(h) = (a + bh)h , h 2
1 , 2 complex conjugates.
(h) = ar h cos(h + b), h 2.
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Autoregressions
Autoregressions
AR Models: PACF
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Autoregressions
Autoregressions
p(y|, v , y1:p ) =
p
=
1
p , v = (0)
0p
p.
p
It can be shown that
that maximizes
MLE. Find
t=p+1
=
) N(0, v 1
T (
p ),
T
Y
T
Y
t=p+1
I
p(yt |, v , y(tp):(t1) )
N(yt |f0t , v ) = N(y|F0 , v In ).
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Autoregressions
Autoregressions
200
Future sample
Forecast function
-100
-200
-300
0
100
200
300
(0.97, 12.73);
(0.81, 5.10);
(0.72, 2.99);
(0.66, 2.23).
voltage (mcv)
200
200
100
100
voltage (mcv)
100
voltage (mcv)
0
-100
0
-100
-200
-200
-300
-300
400
time
100
200
300
400
500
600
100
time
Analisis
Bayesiano de Series Temporales
Autoregressions
Autoregressions
time
-10
-20
log-likelihood
log(n)p + n log(sp2 ).
-30
-40
a a
b m
b m m
m
a ab b m a a a
m
a a
a
a a
b
m
a a
a a a a a
m
b
m
b
b
m
m
m
b
m
a
b
m
m m
b
b
m m m m
b
m
b
b
b a
b
m
b
b
b b
b
a
Marginal:
Here n = T p .
400
2p + n log(sp2 ).
I
300
Analisis
Bayesiano de Series Temporales
200
-50
b
m
-60
a
a
5
10
15
AR order p
20
25
500
600
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
Autoregressions
Autoregressions
It can be shown (e.g., see Box, Jenkins, and Reinsel, 2008) that
Q(y1:T , ) = a 2b0 + 0 C,
with a, b, and C obtained from
a b0
D=
,
b C
and D a (p + 1) (p + 1) with Dij =
I
where
Q(y1:T , ) =
T
X
t=p+1
Analisis
Bayesiano de Series Temporales
yi+r yj+r .
Analisis
Bayesiano de Series Temporales
Autoregressions
r =0
PT +1ji
Then...
I
I
h(j ),
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
ARMA Models
ARMA Models
ARMA Models
ARMA Models
p
X
i yti +
i=1
q
X
j tj + t ,
j=1
= (1 + 1 B + . . . + q B q ) t
|
{z
}
(B)
Analisis
Bayesiano de Series Temporales
Definition
A MA(q) process is identifiable or invertible if the roots of the
MA characteristic polynomial (u) lie outside the unit circle. In
this case is possible to write the process as an infinite order AR.
Example
Let yt MA(1) with MA coefficient . The process is stationary
for all and
h=0
1
h=1
(h) =
2
(1+ )
0
otherwise.
Note that a MA process with coefficient 1/ has the same ACF
the identifiability condition is 1/ > 1.
Analisis
Bayesiano de Series Temporales
ARMA Models
ARMA Models
yt =
(B)(B)t = (B)t =
j tj ,
j=0
p
X
h=1
h jh = 0, j max(p, q + 1),
j
X
h=1
ARMA Models
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
ARMA Models
ARMA Models
Pqh
j=0 j j+h
Pq
(h) =
1+ j=1 j2
h=0
h=1:q
h > q,
p
X
j=1
j (h j) = v
q
X
j=h
j jh ,
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
ARMA Models
ARMA Models
i=r +1
P
j
with (u) = 1 +
j=1 j u , such that
Q
p
1 = (u) i=r +1 (1 i u).
yt
r
X
j ytj + t +
j=1
yt
where
(u)
Qr
r
X
Algorithm
1. Initialize the algorithm by setting i = 0 for all i = 1 : q.
2. For i = (r + 1) : p, update 1 = 1 + i , and then,
I
j tj ,
for j = 2 : q, update j = j + i j1
.
j=1
j ytj + t +
j=1
i=1 (1
i u) = 0.
q
X
j=1
j tj ,
yt 1 yt1 + 2 yt2 + t +
q
X
j tj ,
j=1
Analisis
Bayesiano de Series Temporales
Analisis
Bayesiano de Series Temporales
ARMA Models
ARMA Models
1.5
coefficient
1.0
0.0
-0.5
MA coefficient index
Note: the optimal ARMA(p, q) model for these data, among all
the models with p, q 8, is an ARMA(2, 2). The MLEs for the
MA coefficients are 1 = 1.37 and 2 = 0.51.
Analisis
Bayesiano de Series Temporales
Time Domain Models
ARMA Models
ARIMA yt ARIMA(p, d, q)
(1 1 B . . . p B p )(1 B)d yt = (1 + 1 B + . . . + q B q )t .
SARMA
(1 1 B s . . . p B ps )yt = (1 + 1 B s + . . . + q B qs )t .
0.5