Time Series
http://avaxhome.ws/blogs/ChrisRedfield
SIAM's Classics in Applied Mathematics series consists of books that were previously allowed to go out of print. These books are republished by SIAM as a professional service because they continue to be important resources for mathematical scientists.
EditorinChief Robert E. O'Malley, Jr., University of Washington
Editorial Board Richard A. Brualdi, University of WisconsinMadison
Herbert B. Keller, California Institute of Technology
Andrzej Z. Manitius, George Mason University
Ingram Olkin, Stanford University
Stanley Richardson, University of Edinburgh
Ferdinand Verhulst, Mathematisch Instituut, University of Utrecht
Classics in Applied Mathematics
C. C. Lin and L. A. Segel, Mathematics Applied to Deterministic Problems in the Natural
Sciences
Johan G. F. Belinfante and Bernard Kolman, A Survey of Lie Groups and Lie Algebras with Applications and Computational Methods
James M. Ortega, Numerical Analysis: A Second Course
Anthony V. Fiacco and Garth P. McCormick, Nonlinear Programming: Sequential Unconstrained MinimizationTechniques
F. H. Clarke, Optimization and NonsmoothAnalysis
George F. Carrier and Carl E. Pearson, Ordinary Differential Leo Breiman, Probability
R. Bellman and G. M. Wing, An Introduction to Invariant Imbedding
Abraham Berman and Robert J. Plemmons, Nonnegative Matrices in the Mathematical Sciences
Olvi L. Mangasarian, Nonlinear Programming
Equations
*Carl Friedrich Gauss, Theory of the Combination of Observations Least Subject to Errors:
Part One, Part Two, Supplement. Translated by G. W. Stewart
Richard Bellman, Introduction to Matrix Analysis
U. M. Ascher, R. M. M. Mattheij, and R. D. Russell, Numerical Solution of Boundary Value
Problems for Ordinary Differential
K. E. Brenan, S. L. Campbell, and L. R. Petzold, Numerical Solution of InitialValue
Problems in DifferentialAlgebraic Equations
Charles L. Lawson and Richard J. Hanson, Solving Least Squares Problems
J. E. Dennis, Jr. and Robert B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations
Richard E. Barlow and Frank Proschan, Mathematical Theory of Reliability
Equations
*First time in print.
Classics in Applied Mathematics (continued)
Cornelius Lanczos, Linear Differential Operators
Richard Bellman, Introduction to Matrix Analysis, Second Edition
Beresford N. Parlett, The Symmetric Eigenvalue Problem
Richard Haberman, Mathematical Models: Mechanical Vibrations, Population Dynamics, and
Traffic
Peter W. M. John, Statistical Design and Analysis of Experiments
Tamer Basar and
Emanuel Parzen, StochasticProcesses
Petar Kokotovic,
Control: Analysis and Design
Jean Dickinson Gibbons, Ingram Olkin, and Milton Sobel, Selecting and Ordering Populations: A New Statistical Methodology
James A. Murdock, Perturbations: Theory and Methods
Ivar Ekeland and Roger Témam, Convex Analysis and Variational Problems
Ivar
J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables
David Kinderlehrer and Guido Stampacchia, An Introduction to Variational Inequalities and Their Applications
F. Natterer, The Mathematics of Computerized Tomography
Avinash C. Kak and Malcolm Slaney, Principles of Computerized Tomographic Imaging
Flow
Geert Jan Olsder, Dynamic Noncooperative Game Theory, Second Edition
Hassan K. Khalil, and John O'Reilly, Singular Perturbation Methods in
Stakgold, Boundary Value Problems of Mathematical Physics, Volumes I and II
R. 
Wong, Asympototic Approximations of Integral 
O. 
Axelsson and V. A. Barker, Finite Element Solution of Boundary Value Problems: Theory 
and Computation
David R. Brillinger, Time Series: Data Analysis and Theory
Time Series
Data Analysis and Theory
David R. Brillinger
University of California at Berkeley Berkeley, California
siam
Society for Industrial and Applied Mathematics Philadelphia
Copyright © 2001 by the Society for Industrial and Applied Mathematics.
This SIAM edition is an unabridged republication of the work first published by Holden Day, Inc., San Francisco, 1981.
1098765432 1
All rights reserved. Printed in the United States of America. No part of this book may be reproduced, stored, or transmitted in any manner without the written permission of the publisher. For information, write to the Society for Industrial and Applied Mathematics, 3600 University City Science Center, Philadelphia, PA 191042688.
Library of Congress CataloginginPublication
Data
Brillinger, David R. Time series: data analysis and theory / David R. Brillinger p. cm.  (Classics in applied mathematics ; 36) "This SIAM edition is an unabridged republication of the work first published by Holden Day, Inc., San Francisco, 1981"  T.p. verso. ISBN 0898715016 (pbk.)
1. Timeseries analysis. 2. Fourier transformations.
QA280 .B74 2001
519.5'5dc21
I. Title. II. Series
2001034170
Figure 1.1.3 reprinted with permission from E. W. Carpenter, "Explosion Seismology," Science, 147:363373, 22 January 1965. Copyright 1965 by the American Association for the Advancement of Science.
siamis a registered trademark.
CONTENTS
xi
6 Analysis of A Linear Time Invariant Relation Between A Stochastic Series and Several Deterministic Series
186
6.1 Introduction 186
6.2 Least Squares and Regression Theory 188
6.3 Heuristic Construction of Estimates 192
6.4 A Form of Asymptotic Distribution
6.5 Expected Values of Estimates of the Transfer Function and Error Spectrum 196
6.6 Asymptotic Covariances of the Proposed Estimates 200
6.7 Asymptotic Normality of the Estimates 203
6.8 Estimating the Impulse Response 204
6.9 Confidence Regions 206
6.10 A Worked Example 209
6.11 Further Considerations 219
6.12 A Comparison of Three Estimates of the Impulse Response 223
6.13 Uses of the Proposed Technique 225
6.14 Exercises 227
194
7 Estimating The SecondOrder Spectra of VectorValued Series
232
7.1 The Spectral Density Matrix and its Interpretation 232
7.2 SecondOrder Periodograms 235
7.3 Estimating the Spectral Density Matrix by Smoothing 242
7.4 Consistent Estimates of the Spectral Density Matrix 247
7.5 Construction of Confidence Limits 252
7.6 The Estimation of Related Parameters 254
7.7 Further Considerations in the Estimation of SecondOrder Spectra
260
7.8 A Worked Example 267
7.9 The Analysis of Series Collected in an Experimental Design 276
7.10 Exercises 279
8 Analysis of A Linear Time Invariant Relation Between Two VectorValued Stochastic Series
286
8.1 Introduction 286
8.2 Analogous Multivariate Results 287
8.3 Determination of an Optimum Linear Filter 295
8.4 Heuristic Interpretation of Parameters and Construction of Estimates
299
8.5 A Limiting Distribution for Estimates 304
8.6 A Class of Consistent Estimates 306
8.7
SecondOrder Asymptotic Moments of the Estimates 309
PREFACE TO THE CLASSICS EDITION
"One can FT anything—often
meaningfully."
—John W. Tukey
John Tukey made this remark after my book had been published, but it is surely the motif of the work of the book. In fact the preface of the original book states that
The reader will note that the various statistics presented are imme diate functions of the discrete Fourier transforms of the observed values of the timeseries. Perhaps this is whatcharacterizes thework of this book. The discrete Fourier transform is given such promi nencebecause it has important empirical and mathematical properties. Also, following the work of Cooley and Tukey (1965), it may be computed rapidly.
The book was finished in mid 1972. The field has moved on from its place then. Some of the areas of particular development include the following.
I. Limit theorems for empirical Fourier transforms. Many of the techniques based on the Fourier transform of a stretch of time series are founded on limit or approximation theorems. Examples may be found in Brillinger (1983). There have been developments to more abstract types of processes: see, for example, Brillinger (1982, 1991). One particular type of development concerns distributions with long tails; see Freedman and Lane (1981). Another type of extension concerns series that have socalled long memory. The large sample distribution of the Fourier transform values in this case is developed in Rosenblatt (1981), Yajima (1989), and Pham and Guegan
xiii
xiv
PREFACE TO THE CLASSICS EDITION
(1994).
II. Tapering.
The idea of introducing convergence factors into a Fourier approximation has
In the time series case, this is known as tapering.
Surprising
a long history.
properties continue to be found; see Dahlhaus (1985).
III. Finitedimensional parameter estimation.
Dzhaparidze (1986) develops in detail Whittle's method of Gaussian or approximate likelihood estimation. Brillinger (1985) generalizes this to the third order case in the tool of bispectral fitting. Terdik (1999) develops theoretical properties of that procedure.
IV. Computation.
Time series researchers were astonished in the early 1980s to learn that the fast Fourier transform algorithms had been anticipated many years earlier by K. F. Gauss. The story is told in Heideman et al. (1985). There have since been extensions to the cases of a prime number of observations (see Anderson and Dillon (1996)) and to the case of unequally spaced time points (see Nguyen and Liu
(1999)).
V. 
General methods and examples. A numberof applications to particular physical circumstances havebeen made 
of 
Fourier inference; see the paper by Brillinger (1999) and the book by Bloomfield 
(2000).
D. R. B.
Berkeley, California
December 2000
ANDERSON, C., and DILLON, M. (1996). "Rapid computationof the discrete Fourier transform." SIAM J. Sci, Comput. 17:913919.
BLOOMFIELD, P. (2000). Fourier Analysis of Time Series: An Introduction. Second Edition. New York: Wiley.
BRILLINGER, D. R. (1982). "Asymptotic normality of finite Fourier transforms of stationary generalized processes." J. Multivariate Anal. 12:6471.
BRILLINGER, D. R. (1983). "The finite Fourier transform of a stationary
process." In Time Series in the Frequency Domain,Handbook of Statist. 3, Eds. D.
R. Brillinger and P. R. Krishnaiah, pp. 2137. Amsterdam: Elsevier.
PREFACE TO THE CLASSICS EDITION
xv
BRILLINGER, D. R. (1985). "Fourier inference: some methods for the analysis of array and nongaussian series data." Water Resources Bulletin. 21:743756.
BRILLINGER, D. R. (1991). "Some asymptotics of finite Fourier transforms of a stationary padic process." J. Combin. Inform. System Sci. 16:155169.
BRILLINGER, D. R. (1999). "Some examples of empirical Fourier analysis in scientific problems." In Asymptotics, Nonparametrics and Time Series, Ed. S. Ghosh, pp. 136. New York: Dekker.
DAHLHAUS, R. (1985).
spectral functions." Stochastic Process. Appl. 19:135149.
"A functional limit theorem for tapered empirical
DZHAPARIDZE, K. (1986). Parameter Estimation and Hypothesis Testing in Spectral Analysis of Stationary Time Series. New York: Springer.
FREEDMAN, D., and LANE, D. (1981). "The empirical distribution of the
Fourier coefficients of a sequence of independent, identically distributed long
und
tailed random variables."
Zeitschrift fur
Wahrscheinlichkeitstheorie
Verwandte Gebiete. 58:2140.
HEIDEMAN, M. T., JOHNSON, D. H., and BURRUS, C. S. (1985). "Gauss and the history of the fast Fourier transform." Arch. Hist. Exact Sci. 34:265277.
NGUYEN, N., and LIU, Q. H. (1999). "The regular Fourier matrices and
nonuniform fast Fourier transforms."
SIAM. J. Sci. Comput. 21:283293.
PHAM, D. T., and GUEGAN, D. (1994). "Asymptotic normality of the discrete Fourier transform of long memory series." Statist. Probab. Lett. 21:299309.
ROSENBLATT, M. (1981). "Limit theorems for Fourier transforms of function
als of Gaussian sequences."
Wahrscheinlichkeitstheorie und
Zeitschrift
fur
Verwandte Gebiete. 55:123132.
TERDIK, G. (1999). Bilinear Stochastic Models and Related Problems of Nonlinear Time Series Analysis: A Frequency Domain Approach, Lecture Notes in Statist. 142. New York: Springer.
YAJIMA, Y. (1989). "A central limit theorem of Fourier transforms of strongly dependent stationary processes." J. Time Ser. Anal. 10:375383.
1.1
INTRODUCTION
5
Figure 1.1.3
Signals recorded by an array of seismometers at the time of an event.
These examples are taken from the physical sciences; however, the social sciences also lead to the consideration of vectorvalued time series. Figure 1.1.4 is a plot of exports from the United Kingdom separated by destination during the period 19581968. The techniques discussed in this work will sometimes be useful in the analysis of such a series although the results ob tained are not generally conclusive due to a scarcity of data and departure from assumptions. An inspection of the figures suggests that the individual component series are quite strongly interrelated. Much of our concern in this work will center on examining interrelations of component series. In addition there are situations in which we are interested in a single series on its own. For example, Singleton and Poulter (1967) were concerned with the call of a male killer whale and Godfrey (1965) was concerned with the quantity of cash held within the Federal Reserve System for the purpose of meeting interbank checkhandlingobligations each month. Figure 1.1.5 is a graph of the annual mean sunspot numbers for the period 17601965; seeWaldmeir (1961). This series has often been considered by statisticians; see Yule (1927), Whittle (1954), Brillinger and Rosenblatt (1967b). Generally speak ing it will be enough to consider single component series as particular cases
2.3
CUMULANTS
19
2.3 
CUMULANTS 

Consider for the present an r variate random variable 
, 
Y _{r} ) with 

ave \Yj\ ^{r} < 
<» ,j 
= 
1, , r where the Yjare real or complex. 

Definition 2.3.1 
The rth 
order joint cumulant, cum (Yi, 
, 
Y _{r} ), 
of 

(7i, 
, F,)is given by 

where the summation extends over all partitions (y\, • 
, 
v _{p} ), p = 
, r, 

of (l, ,r). 
An important special case of this definition occurs when F, = YJ = !, The definition gives then the cumulant of order r of a univariate random variable.
Theorem 2.3.1 cum (Y\,
the Taylor series expansion of log (ave exp i J^i Yjtj) about the origin.
. , Y _{r} ) is given by the
coefficient of (iyti • • • t _{r} in
This last is sometimes taken as the definition of cum 
, Y _{r} ). 

Properties of cum (7i, , Y _{r} ) include: 

(i) 
, a _{r} Y _{r} ) = a\ •• a, 
, Y _{r} ) 
for 
, a _{r} 

constant 

(ii) 
cum(Fi, 
, Y _{r} ) is symmetric in its arguments 

(iii) 
if any group of the Y's are independent of the remaining Y's, then 

cum<T,, 
,r _{r} ) = 0 

(iv) 
for the random variable 
, Y _{r} ), cum (Y\ + Z\,Y _{2} , 
, 
Y _{r} ) 

= cum(Yi,Y _{2}_{t} . , Fr) + cum (Z,,r _{2} , 
, Y _{r} ) 

(v) 
for n constant and r = 2, 3, 

(vi) 
if the random pendent, then variables 
, 7 _{r} ) 
and 
,Z _{r} ) 
are inde 

(vii) 
cum Yj 
= $Yjfor y = 
, r 

(viii) 
cum(y/,F/) = var Yjforj = 
!, 
,/ 
• 
(ix)
cum (r _{v} ,F _{f}_{c} ) = cov(r _{y} ,n) fory, A: =
, r.
2.3
CUMULANTS
21
= !,
and only if the ^(r/ _{y} ) 
the elements of the set {/, — /,>;
Mfij)
— ti',j'
,//;
/ =
1,
. The partition is indecomposable if
, M generate all
^ / / <J /} by addition and subtraction.
,/
iK/vy); O'J), 0V)
1
€ /V, m =
We remark that the set {ti — ?,; 1 ^ dependent differences, such as t\ —
the partition is indecomposable, we may find 7— 1 independent differences
, ti\ — ti. It follows that when
^ /} is generated by / — 1 in
/',/'
among the Mnj)  iK/Vy); (/,;), (/',/) € P _{m} ', m = 
1, 
, M. 

Theorem 
2.3.2 Consider 
a twoway array 
of 
random 
variables 
X^; 

j 
= 
, J/; / = 
1, 
,/ . Consider the / random variables 
The joint cumulant cum
, F/) is then given by
where the summation is over all indecomposable partitions v = v\ U • • • U v _{p} of the Table 2.3.4.
This theorem is a particular case of a result of work done by Leonov and Shiryaev (1959).
,^4)
be a 4variate normal random variable. Its cumulants of order greater than 2
will be 0. Suppose we wish cov {X^JtsX*} • Following the details of Theo rem 2.3.2 we see that
We briefly mention an example of the use of this theorem. Let (Xi,
This is a case of a result of Isserlis (1918). We end this section with a definition extending that of the mean'function and autocovariance function given in Section 2.2. Given the r vectorvalued
. , r, and
time series X(f), t E\X _{a} (i)\< oo, we
= 0, ±1, define
.
with components X _{a} (t), a = 1,
, will be called a joint cumulant function of order fe of the series X(f), t = 0,
for
. Such a function
, ak =
1,
r
and /i,
,/ * =
0, ±1,
±1,
2.6
CUMULANT SPECTRA OF ORDER k
25
. has finite
secondorder moments, but does not necessarily satisfy some mixing condi tion of the character of (2.5.1), we can still obtain a spectral representation of the nature of (2.5.5). Specifically we have the following:
In the case that the vectorvalued series X(0, / = 0, ±1,
Theorem 2.5.2 Let X(/), t — 0, ±1,
secondorder stationary with finite autocovariance function cxx(u) = cov {X(/ + u), X(f), for t, u = 0, ±1,Then there exists an r X r matrixvalue d functio n ¥xx(ty, — i r < X ^ TT, whos e entrie s ar e o f bounde d variation and whose increments are nonnegative definite, such that
. be a vectorvalued series that is
This function is given by
t
The function F^X ) is called the spectral measure of the series X(f),
= 0, ±1,
. In the case that (2.5.1) holds, it is given by
The representation (2.5.8) was obtained by Herglotz (1911) in the real valued case and by Cramer (1942) in the vectorvalued case.
2.6 CUMULANT SPECTRA OF ORDER fe
Suppose that the series X(r), t = 0, ± I, of dependence is small enough that
is stationary and that its span
In this case, we define the feth order cumulant spectrum,/,, 
, 
_{0} _{t} (Xi, 
, X*_i) 

for 
—oo < 
\j, 
< 
oo, at, 
, a* = 
1, 
, r, k 
= 
2, 
We will extend 

the 
definition 
(2.6.2) to the case k = 1 by setting f _{a} = c _{a} EX _{a} (f), 
2.7
FILTERS
27
This assumption implies, for / > 0, that wellseparated (in time) values of the process are even less dependent than implied by Assumption 2.6.1, the extent of dependence depending directly on /. Equation (2.6.6) implies that
_{f}_{l}_{f}_{c} (Xi, order ^ /.
If instead of expressions (2.6.1) or (2.6.6) we assume only ave Jf _{a} (/) ^{f}^{c} < °°,
\ _{k} ) appearing in (2.6.4) are Schwartz
f _{a}_{i}
,
,/
X*) has bounded
, then the/ _{f}_{l}_{l}
a _{t}
and uniforml y continuou s
derivatives of
a
distributions of order ^ 2. These distributions, or generalized functions, are
found in Schwartz (1957, 1959). In the case k = 2, Theorem 2.5.2 shows they are measures. Several times in later chapters we will require a stronger assumption than the commonly used Assumption 2.6.1. It is the following:
= !,
,
Assumption 2.6.3 The r vectorvalued series X(f), / = 0, ± Assumption 2.6.1. Also if
satisfies
then
for z in a neighborhood of 0.
This assumption will allow us to obtain probability 1 bounds for various
. is Gaussian, all that is required
is that the covariance function be summable. Exercise 2.13.36 indicates the form of the assumption for another example of interest.
statistics of interest. If X(/), t = 0, ±1,
2.7
FILTERS
In the analysis of time series we often have occasion to apply some manipulatory operation. An important class of operations consists of those that are linear and time invariant. Specifically, consider an operation
whose domain consists of r vectorvalued series X(f), t = 0, ±1, 
. and 
whose range consists of s vectorvalued series Y(f), / = 0, rbl, write 
. We 
to indicate the action of the operation. The operation is linear if for series
Xi(0 , X2(0> t = 0 , ±1,
. i n it s domai n an d fo r constant s ai , 0:2 w e hav e
2.7
FILTERS
31
Lemma 2.7.3 If X(0 is a stationary r vectorvalued series with £X(0 < °°, and {a(/)j is an s X r summable filter, then
t = 0, ±1,
series. If £X(f)* < » , k > 0, then £Y(/)j* < » .
. exists with probability 1 and is an s vectorvalued stationary
An important use of this lemma is in the derivation of additional sta tionary time series from stationary time series already under discussion. For example, if e(r) is a sequence of independent identically distributed r vector variates and {a(f)l is an s X r filter, then the s vectorvalued series
is a strictly stationary series. It is called a linear process. Sometimes we will want to deal with a linear time invariant operation whose transfer function A(X) is not necessarily the Fourier transform of an absolutely summable sequence. In the case that
it is possible to define the output of such a filter as a limit in mean square. Specifically we have
be an r vectorvalued series with
absolutely summable autocovariance function. Let A(\) be an s X r matrix
valued function satisfying (2.7.23). Set
Theorem 2.7.1 Let X(0, t = 0, db 1,
M = 0, ±1,
Then
exists for t — 0, ±1,
Results of this character are discussed in Rosenberg (1964) for the case in which the conditions of Theorem 2.5.2 are satisfied plus
2.9 EXAMPLES OF STATIONARY 
TIME SERIES 
35 

Example 2.8.3 If X(/), Y(f)> t = 0, db 1, related to X through . are both r vectorvalued 
with Y 

then the cumulant spectra of \(f) are given by 

where Bj(\) denotes the transfer function of the filter {&/«)}. 

Later we will see that Examples 2.8.1 and 2.8.3 provide convenient 
means 

of interpreting the cumulant spectra. power spectrum, crossspectrum, 
and 
higher 
order 
2.9 EXAMPLES OF STATIONARY TIMESERIES
The definition of, and several elementary examples of, a stationary time series was presented in Section 2.4. As stationary series are the basic entities of our analysis, it is of value to have as many examples as possible.
. be a sequence
of independent, identically distributed r vectorvalued random variables.
Such a series clearly forms a stationary time series.
Example 2.9.1 (A Pure Noise Series) Let e(f), t = 0, ±1,
Example 2.9.2 (Linear Process)
valued pure noise series of the previous example. Let
Let c(/),
t =
0,
±1,
.
be the r vector
where (a(w)} is an s X r summable filter. Following Lemma 2.7.3, this series is a stationary s vectorvalued series. If only a finite number of the a(w) in expression (2.9.1) are nonzero, then
the series X(t) is referred to as a moving average process. If a(0), a(m) 7* 0
and a(w) = 0 for u > m and u < 0, the process
is said to be of order
m.
Example 2.9.3 (Cosinusoid) with components
Suppose that X(f) is an r vectorvalued series
where /?i,
4>i
of values is considered and then the time points are all shifted by t, their
structure is unchanged.
, <f> _{r} i are unifor m on (—7r,7r), and
f • • • + <£r = 0. Thi s serie s i s stationary , becaus e i f any finit e collectio n
,/?
, are constant,
2.9 EXAMPLES OF STATIONARY
TIME SERIES
37
where t(f)
eigenvalues less than 1 in absolute value.
is an r vector pure noise series and a an r X r matrix with all
Example 2.9.6 (Autoregressive Schemes) Equation (2.9.6) leads us to con sider r vector processes X(t) that are generated by schemes of the form
, a(m) are r X r
matrices. If the roots of Det A(z) = 0 lie outside the unit circle where
where t(i) is an r vector pure noise series and
it can be shown (Section 3.8) that (2.9.7) has a stationary solution. Such an X(/) is referred to as an r vectorvalued autoregressive process of order m.
Example 2.9.7 (Mixing Moving Average and Autoregressive Process)
occasion
sider the r vectorvalued process X(t) satisfying
On
we combine the moving average and autoregressive schemes. Con
. , m are
r X r matrices, and b(&), k =
X(f), satisfying expression (2.9.9), exists it is referred to as a mixed moving average autoregressive process of order (m,n). If the roots of
, n are r X s matrices. If a stationary
where t(t) is an s vectorvalued pure noise series, a(y), j — 1,
lie outside the unit circle, then an X(t) satisfying (2.9.9) is in fact a linear process
where C(X) = A(A)!B(\); see Section 3.8.
Example 2.9.8 (Functions of Stationary Series) If we have a stationary series (such as a pure noise series) already at hand and we form time in variant measurable functions of that series then we have generated another stationary series. For example, suppose X(f) is a stationary series and Y(0 = ^ _{u} &(t — u)X(u) for some 5 X r filter {a(«)}. We have seen, (Lemma 2.7.1), that under regularity conditions Y(r) is also stationary. Alternatively we can form a Y(/) through nonlinear functions as by
2.11 FUNCTIONAL & STOCHASTIC APPROACHES TO TIME SERIES ANALYSIS
41
Then the 7th order cumulant spectrum of the series Y(t), t — 0, ±1, given by 
. is 

where the outer sum is over all the indecomposable partitions {Pi, 
, 
PM], 
M =
We have used the symmetrical notation for the cumulant spectra in Equa tion (2.10.10). Theorem 2 in Shiryaev (1960) provides a related result.
1,2,
o f Table 2.3.4.
2.11
THE FUNCTIONAL AND STOCHASTIC APPROACHES TO TIME SERIES ANALYSIS
Currently two different approaches are adopted by workers in time series:
the stochastic approach and the functional approach. The former, generally adopted by probabilists and statisticians (Doob, (1953) and Cramer and Leadbetter (1967)), is that described in Section 2.2. A given time series is re garded as being selected stochastically from an ensemble of possible series. We have a set 0 of r vector functions 6(0 After defining a probability measure on 6, we obtain a random function X(f,0), whose samples are the given functions d(t). Alternativelygiven X(/), we can set up an index 6 = X( •) and take 6 to be the set of all 6. We then may set X(f,0) = X(f,X()). In any case we find ourselves dealing with measure theory and probability spaces. In the second approach, a given r vector time series is interpreted as a mathematical function and the basic ensemble of time functions takes the
for m X(/,p ) = X( / + D) \ v = 0 , ±1 , ±2 ,
vector function. This approach is taken in Wiener (1930), for example, and
is called generalized harmonic analysis. The distinction, from the point of view of the theoretician, is the different mathematical tools required and the different limiting processes involved.
Suppose that X(f) has components^/), a = 1, approach we assume that limits of the form
.} , wher e X(f ) i s th e give n r
,/• . In the functional
2.12
TRENDS
43
In other words, if the function (of the functional approach) satisfies cer tain regularity conditions, then there is a strictly stationary process whose analysis is equivalent. Conversely: if X(/) is ergodic (metrically transitive), then with probability 1 any sample path satisfies the required limiting properties and can be taken as the basis for a functional approach. In conclusion, we have
1
Theorem 2.11.1 If an r vector function satisfies (i) and (ii) above, then a stationary stochastic process can be associated with it having the same limit ing properties. Alternatively, if a stationary process is ergodic, then with probability 1 any of its sample paths can be taken as the basis for a func tional approach.
These two approaches are directly comparable to the two approaches to statistics through kollectivs (Von Mises (1964)) and measurable functions (Doob (1953)); see also Von Mises and Doob (1941). The condition that X(/) be ergodic is not overly restrictive for our pur poses since it is ergodic when it satisfies Assumption 2.6.1 and is determined by its moments; Leonov (1960). Wenote that a general stationary process is a mixture of ergodic processes (Rozanov (1967)), and the associated process obtained by the above procedure will correspond to some component of the mixture. The limits in (i) will exist with probability 1; however, they will generally be random variables. Wold (1948) discusses relations between the functional and stochastic ap proaches in the case of secondorder moments. We note that the limits required in expressions (2.11.1) and (2.11.2) follow under certain conditions from the existence of the limits in (i); see Wintner
(1932).
We will return to a discussion of the functional approach to time series analysis in Section 3.9.
2.12
TRENDS
One simple form of departure from the assumption of stationarity is that
the series X(t), t = 0,
has the form
^{J} X(/) is ergodic if for any realvalued/[x] with ave /(X(/)]  < o°, with probability 1,
See Cramer and Leadbetter (1967), Wiener et al (1967), Halmos (1956), Billingsley (1965), and Hopf (1937).
2.13
EXERCISES
47
2.13.28 Show that Vi(t\
F _{2} (0 of (2.7.33) come from X(t) through filters with
coefficients \a(u) cos \ou], [a(u) sin XOH}, respectively.
2.13.29 Prove that S(ax) = \a\~ ^{l} 8(x).
2.13.30 Let X(0 be a stationary r vector valued series with X _{}} {t) = pjXfa — 1) +
, /• where e(0 is an r vector pure noise series.
e/0, \pj\
<
1, y =
1,
Prove that
where T = min (/i,
6*4(0}.
, /*), a = (ai,
, a*) and # _{a}_{i}
_{a}_{k} = cum{e _{a} ,(/),
,
2.13.31 Let $(r), 7 = 1, 2,
ties; <S>(70> oo
. be a sequence of positive numbers with the proper
X(0, / = 0,
and $(T+ l)/f>(r)>l
as
r> « .
Let
±1,
be an r vectorvalued function with the property
for u = 0, ±1,
G*;KX), —TT < X ^ T, such that
. Show that there exists an r X r matrixvalued function
Him: Define lxx^(\) as in the proof of Theorem 2.5.2. See Bochner (1959)
p.
/•
/*
329, and Grenander (1954). Prove / \(a)lxx(T)(oi)da> / A(a)dGxx(a)
^
J »
/ T
J T
for A(a) continuous on [—TT.TT] also.
2.13.32 Let X(r), t — 0, ±1, spectra f _{a}_{i} _{a}_{k} (\i,
. be a vectorvalued stationary
series with cumulant
, A*) Evaluate the cumulant spectra of the time
reversed series X(—/), t = 0, ±1,
2.13.33 Show that
for —oo < X <
oo. (The Poisson Summation Formula, Edwards (1967))
2.13.34 Show that the function cxx(u) = 1, for \u\ ^ m and cxx(u) = 0 otherwise, cannot be an autocovariance function.
, / — 1 be / independent realizations
2.13.35 Let Xj(t\ t = 0,
\j =
of a stationary process. Let
Show that Y(t), / = 0, ±1,
spectrum is
fxxO^J)
is a stationary series. Show that its power
3
ANALYTIC PROPERTIES OF FOURIER TRANSFORMS AND COMPLEX MATRICES
3.1
INTRODUCTION
The principal analytic tool that we will employ with time series is the Fourier transform. In this chapter we present those portions of Fourier analysis that will be required for our discussion. All the functions considered in this chapter will be fixed, rather than stochastic. Stochastic properties of Fourier transforms will be considered in the next chapter. Among the topics discussed here are the following: the degree of approxi mation of a function by the partial sums of its Fourier series; the improve ment of this approximation by the insertion of convergence factors; the Fourier transform of a finite set of values; the rapid numerical evaluation of Fourier transforms; the spectrum of a matrix and its relation to the approxi mation of one matrix by another of reduced rank; mathematical properties of functions of Fourier transforms; and finally the spectral or harmonic representation of functions possessing a generalized harmonic analysis. We begin by considering the Fourier series of a given function /4(X).
3.2 FOURIER SERIES
Let A(\),
such that
— o>
<
\
<
oo, be a complexvalued function of period
2ir
49
50
ANALYTIC
PROPERTIES
OF FOURIER
TRANSFORMS
The Fourier coefficients of A(\) are given by
and the Fourier series of A(\) is then given by
There is extensive literature concerning Fourier series and Fourier coeffi cients; see Zygmund (1959) and Edwards (1967), for example. Much of this literature is concerned with the behavior of the partial sums
In this work we will have frequent occasion to examine the nearness of the /4 ^{(}^{n}^{)} (X) to A(\) for large n. Begin by noting, from expression (3.2.2) and Exercise 1.7.5, that
The function
is plotted in Figure 3.2.1 for the values n = 1, 3, 5, 10. We note that it fluctuates in sign and that it is concentrated in the neighborhood of a = 0, becoming more concentrated as n increases. Also
from Exercise 1.7.5. In consequence of these properties of the function (3.2.6) we see that A ^{(}^{n}^{}} (\) is a weighted average of the function A(\ — a), with weight concentrated in the neighborhood of a = 0. We would expect
(X) to be near A(\) for large n, if the function A(a) is not too irregular. In
fact we can show that A ^{(}^{n}^{)} (\) tends to ^(X) as n —> » if, for example, A(a) is
_{^} (n)
of bounded variation; see Edwards (1967) p. 150.
Under supplementary regularity conditions we can measure the rapidity
of approach of /J
(fl)
(X) to A(\) as n —> o°. Suppose
52
ANALYTIC
PROPERTIES
OF FOURIER
TRANSFORMS
This condition is tied up with the degree of smoothness of A(a). Under it A(a) has bounded continuous derivatives of order ^ k. We have therefore
and so
In summary, the degree of approximation of A(\) by A ^{(}^{n} \\) is intimately related to the smoothness of /4(\).
(X) need not necessarily approach A(\) as
n —> °o even in the case that A(X) is a bounded continuous function of X; see Edwards (1967) p. 150, for example. However, the relationship between the two functions is well illustrated by (3.2.5). The behavior of A(\) — A ^{(}^{n}^{)} (\) is especially disturbed in the neighborhood of discontinuities of A(\). Gibbs'
phenomenon involving the nondiminishing overshooting of a functional value can occur; see Hamming (1962) p. 295 or Edwards (1967) p. 172.
We warn the reader that /4
(/l)
3.3 CONVERGENCE FACTORS
Fejer (1900, 1904) recognized that the partial sums of a Fourier series might be poor approximations of a function of interest even if the function were continuous. He therefore proposed that instead of the partial sum (3.2.4) we consider the sum
Using expression (3.2.2) and Exercise 1.7.12 we see that (3.3.1) may be written
The function
'We will make use of the Landau
o, O notations writing a _{n} = oC3 _{n} ) when
an/fin —»0 as n —> °o and writing a _{n} = O(j8 _{n} ) when  a _{n} /fi _{n}  is bounded for sufficiently large n.
3.3
CONVERGENCE FACTORS
53
is plotted in Figure 3.3.1 for « = 2, 4, 6, 11. It is seen to be nonnegative, concentrated in the neighborhood of a = 0 and, following Exercise 1.7.12, such that
It is blunter than the function (3.2.6) of the previous section and has fewer ripples. This greater regularity leads to the convergence of (3.3.2) to A(\) in the case that A(a) is a continuous function in contrast to the behavior of (3.2.5); see Edwards (1967) p. 87. The insertion of the factors 1 — \u\/n in expression (3.3.1) has expanded the class of functions that may be reason ably represented by trigonometric series.
Figure 3.3.1
Plot of sin ^{2} Jna/2ir/j sin ^{2} $«.
3.3
CONVERGENCE FACTORS
55
Table 3.3.1
Some Particular Convergence Factors
Authors
Dirichlet
[Edwards (1967)]
Fejer, Bartlett [Edwards (1967), Parzen (1963)] de la Valle'Poussin, Jackson, Parzen
[Akhiezer(1956),
Parzen (1961)]
Hamming, Tukey
[Blackman and
Tukey (1958)]
Bohman
[Bohman (I960)]
Poisson
[Edwards (1967)]
Riemann, Lanczos
[Edwards (1967),
Lanczos (1956)]
Gauss
Weierstrass
[Akhiezer(1956)]
Cauchy, Abel,
Poisson
[Akhiezer (1956)]
Riesz, Bochner
Parzen
[Bochner (1936),
Parzen (1961)]
Tukey
[Tukey (1967)]
3.3
CONVERGENCE FACTORS
57
Table 3.3.2
Kernel
Dirichlet Fejer de la Valle'Poussin Hamming Bohman Poisson Riemann Gauss Cauchy Riesz Tukey
for some finite K, then
Bandwidths of the Kernels
Expression (3.3.11) gives a useful indication of the manner in which the nearness of (3.3.5) to A(\) depends on the convergence factors employed. If possible it should be arranged that
. If h(x) = h(—x), then this is the case for odd values
of p. The requirement for even p is equivalent to requiring that h(x) be very flat near x = 0. The last function of Table 3.3.1 is notable in this respect.
In fact, the optimum h(u/ri) will depend on the particular A(\) of interest.
be 0 for p = 1, 2,
A considerable mathematical theory has been developed concerning the
best approximation of functions by trigonometric polynomials; see Akhiezer (1956) or Timan (1963), for example. Bohman (1960) and Akaike (1968) were concerned with the development of convergence factors appropriate for a broad class of functions; see also Timan (1962), Shapiro (1969), Hoff
(1970), Butzer and Nessel (1971). Wilkins (1948) indicates asymptotic expansions of the form of (3.3.18) that are valid under less restrictive conditions.
3.3 CONVERGENCE
FACTORS
59
Figure 3.3.2 Transfer function of ideal Hilbert transform and approximations with various factors, n = 7.
3.4 FINITE FOURIER
TRANSFORMS AND THEIR
PROPERTIES
61
is worthwhile to alter our notation slightly
and to consider the general case of a vectorvalued sequence. Specifically we
consider an r vectorvalued sequence X(0), X(l),
, define the finite Fourier transform of this sequence to be
main is 0, 1,
n. We
, X(T — 1), whose do
Before proceeding further, it
. , r — 1rather than —n, —n+ 1,
,
In the
case that T — 2n j
1, n being an
integer, we may write
and thus we see that the only essential difference between definitions of the form (3.4.1) and the form (3.4.2) is a multiplier of modulus 1. Which defini tion is more convenient depends on the situation being discussed. Among the properties of the definition (3.4.2) we note
Also, if the components of X(/) are realvalued, then
These two properties imply that, in the case of realvalued components, the
principal domain of AX ^{(}^{T}^{}} (\) ma y b e take n t o b e 0 ^ X ^ TT. Continuing , w e
note that if X(f), Y(/), t = then
, T — 1 are given and if a and /3 are scalars,
On occasion we may wish to relate the finite Fourier transform of the convolution of two sequences to the Fourier transforms of the two se quences themselves. We have
Lemma 3.4.1
bounded. Let a(/), / = 0,
Let X(/)> t = 0,
Set
±1,
Then there is a finite K such that
be r vectorvalued and uniformly be s X r matrixvalued and such that
3.4 FINITE FOURIER TRANSFORMS AND THEIR
PROPERTIES
63
This transform behaves like the derivatives of the transform of Example 1.
Notice that it is concentrated in the neighborhood of X = 0, ±2ir,
large n. A polynomial J£* «*** will behave as a linear combination of functions of the form (3.4.14).
for
Example 5 (Monomial Amplitude Cosinusoid) then (3.4.10) is
Suppose A{0 =t ^{k} exp{iut},
This is the function of Example 4 translated along by w frequency units.
The general nature of these results is the following: the Fourier transform
of a function X(f) is concentrated in amplitude near \ = 0, ±2ir,
if
X(f) is constan t or slowly changing wit h / . It is concentrated near X = o>,
w ± 2ir,
quency o> multiplied by a polynomial in /. The transform (3.4.2) may be inverted by the integration
if X(t) is a cosinusoid of frequency « or is a cosinusoid of fre
Alternatively it is seen to be inverted by the sum
The Tr vectors Ax ^{{}^{T}^{)} (2ws/T') _{>} s = 0,
the discrete Fourier transform of X(0, t =
numerical evaluation and properties in the next two sections.
, T — 1,are sometimes referred to as
, T — 1. We will discuss its
The discrete Fourier transform may be written in matrix form. Let 9C de
note the r X T matrix whose columns are
Let 3D denote the r X T matrix whose columns are Ax
, X(T — 1) successively.
,
T  1. Let fr denote the T X T matrix with exp { i(2irst/T)} in row s + 1
and column t f 1 for s,t =
(T)
(2irs/T),
s =
, T — 1. Then we see that we have
3.6
APPLICATIONS OF DISCRETE FOURIER TRANSFORMS
67
3.6 APPLICATIONS OF DISCRETE FOURIER TRANSFORMS
Suppose the values X(t\
Y(t\
t =
sometimes require the convolution
If
, T 
1 are available. We will
then we quickly see that the convolution (3.6.1) is the coefficient of
exp{—i\u} in the trigonometric polynomial dx ^{m} (\)dY ^{(}^{T}^{)} (\). given by
It is therefore
This occurrence suggests that we may be able to compute (3.6.1) by means of a discrete Fourier transform and so take advantage of the Fast Fourier Transform Algorithm. In fact we have
Lemma 3.6.1
the convolution (3.6.1) is given by
Given X(t),
Y(t),
t =
In general (3.6.4) equals
, T 
1 and an integer
S > T,
We may obtain the desired values of the convolution from (3.6.4) by taking S large enough. If S is taken to be highly composite then the discrete Fourier transforms required in the direct evaluation of (3.6.4) may be rapidly cal culated by means of the Fast Fourier Transform Algorithm of the previous section. Consequently the convolution (3.6.1) may well be more rapidly computed by this procedure rather than by using its definition (3.6.1) di rectly. This fact was noted by Sande; see Gentleman and Sande (1966), and also Stockham (1966). From (3.6.5) we see that for S  T < \u\ ^ T  1,
3.6
APPLICATIONS OF DISCRETE FOURIER TRANSFORMS
Molto più che documenti.
Scopri tutto ciò che Scribd ha da offrire, inclusi libri e audiolibri dei maggiori editori.
Annulla in qualsiasi momento.