ADAPTIVE FOURIER ANALYSIS FOR UNEQUALLY-

SPACED TIME SERIES DATA

By

Hong Liang


Dissertation submitted to the faculty of the
Virginia Polytechnic Institute and State University
in partial fulfillment of the requirements for the degree of

Doctor of Philosophy
in
Statistics


Robert V. Foutz, Chair
Marion R. Reynolds, J r.
Donald R. J ensen
George R. Terrell
Christine Anderson-Cook



April 16, 2002
Blacksburg, Virginia


Keywords: Adaptive Fourier Analysis, Unequally-Spaced, Time Series, Walsh-Fourier,
Wavelet

Adaptive Fourier Analysis For Unequally-Spaced Time Series
Data

by

Hong Liang

Robert V. Foutz, Chairman

Statistics

(ABSTRACT)

Fourier analysis, Walsh-Fourier analysis, and wavelet analysis have often been used in
time series analysis. Fourier analysis can be used to detect periodic components that have
sinusoidal shape; however, it might be misleading when the periodic components are not
sinusoidal. Walsh-Fourier analysis is suitable for revealing the rectangular trends of time
series. The flaw of the Walsh-Fourier analysis is that Walsh functions are not periodic.
The resulting Walsh-Fourier analysis is more difficult to interpret than classical Fourier
analysis. Wavelet analysis is very useful in analyzing and describing time series with
gradual frequency changes. Wavelet analysis also has a shortcoming by giving no exact
meaning to the concept of frequency because wavelets are not periodic functions. In addi-
tion, all three analysis methods above require equally-spaced time series observations.
In this dissertation, by using a sequence of periodic step functions, a new analysis
method, adaptive Fourier analysis, and its theory are developed. These can be applied to
time series data where patterns may take general periodic shapes that include sinusoids as
special cases. Most importantly, the resulting adaptive Fourier analysis does not require
equally-spaced time series observations.




iii

Acknowledgements



I would like to sincerely express my appreciation to Dr. Robert V. Foutz, my aca-
demic advisor, who introduced me to time series analysis. His excellent guidance, cour-
ses, support, wisdom, patience, and enthusiasm were essential throughout my academic
career at Virginia Tech.

I would like to thank all of my committee members: Dr. Reynolds, Dr. J enson,
Dr. Terrell, and Dr. Anderson-Cook. I want to thank Dr. Reynolds for his wonderful
courses helping me intuitively understand applied statistics and academic research me-
thods. I would like to send thanks to Dr. J ensen for his careful reading of my dissertation.
To Dr. Terrell, I thank him for his comments and suggestions. Lastly, I would also like to
thank Dr. Anderson-Cook for her many suggestions of modifications in my dissertation.

I want to give my thanks to all faculties at Virginia Tech in the Statistics Depart-
ment, Graduate School, and International Student Center. In the Statistics Department, I
would especially like to thank Dr. Good who offered me a reference paper for my disser-
tation. In Graduate School, I would especially like to thank Dr. McKeon, for his outstand-
ing service for international students.

Finally, I would like to thank my wife, Haiming and my son, Sixing for their love
and support.



iv

Table of Contents


1 Introduction 1

2 Review of data analysis methods 4
2.1 Fourier Analysis . 4
2.2 Walsh-Fourier Analysis 7
2.3 Wavelet Analysis . 12
2.4 The spectral analysis methods for unequally-spaced time series data 17
2.4.1 Interpolation methods ..... 17
2.4.2 Least-Squares methods 19
2.4.3 Linear algebra method and the others .. 21
3 Adaptive Fourier Analysis 23
3.1 Motivation 23
3.2 The periodic step functions 24
3.3 The properties of the periodic step functions 27
3.4 Multiresolution analysis 28
3.5 The adaptive Fourier analysis of the digital time series .. 29
3.6 Methodology .. 32
4 Examples 36
Example 4.1 .. 36
Example 4.2 . 45
Example 4.3 . 49



v
Example 4.4 . 52
Example 4.5 54
5 Conclusion and future research 58
5.1 Summary and conclusion . 58
5.2 Proposed future research .. 59
Bibliography 61
Appendix 64

Appendix 1. Proof of Theory 3.1 and its Corollary 3.1 .. 64

Appendix 2. The construction of x
e
(t) . 73

Appendix 3. Proof of x(t) =
n
lim
m
lims
m n,
(t) 75

Appendix 4. A nonstandard inner product <y, z>
4
77

Appendix 5. Proof of (3.12) . 77

Appendix 6. Background .. 79

A6.1 Vector space . 79

A6.2 Inner product space, Hilbert space and their properties 82

A6.3 Orthogonal projection and orthonormal bases 87


Vita 90








vi


List of Figures


Figure 2.1 The first eight Walsh functions . 10
Figure 2.2 Some basis functions for Fourier transform .. 11
Figure 2.3 Four different mother wavelets ..... 13
Figure 2.4 The signal ss =10*cos(pi*t/15)+3*cos(pi*t/10) ... 15
Figure 2.5 Decomposition of SS into the sum of 16 Wavelet functions . 15
Figure 2.6 Doppler(t) =((t*(t-1))**0.5)*sin(2.1*pi/(t+0.5)) 16
Figure 3.1 The periodic step function in the complex plan 25
Figure 3.2 The step function f ) (
, 6 , 2
t
j
..... 26
Figure 4.1 Monthly accidental deaths in the U.S.A., 1973-1978 .. 36
Figure 4.2 Frequency component, k=6 in Ex. 4.1 ... 38
Figure 4.3 Adaptive Fourier line spectrum for Ex. 4.1 ... 39
Figure 4.4 Fourier line spectrum in Ex. 4.1 39
Figure 4.5 Monthly accidental deaths without Mar. in Ex. 4.1 40
Figure 4.6 Frequency component, k=6 in Ex.4.1 without Mar. ... 40
Figure 4.7 Monthly accidental deaths without J an., , J un., respectively .. 42
Figure 4.8 Frequency component, k=6, without J an., , J un., respectively ... 42
Figure 4.9 Monthly accidental deaths by deleting randomly 10 points, first time 43
Figure 4.10 Frequency component, k=6, deleting randomly 10 points, first time... 43
Figure 4.11 Monthly accidental deaths by deleting randomly to points, second time..44



vii
Figure 4.12 Frequency component, k=6, deleting randomly 10 points, second time.. 44
Figure 4.2(1) Original time series in Ex.4.2 . 46
Figure 4.2(2) k =2 freq. Component in Adaptive Fourier analysis 46
Figure 4.2(3) k =2 freq. component(-) in Fourier analysis & original signal(.) 47
Figure 4.2(4) Multiresolution decomposition of the signal 48
Figure 4.2(5) Time-scale plot for the signal .. 48
Figure 4.3(1) Original time series in Ex.4.3 .50
Figure 4.3(2) Reconstruction of the time series in Ex.4.3 .50
Figure 4.3(3) Adaptive Fourier ANOVA in Ex.4.3 ...51
Figure 4.3(4) Fourier ANOVA in Ex.4.3 .. 51
Figure 4.3(5) Frequency component k=3 in Ex.4.3 51
Figure 4.3(6) Frequency component k=2 in Ex.4.3 51
Figure 4.4(1) Original time series in Ex.4.4 ...53
Figure 4.4(2) Reconstruction of the time series in Ex.4.4 .. 53
Figure 4.4(3) Adaptive Fourier line spectrum in Ex.4.4 53
Figure 4.4(4) Frequency component k=3 in Ex.4.4 54
Figure 4.4(5) Frequency component k=2 in Ex.4.4 54
Figure 4.5(1) Original time series in Ex.4.5 56
Figure 4.5(2) Reconstruction of the time series in Ex.4.5 ... 56
Figure 4.5(3) Adaptive Fourier line spectrum in Ex.4.5 . 57
Figure 4.5(4) Fourier line spectrum in Ex.4.5. 57
Figure 4.5(5) Frequency component k=2 in Ex.4.5.57
Figure 4.5(6) Frequency component k=1 in Ex.4.5 . 57



1
1 Introduction



In the statistical analysis of time series, Fourier analysis has provided a general method for

discovering or analyzing the periodicity and examining the global energy-frequency dis-

tribution in time series data. It has been a valuable and powerful tool of time series analy-

sis, but it still has some crucial limitations : the data must consist of equally-spaced obser-

vations; and periodic components in the data must be sinusoidal . Otherwise, the analysis

might give misleading results.


For non-sinusoidal waveforms, such as square-wave and rectangular waveform, Fourier

analysis needs too many additional harmonic components to decompose approximately

the waveform functions, and therefore it spreads the energy over a wide frequency range,

which might cause Fourier ANOVA to be misunderstood. Beauchamp (1975,1984) empiri-
cally demonstrated that when a time series is based upon a sinusoidal waveform, Fourier
analysis is more efficient, and when a time series is rectangular or with sharp discontinui-
ties, Walsh-Fourier analysis is more efficient. However, the Walsh functions are not perio-
dic, and therefore the Walsh-Fourier decomposition is more difficult to interpret than the
Fourier harmonic decomposition.

In actual time series analysis, given a sequence of N values x =(x
1,
x
2,
, x
N
), by using the
discrete Fourier Transform (DFT), x can be evaluated only at a special set of N/2 evenly
spaced frequencies . When x comes from a non-sinusoidal function containing only less
than N/2 frequencies, DFT may cause energy spreading.




2
For the time series of unequally-spaced data, Fourier analysis, the Discrete Fourier Trans-

form, cant be used directly. A variety of methods have been suggested to overcome this

limitation of the DFT due to unequally-spaced data. Among them are the interpolation me-
thod, the Least-squares method, and the algebra method. But these methods have some
shortcomings, and they also suffer the shortcoming of Fourier spectral analysis , since
they are all Fourier based.

In wavelet analysis, a given function x(t) is decomposed into a sum of wavelet functions.
These wavelet functions are derived from a single mother wavelet function (t) by apply-
ing varying translations and dilations to (t). Given a mother wavelet function (t), for
real a>0, and real b, a sequence of wavelet functions can be derived as

b a,
(t) =a
2
1

(
a
b t
),
where a represents the scale parameter and b represents the translation parameter. In a

loose sense, we may identify the wavelet parameters a, b with frequency and time ,

respectively. See Priestley(1996). Wavelet analysis is very useful in analyzing and describ-

ing time series data with gradual frequency changes. However, wavelets are not periodic

functions, and the concepts of frequency and of periodicity have no precise meaning in the
resulting wavelet analysis. In addition, different mother wavelet may have different analy-
sis results.

In this research, we develop a new analysis method, adaptive Fourier analysis and its

theory. The advantage of adaptive Fourier analysis and its theory is that it can be applied




3
to time series data where patterns may take general periodic shapes that include sinusoids
as special cases. Most importantly, the resulting adaptive Fourier analysis does not require
equally-spaced time series observations.

Chapter 2 reviews the present available time series data analysis methods. In Chapter 3,
adaptive Fourier analysis and the definition of ADFT are presented. Chapter 4 gives some
examples. Chapter 5 makes conclusions and suggestions for future research. Proofs are
contained in the Appendix1-5. Appendix6 provides the background for theoretical results
and their proofs in this research. All conceptions and their properties used in Chapter3 and
Appendix1-5, such as vector space, direct sum, inner product, Hilbert space, weight inner
product, orthogonal projection, and Gram-Schmidt procedure, are presented in Appendix6.















4

2. Review of data analysis methods


2.1 Fourier Analysis


2.1.1 Fourier Series

The basic idea of a Fourier series is that any function x(t) L
2
[0,T] can be decomposed

into an infinite sum of cosine and sine functions:

x(t)= ]
2
sin
2
cos [
0
T
kt
b
T
kt
a
k
k k

=
+ , for all t (2.1)

where a
k
=

T
t x
T
0
) (
1
cos
T
kt 2
dt , (2.2)

b
k
=

T
t x
T
0
) (
1
sin
T
kt 2
dt . (2.3)

This is due to the fact that {1, cos
T
kt 2
, sin
T
kt 2
, k=1, 2, 3, ...} form a basis for the space
L
2
[0,T]. The summation in (2.1) is up to infinity, but x(t) can be well approximated in

the L
2
sense by a finite sum with K cosine and sine functions:

X(t) ]
2
sin
2
cos [
0
T
kt
b
T
kt
a
K
k
k k

=
+ . (2.4)

This decomposition shows that x(t) can be approximated by a sum of sinusoidal shapes at

frequencies
k
=2k/T, k =0,1, , K. In addition, the variability in x(t) as measured by

dt t x
T

0
2
| ) ( | can be approximately partitioned into the sum of the variability of the sinusoi-

dal shapes:




5
dt t x
T

0
2
| ) ( | =

T
0
[ ]
2
sin
2
cos [
0
T
kt
b
T
kt
a
K
k
k k

=
+
2
dt
2
0
| |

=
K
k
k
. (2.5)



A standard technique of time series analysis is to treat the partition (2.5) as an analysis of

variance (ANOVA) for identifying sinusoidal periodicities in a time series data set { x(t),

0<t T}. When x(t) has sharp discontinuities or a non-sinusoidal waveform, such as a

rectangular waveform, then we would require a very large number, K, of terms in its

Fourier series in order to get an adequate approximation.



2.1.2 Discrete Fourier Transform (DFT)
For an arbitrary time series data set, x =( x(t
1
), x(t
2
), , x(t
N
) ), if the observation times

are equally spaced, at interval t, then the data set x can be simply written as x =( x(1),

x(2), , x(N) ) by taking t =1 and t
j
=j; and there is an orthogonal system { e
k
itw
: -
2
N
+

1 k
2
N
if n is even and -
2
1 N
k
2
1 N
if n is odd }, so that the Discrete Fourier

Transform of x can be defined by

X
*
(k) =
N
1

N
t
itw
k
e t x
1
) ( , (2.6)

where the frequencies w
k
=2k/N, k=0,1, ,

2 / N , are called the Fourier frequencies;

and

r is the largest integer no larger than r. The Fourier series of x(t) can be written as


x(t) =

=
2 /
2 / ) 1 (
*
) (
N
N k
k X e
k
itw
. (2.7)


Corresponding to (2.5), we have the ANOVA



6

N t x
N
t
=

=
) (
1
2

=
2 /
2 / ) 1 (
2 *
| ) ( |
N
N k
k X . (2.8)


This representation provides an ANOVA for revealing how well the periodicities in x

may be described by the sinusoidal shapes X
*
(-k)e
k
itw

+ X
*
(k)e
k
itw
. The ANOVA

decomposition in (2.8) holds only if the DFT, X
*
(t), is evaluated only at a fixed set of
N/2 equally-spaced frequencies w
k
, and the data set must be equally-spaced.

2.1.3. Spectrum ANOVA for Equally-Spaced Time Series Data

Let L
2
be the set of all continuous-time, complex-valued functions. L
2
={y(t), 0t N}
for which the Lebesque integral

N
t y
0
2
| ) ( | dt is finite. The set L
2
is a Hilbert space with
inner product
<y, z>
1
=
N
1

N
t z t y
0
) ( ) ( dt, (2.9)
where ) (t z is the complex conjugate of z(t). Let A
k
be the subspace of L
2
that contains
all periodic functions that have the form c*exp(i
k
t) for complex-valued scalars c and for
k
=
N
k 2
. Also, let

r be the largest integer no larger than r. Classical analysis shows
that corresponding to every equally-spaced digital times series data set x =(x(1), x(2),,
x(N) ), there is a continuous-time function s in the N-dimensional subspace,
L
N A,
=A
] [
2
1

N
+ +A
] [
2
N
, (2.10)



7
with the property that s(t) =x(t) at t =1, 2, , N. See Koopmans ( 1974, P. 17). The
function is s =P
2
( x | L
N A,
), the projection of x onto L
N A,
with respect to the inner
product
<y, z>
2
=
N
1

=
N
t
t z t y
1
) ( * ) ( . (2.11)

Because the subspaces A
k
in (2.10) are orthogonal with respect to the inner product <y,
z>
2
, the function s =P
2
( x | L
N A,
), has the Fourier series representation,
s(t) =


) | (
2 /
2 / ) 1 (
2
=
N
N k
k
A x P (t), (2.12)
where P
2
(x | A
k
) is the projection of x onto A
k
with respect to <y, z >
2
. The corres-
ponding partition of |s|
2
2
=<s , s>
2
is
|s|
2
2
=


2
2 /
2 / ) 1 (
2
| ) | ( |

=
N
N k
k
A x P . (2.13)

In the Fourier series (2.12), each frequency component P
2
( x | A
k
) +P
2
( x | A
k
) is a
sinusoid with frequency
k
. The corresponding partition (2.13) provides an ANOVA for
revealing how well the periodicities in s ( and in x ) may be described by the sinusoidal
shapes P
2
( x | A
k
) +P
2
( x | A
k
).


2.2 Walsh-Fourier Analysis


Fourier series can be used to partition functions into sinusoidal waves. Unlike Fourier

analysis, Walsh-Fourier analysis deals with the decomposition of functions into the




8
Walsh functions, which are rectangular waves. These waves have been used in signal

transmission, speech analysis and pattern recognition, etc. Walsh-Fourier analysis is

suited to the analysis of discrete valued and categorical valued time series. See Stoffer

(1991).


The Walsh functions are defined as products of the Rademacher functions and the Rade-

macher functions are defined as below.


Definition ( Rademacher functions ) Consider the function defined on the half open

interval [0,1) by


0
(t) =

. ) 1 , 2 / 1 [ , 1
) 2 / 1 , 0 [ , 1
t for
t for
(2.14)


Extend it to the real line by periodicity of period 1 and set
k
(t) =
0
(2
k
t) for k =0,1,

and real t. The functions
k
(t) are called the Rademacher functions. The Walsh system

{ W[0, t], W[1, t], W[2, t], }is obtained by taking all possible products of Rademacher

functions , where W[0, t] is defined as W[0, t] =1 and the other W[n, t]
,
n 1 is defined

by

W[n, t] =

=
k
i
i
i
t
0
)) ( (

,

(2.15)

where n=

=
k
i
i
i
0
2 ,
k
=1 and
i
=0 or 1 for i=0, 1, ,k-1.

There are a number of definitions of the Walsh functions. Another definition of the

Walsh functions { W[0, t], W[1, t], W[2, t], }is by induction as follows:






9
Initialize the induction by defining

W[0, t] =1, t [0, 1),

W[1, t] =

). 1 , 2 / 1 [ , 1
) 2 / 1 , 0 [ , 1
t
t
(2.16)

Then proceed recursively for n =1, 2, , through

W[2n, t] =

<
<
, 1 2 / 1 ], 1 2 , [ ) 1 (
2 / 1 0 ], 2 , [
t t n W
t t n W
n
(2.17)


and W[2n+1, t] =

<
<
+
. 1 2 / 1 ], 1 2 , [ ) 1 (
2 / 1 0 ], 2 , [
1
t t n W
t t n W
n
(2.18)


To compare Walsh functions with sinusoidal functions, Figure 2.1 plots the first eight

Walsh functions, W[0, t], W[1, t], W[2, t], , W[7, t], and Figure 2.2 plots the first three

sine and cosine functions. The even numbered Walsh functions are symmetric about t =
1/2 , while cosine functions are symmetric about t=0; on the other hand, the odd number-
ed Walsh functions are antisymmetric about t=1/2, while sine functions are antisymmetric
about t =0. In fact, even numbered Walsh functions play the role of the cosine terms in
Fourier transform and odd numbered Walsh functions play the role of the sine terms in
Fourier transform.









10
Figure 2.1 The first eight Walsh functions
0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7 0. 8 0. 9 1
-1
0
1
W
[
0
,
t
]
0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7 0. 8 0. 9 1
-1
0
1
W
[
2
,
t
]
0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7 0. 8 0. 9 1
-1
0
1
W
[
4
,
t
]
0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7 0. 8 0. 9 1
-1
0
1
W
[
6
,
t
]

0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7 0. 8 0. 9 1
-1
0
1
W
[
1
,
t
]
0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7 0. 8 0. 9 1
-1
0
1
W
[
3
,
t
]
0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7 0. 8 0. 9 1
-1
0
1
W
[
5
,
t
]
0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7 0. 8 0. 9 1
-1
0
1
W
[
7
,
t
]





11
-4 -3 -2 -1 0 1 2 3 4
-1
0
1
s
i
n
(
t
)
Figure 2.2 Some basis functions for Fourier transform
-4 -3 -2 -1 0 1 2 3 4
-1
0
1
s
i
n
(
2
t
)
-4 -3 -2 -1 0 1 2 3 4
-1
0
1
s
i
n
(
3
t
)
-4 -3 -2 -1 0 1 2 3 4
-1
0
1
c
o
s
(
t
)
-4 -3 -2 -1 0 1 2 3 4
-1
0
1
c
o
s
(
2
t
)
-4 -3 -2 -1 0 1 2 3 4
-1
0
1
c
o
s
(
3
t
)



The Walsh functions, W[n, t] , n 1, form a complete orthogonal sequence on the interval

[ 0, 1 ) and they have square wave shapes in which each Walsh function W[ n, t ] takes on

only the value 1 and +1. For any function x(t) which has period 1 and is Lebesgue inte-

grable on [0,1), it can be decomposed into an infinite sum of the Walsh functions.

x(t)

=0 i
a
i
W[i, t], (2.19)

where a
i =

1
0
x(t) W[i, t] dt , i=0,1,2, (2.20)


For equally spaced time series data x=( x(0), x(1), , x(N-1)) and N is a power of 2, the




12
discrete Walsh Transform is defined by

A
k
=
N
1

=
1
0
], , [ ) (
N
i
i k w i x k=1, 2, , N-1. (2.21)

where w [k, i] =W[k, i/N] , i =1, 2, .N-1. and the inverse transform is given by

x(i) = ] , [
1
0
k i w A
N
k
k

=
, i=1, 2, , N-1. (2.22)

There are many similarities between Fourier analysis and Walsh-Fourier analysis, also

there are differences. Unlike sinusoidal functions, the Walsh functions are not periodic,

and thus, the Walsh-Fourier decomposition is more difficult to interpret than the Fourier

decomposition.



2.3 Wavelet analysis


Wavelet analysis is to decompose a given function x(t) L
2
into a sum of wavelet func-

tions. It involves a mother wavelet (t), which may be any real or complex continuous

function that satisfies certain conditions, such as

(t)dt=0 and

|(t)|
2
dt . Then,
wavelets are themselves derived from their mother wavelet (t) by translations and dila-

tions.


The Haar function can be a mother wavelet defined by


(t) =

<
<
. , 0
1 2 / 1 , 1
2 / 1 0 , 1
otherwise
t for
t for
(2.23)







13
Another commonly used wavelet is Morlet wavelet defined as

(t) =e
2
t
cos( t 2 ln / 2 ) e
2
t
cos(2.885 t). (2.24)

Four different mother wavelets: Haar, Daublet, Symmlet, and Coiflet are shown in Figure

2.3, where the first letter of the wavelet indicates the name: d for Daublet, s for Symmlet,

and c for Coiflet; the number of the wavelet indicates its width and smoothness. See

Bruce and Gao (1996, p.8).



Figure 2.3. Four different mother wavelets
`haar' mother, psi(0,0)
0.0 0.2 0.4 0.6 0.8 1.0
-
1
.
0
-
0
.
5
0
.
0
0
.
5
1
.
0
`d4' mother, psi(0,0)
-1.0 -0.5 0.0 0.5 1.0 1.5 2.0
-
1
.
0
0
.
0
1
.
0
`s12' mother, psi(0,0)
-4 -2 0 2 4 6
-
1
.
0
-
0
.
5
0
.
0
0
.
5
1
.
0
1
.
5
`c12' mother, psi(0,0)
-4 -2 0 2 4 6
-
0
.
5
0
.
0
0
.
5
1
.
0
1
.
5





Given a mother wavelet (t), an infinite sequence of wavelets can be constructed by

varying translations b and dilations a as below


a,b
(t) =|a|
-1/2
(
a
b t
). (2.25)

By defining the continuous wavelet transform W(a,b) as

W(a,b) =<x(t),
a,b
(t) >=

x(t)
a,b
(t)dt , (2.26)




14
we can represent x(t) as

x(t) =
1
1
C

a
-2
W(a,b)
a,b
(t)dadb (2.27)

where C
1
= dt

2
| ) ( |
and ()=

(t)e
-it
dt .

When a and b take on discrete sets of values, we can similarly obtain the discrete wavelet

transform as

W(m,n) =<x(t),
m,n
(t)>=

x(t)
m,n
(t)dt, (2.28)

and x(t) =

= m

= n
W
m,n

m,n
(t). (2.29)

For an equally spaced time series data x =(x(1), x(2), , x(N) ), we can take approximate

wavelet transforms by replacing the (2.28) by an estimate such as:

W(m,n) =

x(t)
m,n
(t)dt

=
N
l
l x
1
) (
m,n
( l ). (2.30)

It follows that a class of discrete wavelet transform (DWT) for equally spaced time series

data can be implemented by using an efficient computational algorithm. See Bruce and

Gao (1996, p37-39).


An example of wavelet approximation is given in Figure 2.5. In the example, the signal

is ss =10*cos( *t/15)+3*cos( *t/10), which is plotted in Figure 2.4. Its wavelet appro-

ximation is plotted in Figure 2.5.






15
Figure 2.4 The signal ss=10*cos( *t/15)+3*cos( *t/10)
Index
s
s
0 10 20 30 40 50 60
-1
0
-5
0
5
1
0






Figure 2.5 Decomposition of ss into the sum of 16 wavelet functions
D3.3
D3.5
D1.1
D3.4
D3.1
D2.1
D3.8
S5.2
D3.6
D3.2
D5.2
D4.3
D4.2
D5.1
D4.1
D4.4
Approx
0 10 20 30 40 50 60



Another example is a wavelet decomposition for the Doppler signal as shown in Figure
2.6. See Bruce and Gao (p.28). In Figure 2.6, the high frequency oscillations at the begin-
ning of the signal are captured mainly by the fine scale detail components D1 and D2,
while the lower frequency oscillations are captured mainly by the coarse scale compon-
ents D6 and S6.



16

Wavelet analysis is very powerful and efficient in the analysis of data or functions, x(t)

with gradual frequency changes. However, wavelets are not periodic functions. For

example, the Morlet wavelet is Fourier based but its oscillations are dampened by the

exponential factor e
2
t
. In addition, the concepts of frequency and periodicity have no

precise meaning in wavelet analysis. See Priestley (1996).


Figure 2.6 Doppler(t)=((t*(t-1))**0.5)*sin(2.1* /(t+0.05))
S6
D6
D5
D4
D3
D2
D1
Data
0 200 400 600 800 1000












17
2.4 The spectral analysis methods for unequally-spaced data


2.4.1 Interpolation Methods

The DFT cant be applied directly to time series with unequally-spaced data or missing

Data. This is due to the fact that formal DFT requires an orthogonal sinusoidal system,

however, there is no such an orthogonal sinusoid system for unequally-spaced time series

data. For overcoming this limitation of the DFT due to unequally- spaced data, many

methods have been suggested. Among them there are interpolation schemes, which are to

interpolate the data to equal spacing. ( e.g., Edelson and Krolik 1988; Mayo, Shay, and
Riter 1974 ). The classical interpolation method is to find a function which approximates
another function. The approximations are derived from the observations by the least-
squares method. These approximation methods are often interpolation polynomials. The
interpolation polynomials for unequal intervals are Lagrange, Newton divided difference;
the interpolation polynomials for equal intervals are often spline functions.

The concept of the nonuniform discrete Fourier transform (NDFT) has been developed by
using interpolation methods. For expressions (2.31) through (2.39) below, see Marvastic
(2001, p326-329).

Definition. The nonuniform discrete Fourier transform ( NDFT ) of a sequence x[n] of

length N is defined by

X(z
k
) =X(z)|
k
z z=
=

1
0
] [
N
n
n
k
z n x , k =0, 1, , N-1, (2.31)

where X(z) is the z-transform of x[n], and z
0
, z
1
, , z
1 N
are distinct points located

arbitrarily in the z-plane.



18


(2.31) can be expressed as a matrix form by

X =Dx , (2.32)

where

X =

) (
) (
) (
1
1
0
N
z X
z X
z X

, x =

] 1 [
] 1 [
] 0 [
N x
x
x

, (2.33)


and
D =

) 1 (
1
2
1
1
1
) 1 (
1
2
1
1
1
) 1 (
0
2
0
1
0
... 1
... 1
... 1
N
N N N
N
N
z z z
z z z
z z z

. (2.34)


If the N sampling points, z
0
, z
1
, , z
1 N
are distinct, then D is nonsingular, and thus the

inverse of NDFT can be determined by

x =D
1
X . (2.35)

If the N sampling points, z
0
, z
1
, , z
1 N
are equally-spaced angles on the unit circle in

the z-plane, then ( 2.35 ) corresponds to the classical DFT. X(z) can be expressed as the

Lagrange polynomial of order N-1,

X(z) = ], [
) (
) (
1
0
k
N
k k k
k
z X
z L
z L

=
(2.36)

where L
0
(z), L
1
(z), , L
1 N
(z) are the fundamental polynomials, defined by

L
k
(z) =

k i
i
z z ), 1 (
1
k =0, 1, , N-1. (2.37)

X(z) can also be expressed as the Newton interpolation,



19

X(z) =c
0
+c
1
(1-z
0
z
1
)+c
2
(1-z
0
z
1
)(1-z
1
z
1
)++c
1 N

2
0
1
) 1 (
N
k
k
z z , (2.38)

where c
0
=X[z
0
],

c
1
=
1
1 0
0 1
z - 1
] [

z
c z X
,

c
2
=
) 1 )( 1 (
) 1 ( ] [
1
2 1
1
2 0
1
2 0 1 0 2

z z z z
z z c c z X
,

(2.39)


Note that in (2.39), each
j
c depends only on X[z
0
] , X[z
1
], , X[z
j
] and z
0
, z
1
, , z
j
.


While interpolation methods may be satisfactory in some applications, they all produce
some distortion and loss of information, see Scargle ( 1989 ), and they may cause some
distortion in the spectrum, especially for the data with high frequency components. In
addition, these interpolation methods cant yield an orthogonal and additive spectrum
ANOVA decomposition for the original time series data.


2.4.2 Least-Squares Methods

The least squares method is used on Fourier expansion or the inverse transform to find

the period which minimizes the unexplained variance of the series. This method is very

similar to multivariate regression analysis when multiple periods are present. In the least-
squares sense, a periodogram analysis is based on the trigonometric regression model
x
f
(t) =

=
K
k
P
k
A
1
( cos(2 t/P
k
) +B
k
P
sin(2 t/P
k
)) + , (2.40)



20
where K is the smallest integer less than or equal to (N-1)/2 and P
k
=N/k, k =1, 2, ..., K.
To minimize the mean square difference between (2.40) and the data, one seeks to mini-
mize

=
N
j 1
[x(t
j
)-x
f
(t
j
)]
2
, where t
1
, ..., t
N
are not necessarily equally-spaced, and t
N
=
N. Thus A
k
P
and B
k
P
may be determined by standard linear least-squares techniques.

Some other least-squares methods can be found. The classical DFT power spectrum,
periodogram is defined by (See Brockwell and Davis 1993, p332)

I(
k
) =
N
1
|
k
it
N
t
t
e X

=

1
|
2


=
N
1
[
2
1
) sin ( t X
t
N
t
t

=
+
2
1
) cos ( t X
t
N
t
t

=
]. (2.41)

For unequally-spaced data, t
1,
t
2, ,
t
N
are not equally-spaced points. Scargle (1982, 1989),

and Lomb(1976) defined a modified periodogram by

I
*
( ) =
2
1
{

=
=

N
j
j
N
j
j j
t
t X
1
2
1
2
) ( cos
)] ( cos [


+

=
=

N
j
j
N
j
j j
t
t X
1
2
1
2
) ( sin
)] ( sin [


}, (2.42)

where is defined by

tan(2) =
j
N
j
t

=1
2 sin /
j
N
j
t

=1
2 cos . (2.43)

They showed that their modified periodogram and least squares fitting of sinusoidal

waves to the data are exactly equivalent. This method has been called as Lomb-Scargle

method , which has recently been used in biomedical sciences. See Schimmel ( 2001 ),
Van Olofsen, VanHartevelt, and Kruyt (1999), Ruf(1999), Schluz and Stattegger (1997).



21
The least-squares method can detect some frequency components in relatively simple
sinusoidal spectral situations, but for complicated spectra, it encounters more difficulties.
See Swan ( 1982 ). When time series data contain fractions of non-Gaussian noise or con-
sist of periodic signals with non-sinusoidal patterns, Lomb-Scargle method makes more
difficult the interpretation of analysis results, and can lead to misleading estimates of fre-
quency components. See Schimmel ( 2001 ).


2.4.3 Linear Algebra Method

From the classical DFT definition , X
*
(k), the Fourier transform of X(t) is defined by

X
*
(k) =

N
t
N kt i
e t X
1
/ 2
) (

, (2.44)

where k =-(N-1)/2, , (N-1)/2, if N is odd ,

k =-N/2, , N/2, if N is even.

(2.44) can be written as in matrix form by


*
X =W
N X
(2.45)

and
X
=W
1
N
*
X . (2.46)

For unequally spaced data
u
X , Kuhn (1982 ) and Swan (1982 ) defined the DFT expan-

sion similar to (2.46) by


u
X =W
u
1
N
*
X , (2.47)

where W
u
1
N
is a matrix function of the unequally spaced data
u
X and
*
X is the DFT of

certain unknown equally spaced data
X .
Substituting (2.45) into (2.46) gives


u
X =W
u
1
N
W
N X .
(2.48)




22
Solving (2.48) gives
X
, and doing DFT on
X
gives
*
X , then , Kuhn(1982) and Paul R.

Swan ( 1982 ) used
*
X to do spectrum analysis of the data. However, the method is

limited in that the deterministic component of noisy signals must be band limited to less

than the usual Nyquist limit. See Swan (1982).


In addition, the other methods include the string length methods, phase dispersion
minimization method, and the CLEAN algorithm; see Rao, Priestley, and Lessi (1997,
p275-286 ). Regarding the CLEAN algorithm, also see Baisch and Bokelmann. ( 1999).
Other discussions and methods for unequally-spaced time series are in Barthes ( 1995 ),
Engle and Russell (1998), and Good and Doog (1958).

The methods above in Section 2.4.1 through 2.4.3 are Fourier based; they also suffer the

shortcoming of Fourier spectral analysis, i.e., they are not efficient for non-sinusoidal

waveforms, and they might lead to misunderstandings of the spectral ANOVA .














23
3. Adaptive Discrete Fourier Analysis


3.1 Motivation


The purpose of the research is to generalize the Fourier analysis of the digital data x. The
generalization begins by replacing each space A
k
in the representation of L
N A,
in ( 2.10)
with a larger space B
k
; and replacing an equally-spaced digital time series data set x =
(x(1),, x(N)) in (2.10) with an arbitrary digital time series data set x =(x(t
1
), , x(t
N
)),
where t
1
, , t
N
are not necessarily equally-spaced and t
N
=N. While each function s in
A
k
has real and imaginary parts that are sinusoids, the real and imaginary parts of func-
tions in B
k
have more general periodic shapes. Like sinusoids, these shapes satisfy (3.1)
below.
s(t) =- s(t-
k
N 2
). (3.1)

This means that the second half of the periodic shape of s(t) for
k
N
2
t
k
N
is the reflec-
tion of its first half about the origin. For the general shapes in B
k
, the first half of the
periodic shape, s(t) , can be any shape at all, provided

k
N
t s
2
0
2
| ) ( | dt . This genera-
lization of Fourier analysis, called adaptive discrete Fourier analysis, is accomplished
via the periodic step functions f
j n k , ,
defined by Foutz and Lee ( 2000 ) instead of sinusoid
functions and via their properties as follows.








24
3.2 The periodic step functions f
j n k , ,



When k >0, the periodic step function sin(
2

t 2
) jumps to 0, 1, and 1 at the times t
where sin( t )=0, 1, and 1 respectively. It follows that the complex valued step function
exp(i
2

t 2
) =cos(
2

t 2
) +i*sin(
2

t 2
) (3.2)
jumps to 1, i, -1 and i with frequency , See Figure 3.1 below. The function exp( i
2

+u
t

2
) is a version of (3.2) that is shifted backwards by

2
u
time periods. For k >0
and
k
=
N
k 2
, let particular time-shifted versions of (3.2) be denoted by
f
j n k , ,
(t) =exp(i
2

+ 1
1 2 2
n
j t
k

),
f
j n k , ,
(t) =exp(i
2

+
n
j t
k
1 2 2

),
f
j n, , 0
(t) =1, for k =0; where j =1, 2, ,n. (3.3)
The functions f
j n k , ,
(t) and f
j n k , ,
(t) are complex conjugates. The appearance of some of
these functions is shown in Figure 3.2.










25
Figure 3.1. The periodic step function f
j n k , ,
in the complex plan
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
The periodic step function in the complex plane














26
Figure 3.2. the step function f
j , 6 , 2
(t)
-3 -2 -1 0 1 2 3
-1
0
1
the s tep function with k=2, n=6
j

=
3
,
i
m
a
g
-3 -2 -1 0 1 2 3
-1
0
1
j

=
6
,
r
e
a
l
-3 -2 -1 0 1 2 3
-1
0
1
j

=
6
,
i
m
a
g

-3 -2 -1 0 1 2 3
-1
0
1
the step function with k=2, n=6
j

=
2
,
r
e
a
l
-3 -2 -1 0 1 2 3
-1
0
1
j

=
2
,
i
m
a
g
-3 -2 -1 0 1 2 3
-1
0
1
j

=
3
,
r
e
a
l




27
3.3. The properties of the periodic step functions, f
j n k , ,


We notice that the following results are true.
3.2.1 The functions { f
1 , ,n k
, , f
n n k , ,
} can span a subspace, to be called B
n k,
of L
2
.
Note that all the functions in B
n k,
are periodic with period
k
2
and each function
s(t) in B
n k,
satisfies (3.1), since each of f
1 , ,n k
, , f
n n k , ,
has this property.
3.2.2 Let
L
n K B , ,
=B
n K,
+ +B
n K,
, (3.4)
then L
n K B , ,
is the direct sum of { B
n K,
, , B
n K,
} and L
n K B , ,
L
2
, see Foutz
and Lee (2000).
3.2.3 Let B
k
=
n
lim B
n k,
for k =1, , K, and L
K B,
=
n
lim L
n K B , ,
, then it follows from
(3.4) that
L
K B,
=B
K
+ +B
K
. (3.5)

In adaptive discrete Fourier analysis, we consider replacing L
N A,
in ( 2.10 ) with L
K B,
in
(3.5) and using (3.4) to approximate (3.5) as discussed in the following sections.

3.2.4. The Theorem for more general periodic functions.

Theorem 3.1: Any function s(t) in L
2
that satisfies s(t) = - s(t-
k

) is the limit of step

functions s
n
(t) in B
n k,
as n , in the L
2
sense , i.e.

n
lim dt t s t s
n

2
| ) ( ) ( | =0.



28
The proof of the Theorem 4.1 is given in Appendix 1. This theorem is the first and most
important result in this research. This is due to the fact that the following Corollary 3.1,
and the property of s
m n,
are based on this theorem, and the adaptive Fourier analysis of x
proceeds by investigating the properties of s
m n,
(t).
3.2.5. The decomposition of L
2
.
Corollary 3.1. L
2
has another decomposition as
L
2
= +B
1
+B
0
+B
1
+,
where B
k
is the subspace of L
2
that contains all periodic functions that have the
form s
] [k
(t), in which s
] [k
(t) =- s
] [k
(t-
k

), for k 0; and B
0
={c, c is complex-valued
scale}.
The proof of the Corollary 3.1 is also given in Appendix 1.


3.4. Multiresolution analysis



The sequence of spaces { L
n K B , ,
}
Z K
represents a ladder of subspaces of increasing
resolution as K increases, and it has the following properties:
1. L
n B , 1 ,
L
n B , 2 ,
;
2. (
h
Z K
L
n K B , ,
)=L
n B , 0 ,
={ 1 };
3.
n
lim(

Z K
L
n K B , ,
) =L
2
.



29
The first property comes directly from the definition of L
n K B , ,
, the second and third pro-
perties follow from the paper by Foutz and Lee . See Foutz and Lee (2000). Here { L
n K B , ,

}
Z K
forms a multiresolution analysis ( MRA ) of L
2
.

It follows that if K
1
<K
2
, then the orthogonal projection x
1
(t) L
n K B , ,
1
of a function
x(t) in L
2
is coarser approximation of x(t) than the projection x
2
(t) L
n K B , ,
2
.

Since L
n K B , ,
is the subspace of L
2
spanned by { f
j n k , ,
(t) : k=-K, , 0, 1, , K; j =1, 2,
, n } and { L
n K B , ,
}
Z K
constitute a MRA of L
2
, the step functions { f
j n k , ,
(t) : k =-K,
, -1, 0, 1, , K; j =1, , n } can be said to generate a multiresolution analysis ( MRA
) of L
2
.



3.5. The adaptive Fourier analysis of the digital time series


The adaptive Fourier analysis of the digital time series, x =( x(t
1
), , x(t
N
) ), proceeds
in the following steps:
3.5.1. Attention is focused on the subspace,

L
K B,
=B
K
+ +B
K


of L
2
. This subspace is then approximated by a sequence of subspaces,

L
n K B , ,
=B
n K,
+ +B
n K,
.




30
3.5.2. The digital data set, x =(x(t
1
), , x(t
N
)) is replaced by a continuous-time function
x
e
(t ) in B
1
+B
0
+B
1
. It is required that x
e
(t
j
) =x (t
j
), at j =1, 2,, N.
Theorem 3.2 For digital data set x =( x(t
1
), x(t
2
), , x(t
N
) ) , where t
1
, , t
N
are not
necessarily equally-spaced and t
N
=N, there always exists a step function x
e
(t) L
1 , B
=
B
1
+B
0
+B
1
such that
x
e
(t
j
) =x (t
j
) at j =1, 2,, N.
The proof of this theorem is in Appendix 2.


3.5.3 x
e
(t ) is projected onto L
n K B , ,
by using a weight inner product <y, z>
m , 3
.

3.5.3.1. Construct a weight inner product

The weight function
w
m
(t) =

+
+
+
otherwise
t t t if m
t t t if m
t t t if m
m
m N m N
m m
m m
,
,
,
,
1
2
1
2
1
2
1
2 2
1
2
2
1
1 2
1
1
(3.6)
steps up to the value m near t =t
1
, , t
N
and approaches 0 elsewhere. w
m
(t) approaches
a Dirac -function, (t), as m . It is used to construct the inner product
<y, z>
m , 3
=
N
1

N
m
dt t z t y t w
0
*
) ( ) ( ) (
with the properties:

m
lim
N
1
dt t z t y t w
N
m

0
2
| ) ( ) ( | ) ( =
N
1
2
1
| ) ( ) ( |

N
j
j j
t z t y , (3.7)



31
and
m
lim<y, y>
m , 3
=
N
1

=
N
j
j
t y
1
2
) ( , (3.8)
whenever y and z are in L
n K B , ,
.

3.5.3.2 The projection
The inner product <y, z>
m , 3
is used to project x
e
(t) onto L
n K B , ,
for any K 1. The projec-
tion, s
m n,
=P
m , 3
( x
e
| L
n K B , ,
), has the property below
x(t) =
n
lim
m
lim s
m n,
(t) (3.9)
at each t =t
1
, , t
N
. This is proven in Appendix 3. The adaptive Fourier analysis of
x proceeds by investigating the properties of s
m n,
(t) for increasing n and for large m.

3.5.4 Adaptive Fourier series representation and ANOVA

The spaces B
n k,
are not orthogonal with respect to the usual inner products <y,z>
1
, <y,
z>
2
and <y, z >
m , 3
. However, the spaces are orthogonal with respect to a nonstandard
inner product <y, z>
4
that is defined in Appendix 4. It follows that s
m n,
has an adaptive
Fourier series representation
s
m n,
(t) =

=
K
K k
m n k
s
, ,
(t), (3.10)
where s
m n k , ,
(t) =P
4
( s
m n,
| B
n k,
) is the projection of s
m n,
onto B
n k,
with respect to <y,
z>
4
. The corresponding adaptive Fourier series representation for x is
x(t) =
n
lim
m
lim

=
K
K k
m n k
s
, ,
(t), (3.11)



32
at each t =t
1
, , t
N
. Finally, Appendix 5 shows that the accompanying adaptive Fourier
ANOVA partition of |s
m n,
(t)|
2
4
=<s
m n,
, s
m n,
>
4
is
|s
m n,
(t)|
2
4
=

=
K
K k
m n k
t s
2
4 , ,
| ) ( | = dt s
N
K
K k
m n k
N

=
2
, ,
0
| |
1
. (3.12)
The components s
m n k , ,
(t) +s
m n k , ,
(t) in (3.10), (3.11), and (3.12) are periodic with fre-
quency
k
. Their shapes are not necessarily sinusoidal, but they may have any of the
general periodic shapes in B
n k,
. The shapes are called adaptive since they depend on the
data x through the projections s
m n,
, s
m n k , ,
and s
m n k , ,
. In addition, the shapes may differ
at each frequency. Most importantly, the adaptive Fourier analysis in (3.10), (3.11), and
(3.12) provides a means for investigating the shape of the periodic component at fre-
quency
k
in the time series data x, by plotting s
m n k , ,
(t) +s
m n k , ,
(t).

3.6 Methodology


Given a digital data set x =( x(t
1
), x(t
2
), , x(t
N
) ), we center the data so that

) (t x =
0 and so that the index k =0 may be omitted from (3.10), (3.11), and (3.12). Write D =
2Kn, and again label D of the functions from (3.3) as (3.13) below.




33
v =

+
+
D
n D
n
n
n
v
v
v
v
v
v
o
o
o
o
1
2
1
1
=

+
+

n n K
n K
n n K
n K
n n K
n K
f
f
f
f
f
f
, ,
1 , ,
, , 1
1 , , 1
, ,
1 , ,
o
o
o
o
. (3.13)


The functions v
1
, , v
D
are shown to be linearly independent in Foutz and Lee (2000).
Therefore, they may be used in the Gram-Schmidt process to construct the matrix M
D

and the resulting column vector
w =M
D
v. (3.14)

The Gram-Schmidt process first picks M
1
=
m
v
, 3 1
| |
1
. It proceeds iteratively to construct

M
J
=

g
J
C
M
1
1
* 0
,

where 0* is the column of J -1 zeros, and the constant g is chosen so that |w
J
|
m , 3
=1. The
(J -1)-dimensional row vector C satisfies
gC =- (<v
J
, v
1
>
m , 3
, , <v
J
, v
1 J
>
m , 3
)M
*
1 J
M
1 J
,
where M
*
1 J
is the complex conjugate of M
1 J
.

Since the orthonormal functions w
1
(t), , w
D
(t) span L
n K B , ,
, it follows that the projec-
tion s
m n,
(t) =P
m , 3
( x
e
| L
n K B , ,
) is



34
s
m n,
(t) =

=
D
r
r r
t w d
1
) ( , (3.15)
where d
r
=<x
e
, w
r
>
m , 3
. (3.16)

Let a
s r,
be the element of M
D
in the r
th
row and s
th
column, and let a
*
,s r
be its complex
conjugate. Substituting M
D
v for w in (3.15) and (3.16) gives
s
m n,
(t) =

= =
D
r
s s r r
r
s
t v a d
1
,
1
) ( =

= =
D
s
s s r r
D
s r
t v a d
1
,
) ( , (3.17)
and d
r
=
m
r
s
s e s r
v x a
, 3
1
*
,
, > <

=
. (3.18)

Since v
1 ) ( + + n K k
(t), , v
n n K k + + ) (
(t) in (3.13) and (3.17) span B
n k,
; and since B
n K,
, ,
B
n K,
are orthogonal with respect <y, z >
4
, it follows from ( 3.17 ) that the projection
P
4
( s
m n,
| B
n k,
) is given by
s
m n k , ,
(t) =P
4
( s
m n,
| B
n k,
) =

+ +
+ + = =
n n K k
n K k s
s s r r
D
r
t v a d
) (
1 ) (
,
1
) ( . (3.19)

Definition. Fix a positive integer n. The adaptive discrete Fourier transform (ADFT ) of
the continuous-time extension x
e
(t) of the digital time series x is the set of n N scalars

j n k , ,
=<x
e
, f
j n k , ,
>
m , 3

for j =1, 2, , n and K k K.

Property. ADFT
j n k , ,
=<x
e
, f
j n k , ,
>
m , 3
are relative robust to the construction of x
e
(t),
they only require that x
e
(t
i
) =x(t
i
) for i =1, 2, ... , N.



35
Proof: For every x
e
(t) satisfying x
e
(t
i
) =x(t
i
), for i =1, 2, ... , N,

j n k , ,
=
N
1

N
0
x
e
(t) f
j n k , ,
(t)w
m
(t)dt, and

m
lim
j n k , ,
=
N
1

=
N
i 1
x(t
i
) f
j n k , ,
(t
i
).







































36
4. Examples

Example 4.1 The time series data set x(1), x(2), , x(72) in Table 4.1 and Figure 4.1 is
taken from Brockwell and Davis (1991, p7), It contains the monthly accidental deaths in
the U.S.A. from J anuary, 1973 through December, 1978.

Table 4.1 Monthly Accidental Deaths in the U.S.A., 1973-1978

1973 1974 1975 1976 1977 1978
J an. 9007 7750 8162 7717 7792 7836
Feb. 8106 6981 7306 7461 6957 6892
Mar. 8928 8038 8124 7776 7726 7791
Apr. 9137 8422 7870 7925 8106 8129
May 10017 8714 9387 8634 8890 9115
J un. 10826 9512 9556 8945 9299 9434
J ul. 11317 10120 10093 10078 10625 10484
Aug. 10744 9823 9620 9179 9302 9827
Sep. 9713 8743 8285 8037 8314 9110
Oct. 9938 9129 8433 8488 8850 9070
Nov. 9161 8710 8160 7874 8265 8633
Dec. 8927 8680 8034 8647 8796 9240


0 10 20 30 40 50 60 70 80
6500
7000
7500
8000
8500
9000
9500
10000
10500
11000
11500
F igure 4. 1 M onthly acc idental deaths in U. S . A . , 1973-1978




37

(1). In Table 4.2, the classical Fourier ANOVA reveals that the time series contains a
major seasonal component with period 12 , and the adaptive Fourier ANOVA reveals
the same major frequency component too. In adaptive Fourier ANOVA, the frequency
number K =6 is used for the space L
n K B , ,
=B
n K,
+... +B
n K,
; the dimension n =2 is used
for the spaces B
n k,
and B
n k ,
where k=1,2, ...,K; the time series, x =(x(1), x(2), ..., x(72))
is extended to the step function x
e
(t) in L
2
; the value m =1000 is used for the inner pro-
duct <y, z >
m , 3
, and this inner product is used to project x
e
(t) onto L
n K B , ,
, resulting in
the adaptive Fourier series,
P
m , 3
( x
e
(t)| L
n K B , ,
)(t) =

=
K
K k
m n k
s
, ,
(t).
The resulting adaptive Fourier ANOVA is as below:
|P
m , 3
( x
e
(t)| L
n K B , ,
)|
2
4
=
2
4 , ,
| |

=
K
K k
m n k
s ,
which is also given in Table 4.2, again shows that the adaptive frequency component for
k =6 , s
m n, , 6
+s
m n, , 6
is the major frequency component . The major adaptive frequency
component s
m n, , 6
+s
m n, , 6
is plotted on Figure 4.2; and the power distribution for adaptive
Fourier analysis is plotted on Figure 4.3, while the one for Fourier analysis is plotted on
Figure 4.4. The results show that for this equally-spaced time series, x =( x(1), x(2), ... ,
x(72)), Fourier ANOVA and adaptive ANOVA both can reveal the same major frequency
component with k =6 . However, the component in Fourier ANOVA is sinusoidal shape,
while adaptive frequency component with k =6 is not sinusoidal shape shown in Figure
4.2.



38


Table 4.2 *10^5
K Period

k
N

ANOVA
|P
2
(x|A
k
)|
2
2
+|P
2
(x|A
k
)|
2
2

Adaptive
|s
m n k , ,
|
2
4
+|s
m n k , ,
|
2
4


k=1 72 1.2080 1.0852
k=2
36 0.1747 0.3414
k=3 24 0.3197 0.1907
k=4 18 0.1009 0.6844
k=5 72/5 0.0492 0.1493
k=6 12 5.2262 6.0817
Total 7.0787 8.5327



Figure 4.2 Frequency component with k=6 in Example 4.1





39
1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6
0
2
4
6
8
x 10
5
Figure 4.3: Adaptive Fourier line spectrum for Example 4.1
1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6
0
2
4
6
x 10
5
Figure 4.4 Fourier line spectrum in Example 4.1



(2). To illustrate that adaptive Fourier analysis can be applied to unequally-spaced time
series data, we delete the Marchs records, then the data is reduced to an unequally-spa-
ced time series with 66 sample points, x =( x(t
1
), x(t
2
), ..., x(t
66
)). In this case, classical
Fourier ANOVA cant be used. However, adaptive Fourier ANOVA can be used to re-
veal the frequency component shapes and discovering periodicity. Similar to the above,
we pick K =6, n =2 and m=1000 respectively. The resulting adaptive Fourier ANOVA is
given in Table 4.3, it also reveals that this unequally-spaced time series has the same
major frequency component with k =6. The time series x =( x(t
1
), x(t
2
), ... , x(t
66
) ) is
plotted on Figure 4.5 and the major frequency component s
m n, , 6
+s
m n, , 6
is plotted on
Figure 4.6.




40

Table 4.3 *10^5
K Period

k
N

Adaptive ANOVA
|s
m n k , ,
|
2
4
+|s
m n k , ,
|
2
4

k=1 72 1.1436
k=2 36 0.7401
k=3 24 0.1260
k=4 18 0.4762
k=5 72/5 0.1795
k=6 12 6.5532
Total 9.2187







(3). If we take J anuarys record out from the first year and Februarys record out from
the second year, , J unes record out from the last year, then the data will be reduced to
an another unequally-spaced time series with 66 sample points, x =( x( t
1
), x( t
2
), ...



41
, x(t
66
) ). In this case, adaptive Fourier ANOVA can be also used to reveal the frequency
component shapes and discovering periodicity. Similar to the above, we pick K=6, n =2
and m =1000 respectively. The resulting Adaptive Fourier ANOVA is given in Table 4.4.
It also reveals that this unequally-spaced time series has the same major frequency com-
ponent with k=6. The time series x =( x(t
1
), x(t
2
), ..., x(t
66
)) is plotted on Figure 4.7 and
the major frequency component s
m n, , 6
+s
m n, , 6
is plotted on Figure 4.8.


Table 4.4 *10^5
K Period

k
N

Adaptive ANOVA
|s
m n k , ,
|
2
4
+|s
m n k , ,
|
2
4

k=1 72 1.1122
k=2 36 0.5772
k=3 24 0.1494
k=4 18 0.5806
k=5 72/5 0.1614
k=6 12 5.8413
Total 8.4222






42




(4). If we take randomly 10 records out from the original data set, then the data will be
reduced to an unequally-spaced time series with 62 sample points, x =( x(t
1
), x(t
2
), ... ,
x( t
62
) ). The resulting adaptive Fourier ANOVA is given in Table 4.5, it also reveals
that this unequally-spaced time series has the same major frequency component with k=6.
The time series x =( x( t
1
), x( t
2
), ... , x(t
62
) ) is plotted on Figure 4.9 and the major fre-
quency component s
m n, , 6
+s
m n, , 6
is plotted on Figure 4.10.










43
Table 4.5 *10^5
K Period

k
N

Adaptive ANOVA
|s
m n k , ,
|
2
4
+|s
m n k , ,
|
2
4

k=1 72 1.3207
k=2 36 0.3167
k=3 24 0.1788
k=4 18 0.6656
k=5 72/5 0.4468
k=6 12 6.5826
Total 9.5111






(5). We repeat (4) above again, and obtain similar results shown in Table 4.6 and Figure
4.11 and 4.12.







44
Table 4.6 *10^5
K Period

k
N

Adaptive ANOVA
|s
m n k , ,
|
2
4
+|s
m n k , ,
|
2
4

k=1 72 1.1929
k=2 36 0.4377
k=3 24 0.1794
k=4 18 0.5834
k=5 72/5 0.1726
k=6 12 6.8078
Total 9.3739







This example implies that when a small fraction of the original data, 10/72 14% in this
example, is removed or missed, the adaptive Fourier analysis can still detect the main
frequency components in the original time series data.




45
Example 4.2. To illustrate that for some signals with jumps, adaptive Fourier analysis
outperforms Fourier analysis, we take an equally-spaced sample s =(s
1,
s
2
, , s
192
) from
the signal s(t) with sample size N =192 to do adaptive Fourier analysis and Fourier analy-
sis. Here s(t) is given by


s(t) =

<
< +
<
+
. 192 144 ), 32 / 2 sin( 1
144 96 ), 32 / 2 sin( 1
96 48 , ) 32 / 2 sin( 1
48 0 ), 32 / 2 sin( 1
t t
t t
t t
t t

K =3, n =12, and m=1200 are used to do adaptive Fourier analysis. The resulting adaptive
Fourier ANOVA and Fourier ANOVA are together given in Table 4.2(1). They both re-
veal that the signal has a major frequency component with k =2. The sample signal is
plotted on Figure 4.2(1); the major adaptive Fourier frequency component, s
m n, , 2
+s
m n, , 2
,
is plotted on Figure 4.2(2); and the sample signal (.) and its major k =2 Fourier frequency
component (-) are together plotted on Figure 4.2(3).

The sample power,
192
1

=
192
1
2
i
i
s =2.7691. The major frequency component with k =2 in
Fourier ANOVA is sinusoidal shape as shown in Figure 4.2(3), and represents 79.5%
(=2.2004/2.7691) of total sample power, while the major frequency component with k =2
in adaptive Fourier ANOVA is not sinusoidal shape, but is very similar to the original
signal shape. By comparing Figure 4.2(1) and Figure 4.2(2), it represents 98.8%(2.7639/
2.7691 ) of the total sample power. In fact, the sample signal has only one frequency



46
component with k =2 and thus in Fourier ANOVA, 20.5% of the sample energy has
leaked onto a wide frequency range, which implies that in this case adaptive Fourier
analysis outperforms Fourier analysis.


Table 4.2(1)
K Period

k
N

ANOVA
|P
2
(x|A
k
)|
2
2
+|P
2
(x|A
k
)|
2
2

Adaptive ANOVA
|s
m n k , ,
|
2
4
+|s
m n k , ,
|
2
4


k=1 192 0.0000 0.0008
k=2
96 2.2004 2.7639
k=3 62 0.0000 0.0008
Total 2.2004 2.7647








47
0 1 2 3 4 5 6
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Figure 4.2(3): k=2 freq. component(-) in Fourier Analysis & original signal(.)
Fourier freq. comp.(k=2)
originial signal



To illustrate that adaptive Fourier analysis is different from wavelet analysis, we apply
wavelet analysis to the sample data s =( s
1,
s
2
, , s
192
) in Figure 4.2(1). The multiresolu-
tion decomposition of the signal is plotted in Figure 4.2(4) and the time-scale plot for the
signal is plotted on Figure 4.2(5). In wavelet analysis, fine scale wavelet functions usu-
all reveal where or when the signal has high frequency components in a loose sense.
However, in this case, all the fine scale wavelet functions focus on the discontinuous
points of the signal shown in Figure 4.2(4) and especially in Figure 4.2(5). Therefore, in
this case, wavelet analysis cant provide any information about the frequency component.




48
S5
D5
D4
D3
D2
D1
Data
0 50 100 150
Figure 4.2(4): Multiresolution decomposition of the signal


Figure 4.2(5). Time-scale plot for the signal
Time
1
/
S
c
a
l
e
0 50 100 150
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0














49
Example 4.3. To illustrate that for sinusoidal signal, with equally-spaced data, Fourier
analysis and adaptive Fourier analysis are almost the same, we take an equally-spaced
sample s =( s
1
, s
2
, ..., s
60
) from the signal s( t ) =10*cos(2 t/15) +3cos(2 t/10) over
[1, 30], sample size N =60. The sample series, s =( s
1
, s
2
, ... , s
60
) is plotted on Figure
4.3(1). The Fourier ANOVA in Table 4.3(1) reveals that the sample series contains a
major component k=2 and a minor one k =3. In adaptive Fourier ANOVA, the frequency
number K =3, the dimensions n =5 and m =500 are used, and the ANOVA result is also
given in Table 4.3(1), again showing that the adaptive frequency component for k =2,
namely s
m n, , 2
+ s
m n, , 2
, is the major component ; for k =3, s
m n, , 3
+ s
m n, , 3
is the minor
component. The power distribution for the first three frequencies of adaptive Fourier
analysis is plotted on Figure 4.3(3); the one for Fourier Analysis is plotted on Figure
4.3(4); s
m n, , 3
+s
m n, , 2
+s
m n, , 1
+s
m n, , 1
+s
m n, , 2
+s
m n, , 3
is plotted on Figure 4.3(2); s
m n, , 3

+s
m n, , 3
is plotted on Figure 4.3(5); and s
m n, , 2
+ s
m n, , 2
is plotted on Figure 4.3(6). Table
4.3(1) shows that Fourier and adaptive Fourier ANOVA are very close; Figure 4.3(5) and
Figure 4.3(6) show that major and minor adaptive frequency components both have a
sinusoidal shape. The results together show that for equally-spaced sampled series s =
(s
1
, s
2
, ..., s
60
) from a sinusoidal signal, Fourier ANOVA and adaptive ANOVA are very
close.





50







Table 4.3(1)
K period

k
N


ANOVA
|P
2
(x|A
k
)|
2
2
+|P
2
(x|A
k
)|
2
2

Adaptive ANOVA
|s
m n k , ,
|
2
4
+|s
m n k , ,
|
2
4

k=1 60 0.162 0.1744
k=2 30 47.206 46.8258
k=3 20 5.2238 5.2992
Total 52.5918 52.2994







51
1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3
0
10
20
30
40
50
Figure 4.3(4): Fourier line spectrum
1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3
0
10
20
30
40
50
Figure 4.3(3): Adaptive Fourier line spectrum






52

Example 4.4. To illustrate that adaptive Fourier analysis can be applied to unequally-
spaced data from a sinusoidal signal, we use the uniform distribution to take a random
sample s =( s(t
1
), s(t
2
), ..., s(t
60
)) from the signal s(t) in Example 4.3 above. The sample
size is N =60. The sample series s =( s(t
1
), s(t
2
), ... , s(t
60
) ) is plotted on Figure 4.4(1).
Similarly to Example 4.3, we pick K =3, n =7 and m =500 respectively. The resulting
adaptive Fourier ANOVA is given in Table 4.4(1), and plotted on Figure 4.4(1); s
m n, , 3
+
s
m n, , 2
+s
m n, , 1
+s
m n, , 1
+s
m n, , 2
+s
m n, , 3
is plotted on Figure 4.4(2); s
m n, , 2
+s
m n, , 2
is plotted
on Figure 4.4(3); and s
m n, , 3
+s
m n, , 3
is plotted on Figure 4.4(4). Table 4.4(1) shows that
adaptive Fourier analysis can reveal the two frequency components with k =2 and k =3.
Figure 4.4(4) and Figure 4.4(5) show that these two frequency components are sinu-
soidal .

Table 4.4(1)
K period

k
N


Adaptive ANOVA
|s
m n k , ,
|
2
4
+|s
m n k , ,
|
2
4

k=1 60 0.1624
k=2 30 49.4853
k=3 20 3.5624
Total 53.2102





53


1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3
0
5
10
15
20
25
30
35
40
45
50
Figure 4.4(3): Adaptive Fourier line spectrum




54





Example 4.4 implies that for an unequally-spaced time series where the observation times
follow uniform distribution , the adaptive Fourier analysis can still detect the main fre-
quency components in the original time series data.


Example 4.5. To illustrate that for non-sinusoidal signal, adaptive Fourier analysis out-
performs Fourier analysis, we take an equally-spaced sample s =( s
1
, s
2
, ..., s
60
) from
the signal s(t) with sample size N =60 to do adaptive Fourier analysis. The signal s(t) is
given by




55
s(t) =

<
<
<
<
<
<
. 6 5 , 2
5 4 , 1
4 3 , 2
3 2 , 2
2 1 , 1
1 0 , 2
t for
t for
t for
t for
t for
t for


Here K =2, n =15, and m =500 are used to do the analysis, and the resulting adaptive
Fourier ANOVA is given in Table 4.5. The sampled series s =(s
1
, s
2
, ... , s
60
) is plotted
on Figure 4.5(1); s
m n, , 2
+ s
m n, , 1
+s
m n, , 1
+ s
m n, , 2
is plotted on Figure 4.5(2); the power
distribution for adaptive Fourier analysis is plotted on Figure 4.5(3); the power distribu-
tion of the first three frequency components for Fourier analysis is plotted on Figure
4.5(4); s
m n, , 2
+ s
m n, , 2
is plotted on Figure 4.5(5); and s
m n, , 1
+s
m n, , 1
is plotted on Figure
4.5(6). Figure 4.5(2) shows that the signal reconstruction from the sampled series is very
good. Table 4.5(1) and Figure 4.5(4) show that Fourier analysis results in the power
spreading over a wide frequency range, and Figure 4.5(6) shows that adaptive Fourier
analysis is able to reveal the real shape of the frequency component.

Table 4.5(1)
K period

k
N


ANOVA
|P
2
(x|A
k
)|
2
2
+|P
2
(x|A
k
)|
2
2

Adaptive ANOVA
|s
m n k , ,
|
2
4
+|s
m n k , ,
|
2
4

k=1 60 1.8254 2.9940
k=2 30 0.0000 0.0000
Total 1.8254 2.9940

where var(s) =3.0508.





56






57
1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2
0
1
2
3
Figure 4.5(3): Adaptive Fourier line spectrum
1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3
0
0.5
1
1.5
2
Figure 4.5(4): Fourier line spectrum







58
5. Conclusion and Future Research



5.1 Summary and Conclusion


The goal of this dissertation is to develop theory and methods that can be applied to
equally and unequally-spaced time series in which the frequency components of time
series may take general periodic shapes that include sinusoids as special cases. The re-
sults of the research are summarized as follows:

(1) Theorem 3.1 Any function s(t) in L
2
that satisfies s(t) =-s(t-
k

) is the limit of
step functions s
n
(t) in B
n k,
as n , in the L
2
sense.

(2) Corollary 3.1. L
2
has another decomposition as
L
2
= +B
1
+B
0
+B
1
+
where B
k
is the subspace of L
2
that contains all periodic functions that have
the form s
] [k
(t), in which s
] [k
(t) = - s
] [k
( t-
k

), for k 0; and B
0
={c, c is
complex-valued scale}.

(3) Through a weight inner product, <y, z >
m , 3
, a new method of projection has
been developed, which can be applied to project x
e
(t) onto L
n K B , ,
as s
m n,
(t) =
P
m , 3
( x
e
(t) | L
n K B , ,
), where x
e
(t) is the continuous- time extension of equally
or unequally-spaced time series. The adaptive Fourier analysis of time series
x proceeds by investigating the properties of the projection, s
m n,
(t).



59

(4) By theorem 3.1 an important property of s
m n,
(t) can be proved as
x(t) =
n
lim
m
lim s
m n,
(t),
at each t =t
1
, , t
N
.

(5) A multiresolution analysis (MRA) of L
2
has been presented, which showed
that the step functions used in this research can generate a MRA of L
2
.

(6) Through a nonstandard inner product <y, z >
4
, adaptive Fourier ANOVA has
been developed, which can be applied to equally and unequally-spaced time
series.

(7) Examples: Example 4.1 gives an application of adaptive Fourier analysis to a
real data set. Example 4.2 illustrates that for some signals with jumps, adap-
tive Fourier analysis outperforms Fourier analysis and is different from wave-
let analysis. Example 4.3 illustrates that adaptive Fourier analysis can be
applied to the time series with general patterns that include sinusoids as
special cases. Example 4.4 illustrates that adaptive Fourier analysis can direct-
ly deal with unequally-spaced time series with sinusoidal frequency compo-
nents. Example 4.5 illustrates that adaptive Fourier analysis outperforms
Fourier analysis for a non-sinusoidal signal.

5.2 Proposed Future Research


(1) A better algorithm is needed for computing the adaptive Fourier trans-
form. Before a better algorithm is developed, use of adaptive Fourier



60
analysis in analyzing time series with large sample size would be time-
consuming.

(2) To do statistics tests for hidden periodic components, relevant theory
and methods for adaptive Fourier analysis are needed.

(3) The orthogonal series approach to nonparametric regression has be-
come popular lately. People can approximate a function by polynomi-
als, sinusoids, step functions, and wavelets and apply these approxima-
tions to relevant nonparametric regressions. Nonparametric regression
by the step functions used in this research seems promising and needs
to be developed. We expect that this kind of nonparametric regression
would be applied to unequally-spaced data observations and would
have fewer terms in the regression models for some cases, i.e. would
use fewer degrees of freedom.












61
Bibliography



Anderson, J .A. (1969), Real Analysis. Gordon and Breach Science Publishers, New
York.

Bruce, Andrew and Gao, Hong-Ye (1996), Applied Wavelet Analysis With S-Plus.
Springer-Verlag Inc, New York.

Barthes, D. (1995), Cleaning Power Spectra of Short, Unevenly Spaced Time-Series
Application to LPVS, Astronomy & Astrophysics Supplement Series. 111(2):
373-386 J un 1995.

Beauchamp, K.G. (1984), Applications of Walsh and Related Functions. London:
Academic Press.

Brockwell, P.J . and Davis, R.A. (1990), Time Series: Theory and Methods (Second
Edition). Springer-Verlag Inc, New York.

Bowen, Ray M. and Wang, C. C. (1976), Introduction to Vectors and Tensors Linear and
Multilinear Algebra. Plenum Press, New York.

Baisch, S. and Bokelman, GHR (1999), Spectral Analysis with Incomplete Time Series:
An Example from Seismology, Computers & Geosciences. 25(7): 739-750 Aug.
1999.

Carothers, N.L. (2000), Real Analysis, Cambridge University Press.

Dorny, C. Nelson (1980), A Vector Space Approach to Models and Optimization. Robert
E. Krieger Publishing Company.

Edelson, R.A. and Krolik, J .H. (1988), The Discrete Correlation Function: A New
Method For Analyzing Unevenly Sampled Variability Data, The Astrophysical
J., 333: 646-659.

Engle, RF and Russell J R (1998), Autoregressive Conditional Duration: A New Model
for Irregularly Spaced Transaction Data, Econometrica. 66(5): 1127-1162.

Foutz, R.V. and Lee, H. (2000), Adaptive Fourier Series and The Analysis Of
Periodicities in Time Series Data, J. Time Ser. Anal. Vol. 21, No. 6 p649-662.

Frazier, Michael W. (1999), An Introduction to Wavelets Through Linear Algebra,
Springer-Verlag Inc, New York.




62
Golubov, B., Efimov, A. and Skvorstsov, V. (1991), Walsh Series and Transforms:
Theory and Applications, Kluwer Academic Publishers.

Good, I. J . and Doog, K. C. (1958), A Paradox Concerning Rate of Information,
Information and Control, Volume 1, 113-126.

Halmos, P. R. (1957), Introduction to Hilbert Space and the Theory of Spectral
Multiplicity. New York: Chelsea.

Koopmans, L.H. (1974), The Spectral Analysis of Time Series. New York: Academic
Press.

Kuhn, J .R. (1982), Recovering Spectral Information From Unevenly Sampled Data:
Two Machine-efficient Solutions, The Astronomical J., Vol. 87, No. 1, pp 196-
202.

Lomb, N.R. (1976), Least-Squares Frequency Analysis of Unequally Spaced Data,
Astrophysics and Space Science Vol. 39, pp447-462.

Marvasti, Farokh (2001), Nonuniform Sampling Theory and Practice. Kluwer
Academic/Plenum Publishers.

Meisel, D.D. (1978), Fourier Transforms of Data Sampled at Unequal Observational
Intervals, The Astronomical J., Vol. 83, No. 5, P538-545.

Priestley, M.B. (1996), Wavelets and Time-dependent Spectral Analysis, J. Time Ser.
Anal. 17, 85-103.

Rao, T. Subba, Priestley, M.B. and Lessi, O. (1997), Applications of Time Series
Analysis in Astronomy and Meteorology. Chapman & Hall.

Ruf T. (1999), The Lomb-Scargle Periodogram in Biological Rhythm Research:
Analysis of Incomplete and Unequally Spaced Time-Series, Biological Rhythm
Research 30(2): 178-201 APR 1999.

Scargle, J .D. (1982), Studies in Astonomical Time Series Analysis. II. Statistical
Aspects of Spectral Analysis of Unevenly Spaced Data, The Astronomical J.,
263: 835-853.

Scargle, J .D. (1989), Studies in Astronomical Time Series Analysis. III. Fourier
Transforms, Autocorrelation Functions, and Cross-Correlation Functions of
Unevenly Spaced Data, The Astronomical J., Vol. 343, pp874-877.

Schimmel, M. (2001), Emphasizing Difficulties in The Detection of Rhythms with
Lomb-Scargle Periodograms, Biological Rhythm Research. 32(3): 341-345 J ul.
2001.



63

Schulz, M. and Stattegger, K. (1997), Spectrum: Spectral Analysis of Unevenly Spaced
Paleoclimatic Time Series, Computers & Geosciences 23(9): 929-945. 11/1997.


Stoffer, D.S. (1991), Walsh-Fourier Analysis and Its Statistical Applications, J. Am.
Stat. Assoc. 86, 461-85.

Swan, P.R. (1982), Discrete Fourier Transforms of Nonuniformly Spaced Data, The
Astronomical J., Vol.87, No.11 p1608-1615.

Van, Dugen HPA, Olofsen, E. VanHartevelt, J H, Kruyt, EW (1999), A Procedure of
Multiple Period Searching in Unequally Spaced Time-Series with The Lomb-
Scargel Method, Biological Rhythm Research, 30(2): 149-177 APR 1999.




















64
Appendix 1


Theorem 3.1. Any function s(t) in L
2
that satisfies s(t) =- s(t -
k

) for all t R , is the

limit of step functions s
n
(t) in B
n k,
as n , in the L
2
sense.

Proof: Let k be a positive integer, and n =2
m
for some positive integer m, G
j
and H
j
are
any real numbers, for j =1, 2, , n. The function

x
n
(t) = ) (
1

=
+
n
j
j j
H i G f
j n k , ,
(t) is in B
n k,
,

since each f
j n k , ,
(t) is in B
n k,
. We pick the constants G
j
and H
j
symmetrically such that

G
1
=- H
1
,
(G
2
, H
2
) =(G
n
, - H
n
),
(G
3
, H
3
) =(G
1 n
, - H
1 n
),

(G
2
n
, H
2
n
) =(G
2
2
+
n
, - H
2
2
+
n
),
H
1
2
+
n
=0.

We denote the imaginary part of a function x(t) as I
m
{x(t)} and divide the interval -
k
1

t <-
k
1
+
k 2

into n equally spaced small intervals as I

1
=[-
k
1
, -
k
1
+
kn 2

), I
2
=[-
k
1
+
kn 2

, -
k
1
+
kn 2
2
), , I
l
=[-
k
1
+
kn
l
2
) 1 (
,-
k
1
+
kn
l
2

), , and I
n
=[-
k
1
+
kn
n
2
) 1 (
, -
k
1
+
kn
n
2

). Now note that

65

f
j n k , ,
(t) =exp(i

+ 1
1 2 2
2 n
j kt

);
f
j n k , ,
(t)=exp(i

n
j kt 1 2 2
2

);
f
j n, , 0
(t) =1.

Then,
1
0
. When t I
1
=[-
k
1
, -
k
1
+
kn 2

),
I
m
{x
n
(t)} =I
m
{( G
1
+iH
1
)f
1 , ,n k
(t)} +I
m
{ ) (
2

=
+
n
j
j j
H i G f
j n k , ,
(t)}
=G
1
+(H
2
+H
3
+ +H
n
)
=G
1
.
Let s
1
=G
1
, then
I
m
{x
n
(t)} =s
1
=G
1
, for t I
1
=[-
k
1
, -
k
1
+
kn 2

);
2
0
. When t I
2
=[-
k
1
+
kn 2

, -
k
1
+
kn 2
2
),
I
m
{x
n
(t)} =I
m
{( G
1
+iH
1
)f
1 , ,n k
(t)+(G
2
+iH
2
)f
2 , ,n k
} +I
m
{ ) (
3

=
+
n
j
j j
H i G f
j n k , ,
(t)}
=( G
1
+G
2
)+(H
3
+ +H
n
)
=G
1
+G
2
- H
2
,
Let s
2
=G
1
+G
2
- H
2
, then
I
m
{x
n
(t)} =s
2
=G
1
+G
2
- H
2

=s
1
+G
2
- H
2
, for t I
2
=[-
k
1
+
kn 2

, -
k
1
+
kn 2
2
);



66
3
0
. When t I
3
=[ -
k
1
+
kn 2
2
, -
k
1
+
kn 2
3
),
I
m
{x
n
(t)} =I
m
{ ) (
3
1

=
+
j
j j
H i G f
j n k , ,
(t)} +I
m
{ ) (
4

=
+
n
j
j j
H i G f
j n k , ,
(t)}
=(G
1
+G
2
+G
3
)+(H
4
+ +H
n
)
=G
1
+G
2
+G
3
- H
2
- H
3

=s
2
+G
3
- H
3
,
Let s
3
=s
2
+G
3
- H
3
, then
I
m
{x
n
(t)} =s
3
=s
2
+G
3
- H
3
, =G
1
, for t I
3
=[ -
k
1
+
kn 2
2
, -
k
1
+
kn 2
3
).
4
0
. Similarly, for l
2
n
, we can obtain
I
m
{x
n
(t)} =s
l

=s
1 l
+G
l
- H
l
for t I
l
.
and s
l
=(G
1
++G
l
) (H
2
++H
l
) for 2 l
2
n
.
5
0
. For l =
2
n
+1, and t I
1
2
+
n
,
I
m
{x
n
(t)} =I
m
{ ) (
1
1
2

+
=
+
n
j
j j
H i G f
j n k , ,
(t)} +I
m
{ ) (
2
2

+ =
+
n
j
j j
n
H i G f
j n k , ,
(t)}
=(G
1
++G
1
2
+
n
)+(H
2
2
+
n
++H
n
)
=s
2
n
+G
1
2
+
n
.
6
0
. Similarly, for l =
2
n
+2 and t I
2
2
+
n
, we have
I
m
{x
n
(t)} =s
2
2
+
n

=s
1
2
+
n
+G
2
n
+H
2
n
.



67
7
0
For
2
n
+2 l n and t I
l
, similarly we have
I
m
{x
n
(t)} = s
l

=s
1 l
+G
2 + l n
+H
2 + l n
.

Based on 1
0
through 7
0
above, we have
I
m
{x
n
(t)} =

+ +
+ +
. ,
,
,
,
,
,
2 2
1 1
3 3
2 2
1 1
2 2
2 2
n n
I t for s
I t for s
I t for s
I t for s
I t for s
I t for s
n n
n n

(A1)

where

+ + =
+ + = + =
+ =
+ =
+ =
=

+ + + +
+ +
+
.
2 2 1
1 2 1 2
1 1
3 3 2 3
2 2 1 2
1 1
2 2 2 2 2
2
2 2
2 2 2
H G s s
H G s H G s s
G s s
H G s s
H G s s
G s
n n
n n n n
n
n n
n n n

(A2)

The equation (A2) above can be represented by matrix form as




68
C
n n

+
2
2
2
1
2
1
n
n
H
H
G
G
G

1
2 3
1 2
1
n n
s s
s s
s s
s

, (A3)

where
C
n n
=

0 0 0 1 0 0 0 0 1 0
0 0 1 0 0 0 0 1 0 0
0 1 0 0 0 0 1 0 0 0
1 0 0 0 0 1 0 0 0 0
0 0 0 0 1 0 0 0 0 0
1 0 0 0 0 1 0 0 0 0
0 1 0 0 0 0 1 0 0 0
0 0 1 0 0 0 0 1 0 0
0 0 0 1 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 1












.

Over the next interval -
k
1
+
k 2

t <-
k
1
+
k

, I
m
{ x
n
(t) } has n equally spaced steps with
corresponding values s
1 + n
=s
n
, s
2 + n
=s
1 n
, ,
n
s
2
=s
1
; over the next intervals -
k
1
+
k

t <-
k
1
+
k 2
3
and -
k
1
+
k 2
3
t <-
k
1
+
k
2
, I
m
{x
n
(t)} =I
m
{x
n
( t -
k

)} and then I
m
{x
n
(t)}
repeats itself periodically over interval of length
k
2
.




69
Set s
n
=1 and s
j
=0 if 1 j < n, then the equation (A3 ) has an unique solution since
rank( C
n n
) =n. Corresponding to the solution, I
m
{ x
n
( t ) }, is denoted as I
m
x
n n,
( t ),
where


I
m
x
, n n
( t ) =

=
=

=
=

+
+ +
+
+ +

) ... ( , 0
, 1
, 1
) ... ( , 0
, 1
, 1
) ... ( , 0
4 2 3
1 3 1 3
3 3
1 3 2
1 1
1 1
n n
n n
n n
n n
n n
n n
n
I I t for
I t for s
I t for s
I I t for
I t for s
I t for s
I I t for
,


For each l , let the step function I
m
x
n l n ,
( t ) =I
m
x
n n,
( t +
kn
l
2

) be a location shifted
version of I
m
x
n n,
( t ), then we obtain 2n-1 step functions, I
m
x
n l n ,
( t ) in B
n k,
, for l =
0, 1, 2, , (n-1).

For any >0, there is a step function H(t) with || I
m
{s(t)}I
A
(t)-H(t)||
2
<
k 8

, where I
A
(t)
=1 if t A =[-
k
1
, -
k
1
+
k

) and I
A
(t) =0 if t A =[-
k
1
, -
k
1
+
k

), See Carothers (2000, p

350 ). If we define H(t) =- H( t-
k

) for t [-
k
1
+
k

, -
k
1
+
k
2
), and let H(t) repeat itself
periodically over interval of length
k
2
, and let A
j
=[ -
k
1
+
k
j ) 1 (
, -
k
1
+
k
j
), for j =1, 2, ,
, 2k. Then, over C[0, 2 ], we have



70

|| I
m
{s(t)}-H(t)||
2
2
=

=
k
j
2
1
|| I
j
A
(t) (I
m
{s(t)}-H(t)) ||
2
2

<2k(
k 8

)
2

=
4
2

, (A4)

where I
j
A
(t) =1 if t A
j
and I
j
A
(t) =0 if t A
j
.

Furthermore, since H( t ) is a step function, H( t ) must be constant on each of the open
intervals (t
j
, t
1 + j
), where -
k
1
=t
0
<t
1
< <t
L
=-
k
1
+
k
2
, and j =0, 1, 2, , L-1. Note
that step function is bounded, we can set M =
) , [
2 1 1
max
k k k
t

+
| H(t)|. Considering the subinter-
vals I
1
I
2
, I
3
I
4
, , I
1 4 n
I
n 4
, where I
1
I
2
I
3
I
4
I
1 4 n
I
n 4
=[-
k
1
,
-
k
1
+
k
2
), we define 2n constant values c
1
, c
2
, , c
n 2
by

c
i
=

=


L j j some for I I t and I I t if M
L j for I I t and I I t if t H
i i j i i
i i j i i
0 , , ,
1 ..., , 2 , 1 , 0 , ), (
2 1 2 2 1 2
2 1 2 2 1 2
(A5)

for i =1, 2, , 2n.

Set h(t) =

n
l
n l m l
t x I c
1
), 1 2 (
) ( , (A6)



71
then, h(t) is a step function in B
n k,
, and satisfies that h(t)=-h( t-
k

) and |h(t)-H(t)|2M.
Thus, over C[
k
1
, -
k
1
+
k
2
), we have

|| h(t)-H(t)||
2
2
L(2M)
2
*
nk

=
nk
LM
2
4
. (A7)

When n >
2
2
16

LM
, based on (A7), over C[0, 2 ], we obtain

|| h(t)-H(t)||
2
2

4
2

. (A8)

(A4) and (A8) together show

|| I
m
{s(t)}-h(t)||
2
|| I
m
{s(t)}-H(t)||
2
+|| H(t)-h(t)||
2


2

+
2

=

Similarly, the real part of s(t) has the same conclusion , therefore conclusions together
prove the Theorem 3.1.


Corollary 3.1 L
2
has another decomposition as below
L
2
= +S
1
+S
0
+S
1
+



72
where S
k
is the subspace of L
2
that contains all periodic functions that have the s
] [k
(t),
in which s
] [k
(t) satisfies s
] [k
(t) =- s
] [k
(t-
k

), for k 0, and S
0
={ c, c is complex-valued
scale }.

Proof : Since any function s(t) in S
k
satisfies that s(t) =-s(t-
k

), it follows from Theorem

3.1 that S
k
is a subspace of B
k
, where B
k
=
n k
n
B
,
lim

in the sense that B
k
is the smallest
subspace of L
2
containing B
n k,
for each n, and therefore +S
1
+S
0
+S
1
+ is a
subspace of +B
1
+B
0
+B
1
+ . Note that L
2
= +B
1
+B
0
+B
1
+ , see Foutz
and Lee (2000). On the other hand, since the function s(t) = exp(ikt) spans A
k
and
satisfies s(t) =-s(t-
k

), A
k
is a subspace of S
k
, and thus L
2
= +A
1
+A
0
+A
1
+
is a subspace of +S
1
+S
0
+S
1
+ . The conclusions together prove Corollary 3.1.














73

Appendix 2


Construct x
e
(t) as follows.
Case 1. Assume there are not t
i
=t
j

2
N
for any i , j {1, 2, , N} where t
N
=N. Set
t =
N j j ' , 1
min {|t
j
-(t
' j

2
N
)|. Define x
e
(t) as below
.

x
e
(t) =

+ <
+ <


otherwise
t t t for t x
t t t for t x
t N
j
t N
j j
t
j
t
j j
, 0
), (
), (
2 2 2 2
2 2
,

where j =1, 2, , N.
Thus, x
e
(t) =- x
e
(t -
2
N
) for 0 t N, and x
e
(t) L
1 , B
.

Case 2. If there are t
i
=t
j

2
N
for some i and j, then we can pick a small constant >0,
such that there are not t
i
=t
j

2
+ N
for any i and j, and set
t =
N j j ' , 1
min {|t
j
-(t
' j

2
+ N
)|.
Thus, we can define x
e
(t) as below.




74
x
e
(t) =

+ <
+ <


otherwise
t t t for t x
t t t for t x
t N
j
t N
j j
t
j
t
j j
, 0
), (
), (
2 2 2 2
2 2


where j =1, 2, , N.
Clearly, x
e
(t) =- x
e
(t -
2
+ N
) for 0 t N, and x
e
(t) L
1 , B
.





















75
Appendix 3

We need to prove
x(t) =
n
lim
m
lims
m n,
(t) , for t =t
1
, , t
N
. (3.9)
Note the construction of x
e
(t) in Appendix 2, we know that x
e
(t) is continuous at all t =
t
j
, j =1, 2, , N. By Theorem 3.1, x
e
(t) is a limit of step functions e
n , 1
(t) in L
n B , 1 ,
for
increasing n. In addition, x
e
(t) is continuous at t =t
1
, , t
N
, based on its construction in
Appendix 2. Thus, given any >0, when n is large enough, we have
| x
e
(t) - e
n , 1
(t) | <
2
1
, for t =t
1
, , t
N
. (A3.1)
which implies that

n
lim
N
t t t
1
sup | x
e
(t) - e
n , 1
(t) | =0. (A3.2)
Now note x(t) =x
e
(t) , for t =t
1
, , t
N
. It follows from (A3.1) or (A3.2) that

n
lim| x(t) - e
n , 1
(t) | =0 , for t =t
1
, , t
N
. (A3.3)
The function e
n , 1
(t) is in L
n K B , ,
when K 1, and the projection s
m n,
(t) =P
m , 3
( x
e
|
L
n K B , ,
) minimizes | x
e
(t)- s
m n,
(t) |
m , 3
over functions in L
n K B , ,
. Thus,
| x
e
- s
m n,
|
m , 3
| x
e
(t) - e
n , 1
(t) |
m , 3
. (A3.4)
It follows from (A3.4) that

N
1
dt t s t x t w
m n e
N
m
2
,
0
| ) ( ) ( | ) (

N
1
dt t e t x t w
n e
N
m
2
, 1
0
| ) ( ) ( | ) (

. (A3.5)
Taking limits on both sides of (A3.5) for increasing m, we have



76

m
lim
N
1
dt t s t x t w
m n e
N
m
2
,
0
| ) ( ) ( | ) (

m
lim
N
1
dt t e t x t w
n e
N
m
2
, 1
0
| ) ( ) ( | ) (

. (A3.6)
By (3.7) in section 3.5, (A3.6) becomes

N
1

=
N
j 1
| x
e
(t
j
)- s
m n,
(t
j
)|
2

N
1

=
N
j 1
| x
e
(t
j
)- e
n , 1
(t
j
)|
2
. (A3.7)
Because x(t) =x
e
(t) at t =t
1
, , t
N
, and from (A3.7), we obtain (A3.8) as below,

m
lim
N
1

=
N
j 1
| x (t
j
)- s
m n,
(t
j
)|
2
=
m
lim

=
N
j 1
| x
e
(t
j
)- s
m n,
(t
j
)|
2


N
1

=
N
j 1
| x
e
(t
j
)- e
n , 1
(t
j
)|
2

=
N
1

=
N
j 1
| x (t
j
)- e
n , 1
(t
j
)|
2
. (A3.8)
It follows from (A3.8) that

m
lim

=
N
j 1
|x (t
j
)- s
m n,
(t
j
)|
2

=
N
j 1
| x (t
j
)- e
n , 1
(t
j
)|
2
. (A3.9)
By taking limits on both sides of (A3.9) for increasing n, and by (A3.3), we have

n
lim
m
lim

=
N
j 1
| x (t
j
)- s
m n,
(t
j
)|
2

n
lim

=
N
j 1
| x (t
j
)- e
n , 1
(t
j
)|
2

=0. (A3.10)
Conclusion (3.9) follows from (A3.10).








77

Appendix 4

A nonstandard inner product <y, z >
4
may be defined on L
n K B , ,
=B
n K,
+ + B
n , 1
+
+ B
n , 1
+ + B
n K,
as follows: Theorem 1 of Foutz and Lee (2000) shows that each
y in L
n K B , ,
has an unique representation,
y(t) =

=
K
K k
k
y (t) , (A4.1)
where y
k
(t) is a unique function in B
n k,
for each k. Shift each frequency component in
y
k
(t) in (A4.1) by the fraction u of its period to obtain
y
u
(t) =

=
K
K k
k
y (t +
k
uN
). (A4.2)
Define the inner product between y and z in L
n K B , ,
to be
<y, z >
4
=

<
1
0
,
u u
z y >
1
du . (A4.3)
In Foutz and Lee (2000) it is proven that the spaces B
n k,
for k =-K, , K are orthogonal
with respect to <y, z >
4
.


Appendix 5.

It remains to verify (3.12). The left side of (3.12) is



78
|

=
K
K k
m n k
t s ) (
, ,
|
2
4
,
This equals

=
K
K k
m n k
t s
2
4 , ,
| ) ( | ,
because the frequency component s
m n k , ,
( t ) is in B
n k,
and because the spaces B
n k,
are
orthogonal with respect to the inner product <y, z >
4
. This sum in turn equals the right
side of (3.12), because s
m n k , ,
(t) is periodic with period
k
N
and therefore
| s
m n k , ,
(t) |
2
4
=

1
0
1
N
2
0
, ,
| ) ( |

+
N
m n k
k
uN
t s dtdu
=

1
0
1
N
2
0
, ,
| ) ( |

N
m n k
t s dtdu
=
N
1
2
0
, ,
| ) ( |

N
m n k
t s dt.














79
Appendix 6. Background


The concepts and results listed in this Appendix will provide an adequate background for
theoretical results and their proofs in Chapter 3 and the Appendix1-5. All conceptions and
their properties used in Chapter3 and Appendix1-5, such as vector space, direct sum, inner
product, Hilbert space, weight inner product, orthogonal projection, and Gram-Schmidt
procedure, are presented in this Appendix.


A6.1 Vector Spaces


Definition 6.1. (Vector space) Let F be a field, and V a set with operations addition and
scale multiplication. V is a vector space if it satisfies the following properties:
1) For operation addition +, V has properties (1) through (5) as below.
for any x, y V, x +y V (closure for addition).
(1) (x + y) +z =x + (y + z) (Associativity for addition).
(2) x + y = y + x (commutativity for addition).
(3) There is an element 0 V such that x + 0 = x for any x V (Existence of
additive identity).
(4) For any x V, there is an element x such that x + (-x) =0.
2) For operation scale multiplication, V has properties 6 through 10 as below for any ,
F and any x, y V.
(6) x V (Closure for scale multiplication).
(7) ( x) =( )x (Associativity for scale multiplication).
(8) ( + )x = x + x (First distributive).



80
(9) (x + y) = x + y (Second distributive).
(10) 1x = x.

Remark. If the field F is real numbers R, the space V is called a real vector space;
similarly if the field F is complex numbers C, the space V is called a complex vector
space.

Definition 6.2. ( Linear independent ) A set of vectors { x
1
, x
2
, , x
n
} in a vector
space V is said to be linearly dependent if there is a set of scales {
1
,
2
, ,
n
} not all
zero, such that


1
x
1
+
2
x
2
+ +
n
x
n
=0. (6.1)

If (6.1) holds only when
i
=0 for any i =1, 2, , n, we say that { x
1
, x
2
, , x
n
} are
linearly independent.

Definition 6.3. (Basis and dimension) Let { x
1
, x
2
, , x
n
} be linearly independent set
in vector space V, if for any vector x V , { x
1
, x
2
, , x
n
, x } are linearly dependent,
we say that { x
1
, x
2
, , x
n
} is a basis of V and that V is finite dimensional and the
dimension of V is n, written dim V, namely n=dim V.





81
Definition 6.4. (Span) Let V be a vector space, and UV. The span of U is the set of all
linear combinations of elements of U.

Definition 6.5. ( Subspace ) A nonempty subset U of a vector space V over F is a sub-
space if:
x, y U x + y U, for any x, y U, and any , F.

Definition 6.6. (Sum of spaces) Let U and W be subspaces of a vector space V. The sum
of U and W, written U+W, is the set

U+W={ x + y | x U, y W}.

The intersection of U and W, written U W , is the set

U W ={ x | x U and x W}.

The union of U and W, written U W, is the set

U W ={ x| x U or x W }.

Some properties of these operations are stated in the following theorems. ( Theorem 6.1
through Theorem 6.4), see Bowen and Wang(1976, P53).

Theorem 6.1. If U and W are subspaces of V, then U+W is a subspace of V.



82

Theorem 6.2. If U and W are subspaces of V, then U and W are also subspaces of U+W.

Theorem 6.3. If U and W are subspaces of V, then the intersection U W is a subspace
of V.

Theorem 6.4. Let x be a vector in U +W, where U and W are subspaces of V. The
decomposition of x U+W into the form x = y + z, where y U and z W, is unique if
and only if U W ={ 0 }.

Definition 6.7. (Direct sum) The sum of two subspaces U and W in V is called the direct
sum of U and W and written by

UW, if U W ={ 0 }.


A6.2 Inner product spaces , Hilbert spaces and their properties



Definition 6.8. (Inner product space) A complex vector space V is said to be an inner
product space if there exists an operation called inner product by which any ordered pair
of vectors y and z in V determines an element of C, written by <y, z>such that:
(a) <y, z >=
_______
, > < y z , where
_______
, > < y z is the complex conjugate of <z, y >;
(b) <y +x, z>=<y, z>+<x, z>;
(c) <y, z >=< y, z >;



83
(d) <y, y > 0 and <y, y >=0 if and only if y =0;
for all x, y, z V and C.

Definition 6.9. (Norm). The norm of an element x of an inner product space is an opera-
tion, written by || ||, defined by

|| x || = > < x x, .

Theorem 6.5(The Schwarz inequality)

| <y, z >| || y || || z ||


is valid for any two vectors y, z in an inner product space.


For Theorem 6.5 and its proof, see Bowen and Wang (1976, p60).


Theorem 6.6( The triangle inequality)

|| y + z || || y || +|| z ||

is valid for any vectors y, z in an inner product space.


For Theorem 6.6 and its proof, see Bowen and Wang (1976, p60).


Proposition 6.1. (Properties of the norm). A norm on an inner product space V over C
satisfies the following properties:
(a) || y || =| | || y ||;



84
(b) || y || 0 and || y || =0 if and only if y =0;
for all y, z V and all C.

For Proposition 6.1 and its proof, see Bowen and Wang (1976, p61).

Definition 6.10. (Convergence in norm) A sequence { x
n
, n =1, 2, } of vectors of an
inner product space V is said to converge in norm to x V if

|| x
n
- x || 0 as n .

For Definition 6.10, see Brockwell and Davis (1994, p. 45).


Proposition 6.2. (Continuity of the inner product) If { x
n
, n =1, 2, } and { y
n
, n =
1, 2, } are sequences of vectors of an inner product space V such that || x
n
- x || 0 as
n and || y
n
- y || 0 as n , then

(a) || x
n
|| || x || and

(b) <x
n
, y
n
> <x, y >.

For Proposition 6.2 and its proof, see Brockwell and Davis (1994, p. 45).




85
Definition 6.11. (Cauchy sequence) A sequence { x
n
, n =1, 2, } of elements of an
inner product space is said to be a Cauchy sequence if

|| x
n
- x
m
|| 0 as n, m .
For Definition 6.11, see Brockwell and Davis (1994, p. 45).


Definition 6.12. (Hilbert space) A Hilbert space H is an inner product space which is
complete, i.e. an inner product space in which every Cauchy sequence { x
n
}converges in
norm to some element x H.
For Definitions 6.12 , see Brockwell and Davis (1994, p. 45).

Example 6.1. ( The standard inner product for C
n
). The standard inner product for
C
n
is defined by

<x, y >=

=
n
i
i i
y x
1
_
,
where x =( x
1
, x
2
, , x
n
)and y =( y
1
, y
2
, , y
n
).
Remark. C
n
is defined as the set of all n-tuples of complex numbers, that is
z =

n
z
z
z

2
1
,
where z
1
, z
2
, , z
n
C.



86

For Examples 6.1, see Dorny (1980, p240).


Example 6.2. (The standard inner product for function spaces). The standard inner
product for a function space C(a, b) is defined by

<f , g>=

b
a
t g t f ) ( ) ( dt ,

where f and g C(a, b). Where C(a, b) is the set of continuous, complex-valued func-
tions on [a, b].

For Examples 6.2 , see Dorny (1980, p240-241).

Example 6.3 (A weighted Inner product for function spaces). An inner product of the
following form is often appropriate for C(a, b):

<f , g>=

b
a
t g t f t w ) ( ) ( ) ( dt , 0 <w(t) <.

For Examples 6.3, see Dorny (1980, p242).







87
A6.3 Orthogonal Projection and Orthonormal Bases


Definition 6.13. ( Orthogonal ) Let V be a complex inner product space. For x, y V,
we say x and y are orthogonal, written x y , if <x, y>=0.

For Definition 6.13, see Frazier (1999, p84).

Definition 6.14. (Orthogonal and orthonormal set) Let V be a complex inner product
space and X be a set in V. X is called an orthogonal set if any two different vectors of X
are orthogonal; X is called an orthonormal set if X is an orthogonal set and || x || =1 for
all x X.

For Definition 6.14, see Frazier (1999, p84).


Theorem 6.7. Let V be a complex inner product space, and let X ={ x
1
, x
2
, , x
n
} be
an orthogonal set of vectors in V and 0 X. Then X is a linearly independent set.

For Theorem 6.7 and its proof, see Frazier (1999, p84).

Theorem 6.8. Let V be a complex inner product space, and let X ={ x
1
, x
2
, , x
n
} be
an orthogonal set in V with x
j
0 for all j. If x span X, then

x =
j
n
j
j
j
x
x
x x

=
> <
1
2
|| ||
,
.

For Theorem 6.8 and its proof, see Frazier (1999, p84-85).




88

Definition 6.15 ( Orthogonal projection ) Let V be a complex inner product space, and
X ={ x
1
, x
2
, , x
n
} be an orthogonal set in V with x
j
0 for all j. Let S=span X. For x
V, define the orthogonal projection P( x | S ) of x onto S by
P( x | S ) =
j
n
j
j
j
x
x
x x

=
> <
1
2
|| ||
,
.

For Definition 6.15, see Frazier (1999, p85).


Theorem 6.9. Let V, X, S and P( x | S) be as in Definition 6.15. Then
(a) P( x | S ) is a linear transformation.
(b) For every x V, P( x | S ) S.
(c) If y S, then P( y | S ) =y.
(d) For any x V and y S,

( x - P( x | S )) y.

(e) For any x V and y S,

|| x - P( x | S ) || || x y ||,

with equality if and only if y =P( x | S ).

For Theorem 6.9 and its proof, see Frazier (1999, p85-86).




89




Theorem 6.10. ( Gram-Schmidt procedure). Let V be a complex inner product space,
and { x
1
, x
2
, , x
n
} be a linearly independent set in V. Then there is an orthonormal
set { x
1
, x
2
, , x
n
} with the same span .

For Theorem 6.10 and its proof, see Frazier (1999, p87).

Definition 6.16. ( Orthonormal basis ). Let V be a complex inner product space. An
orthonormal basis for V is an orthonormal set in V that is also a basis.

For Definition 6.16, see Frazier (1999, p88).

Theorem 6.11. Let V be a complex inner product space with orthonormal basis X ={ x
1
,
x
2
, , x
n
}.
(a) For any x V,

x =

=
> <
n
j
j j
x x x
1
, .

(b) (Parsevals relation) For any x, y V,

<x, y >=

=
> < > <
n
j
j j
x y x x
1
, , .

(c) (Placherels formula) For any x V,

|| x ||
2
=

=
> <
n
j
j
x x
1
2
| , | .


For Theorem 6.11 and its proof, see Frazier (1999, p88).



90
Vita


Hong Liang earned a B.S. degree in Mathematics in 1982 and a M.Ed. in Edu-
cation Research in 1987 from South China Normal University. From 1982-1984, he was
on the faculty in Zhanjiang Agricultural College, now named Zhanjiang Ocean Univer-
sity, Zhanjiang, P.R. China. he taught Calculus and Biostatistics there. From 1987-1994,
he was on the faculty in South China Normal University and taught Educational Statistics
and Measurement, Design and Analysis of Experiments, Calculus, and did researches.

He entered Virginia Tech in 1994, began at his studies in Education, Research &
Evaluation (EDRE), working as a graduate teaching assistant and completing all require-
ments for the Ph.D. degree except the dissertation by 1997. He earned a M.S. degree in
statistics in 1997, a M.S. degree in Electrical Engineering in 1999, and a Ph.D. degree in
Statistics in 2002 from Virginia Tech. From 1999-2002, he worked as a graduate teaching
assistant at the Statistical Consulting Center at Virginia Tech.