Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
(1963) Probability Density Functionals and Reproducing Kernel Hilbert Spaces (Parzen)
Caricato da
Georg Ohm0 valutazioniIl 0% ha trovato utile questo documento (0 voti)
168 visualizzazioni15 pagineRelaciones entre funciones de densidad de probabilidad y los RKHS, aplicado a teoría estadística de comunicaciones. Este planteamiento hecho por Parzen en el 63, es hoy usado en muchas aplicaciones de las matemáticas, en particular la llamada estimación de densidad de funciones núcleo (kernel density estimation), mendiante estimación de la ventana de Parzen (Prazen window estimation).
Titolo originale
(1963) Probability Density Functionals and Reproducing Kernel Hilbert Spaces [Parzen]
Copyright
© © All Rights Reserved
Formati disponibili
PDF o leggi online da Scribd
Condividi questo documento
Condividi o incorpora il documento
Hai trovato utile questo documento?
Questo contenuto è inappropriato?
Segnala questo documentoRelaciones entre funciones de densidad de probabilidad y los RKHS, aplicado a teoría estadística de comunicaciones. Este planteamiento hecho por Parzen en el 63, es hoy usado en muchas aplicaciones de las matemáticas, en particular la llamada estimación de densidad de funciones núcleo (kernel density estimation), mendiante estimación de la ventana de Parzen (Prazen window estimation).
Copyright:
© All Rights Reserved
Formati disponibili
Scarica in formato PDF o leggi online su Scribd
0 valutazioniIl 0% ha trovato utile questo documento (0 voti)
168 visualizzazioni15 pagine(1963) Probability Density Functionals and Reproducing Kernel Hilbert Spaces (Parzen)
Caricato da
Georg OhmRelaciones entre funciones de densidad de probabilidad y los RKHS, aplicado a teoría estadística de comunicaciones. Este planteamiento hecho por Parzen en el 63, es hoy usado en muchas aplicaciones de las matemáticas, en particular la llamada estimación de densidad de funciones núcleo (kernel density estimation), mendiante estimación de la ventana de Parzen (Prazen window estimation).
Copyright:
© All Rights Reserved
Formati disponibili
Scarica in formato PDF o leggi online su Scribd
Sei sulla pagina 1di 15
CHAPTER 11
Probability Density Functionals and
Reproducing Kernel Hilbert Spaces*
Emanuel Parzen, Stanford University
ABSTRACT
‘The extraction, detection, and prediction of signals in the presence of noise
are among the central problems of statistical communication theory. Over
the past few years I have sought to develop an approach to those problems that
would simultaneously apply to stationary or nonstationary, discrete parameter
or continuous parameter, and univariate or multivariate time series and would
distinguish between their statistical and analytical aspects. In particular,
they would clarify the role played by various widely employed analytical
techniques (sach as the Wiener-Hopf equation and eigenfunction expansions).
In the development of this approach, two basic concepts are used: the notion
of the probability density functional ol a time series and the notion of a repro-
ducing kernel Hilbert space. The purpose of this chapter is to sketch the
relation between these concepts.
1. THE PROBABILITY DENSITY FUNCTIONAL
OF A NORMAL TIME SERIES
Let [S(), ¢ € T] and [N(@®,t & 7] be time series, called, respectively, the
signal process and the noise process. Let @ be the space of all real-valued
functions on 7. Let Py and Psy be probability measures defined on the
measurable subsets B of 2 by
Py{B] = prob ((V(), #E 1] € BY a)
Ps,n{B] = prob {[S@ + NO, t€ 7) EB). @)
We are trying to determine, if it exists, a function p on 0-with the property that
PssxlB) = f,pdPx. ®
= Prepared under contaact Nonr 3440(00) for tho Office of Naval Research, Reproduc-
tion in whole or in part is permitted for any purpose of the United States Government.
Reprinted by permission from Time Series Analysis, Murray Rosenblatt, ed., John Wiley &
Sons, Ine., New Yorks, 1963, pp, 155-169
ATT 478
156 STRUCTURAL PROBLEMS
‘The function p may be called the probability density functional of Psy with
respect to Py in order to emphasize that its argument is © function [X(,
£ET]. It is also denoted plX(®), t € 7] and called the probability density
functional of the signal-plus-noise process
AO =SH+NO, LET, a
with respect to the noise process (N(#), ¢ T]. ‘The function pis often written
symbolically as a derivative,
aP say
““aPy ®
and called the Radon-Nikodym derivative of Ps, with respect to Py [sce
Halmos (1950), p. 329].
A necessary and sufficient condition that the probability density (5) exist is
that Poy be absolutely continuous with respect to Py in the sense that, for
every measurable subset A of 2,
p
Py[A] = 0 implies Ps, x[A] = 0. (6)
In order that Ps, be not absolutely continuous with respect to Py, itis neces
sary and sufficient that there exist a set A such that
Py[A] = 0 and Psyx[A]} > 0. (7)
‘The probability measures Py and Ps,.v are said to be orthogonal if there exists
aset A such that
PrlAj = Oand Pszxl4] = 1. @)
We can regard (8) as an extreme case of being not absolutely continuous.
The notion of orthogonality derives its importance from detection theory
(the theory of testing hypotheses). The simple hypotheses
Ho: XQ) = NO)
Hi XQ) = SO) + NG)
are said to be perfectly detectable if there exists a set A such that
Px(A] = prob {[X(0, ¢€ T] € AlHo} = 0
Psywl4] = prob [[X(0,1€ 7} © AlHi} = 1.
Clearly, the hypotheses Ho and H; ere perfectly detectable if and only if Py
and Pg,y are orthogonal.
Given the probability measures Py and Ps,.w, the following questions arise:
1. Determine whether Py and Psyn are orthogonal.
2. Determine whether Ps; is absolutely continuous with respect to Py.
3. Determine the Radon-Nikodym derivative (5) if it exists.
(9)
‘To answer these questions, the natural way to proceed is to approximate the 479
PROBABILITY FUNCTIONALS AND HILBERT SPACES 187
infinite dimensional case by finite dimensional cases. For any finite subset,
Pea(h, +++, to? (10)
let Py,p and Ps4y,1 denote the probability distributions of [X(), ¢ € 2]
under Py and Ps,y, respectively. Assume that Pgyy,r is absolutely con-
tinuous with respect to Py, with Radon-Nikodym derivative denoted
dP san 7
= sen, 1
Pe = ap
‘The divergence between Psy. and Py on the basis of having observed (X(0),
i€ T’) is defined by
Jr = Esix(log pr) — Ex (log pr). (12)
Using the theory of martinggles, it may be shown that
OS Se S Son ff CTY. (13)
Consequently, the limit
Jr= im. Jr (14)
exists and is finite or infinite. Further, it may be shown [see Hajek (1958)}
that (a) if Jr < , then Ps,y is absolutely continuous with respect to Py and
Psy
= "apy 7 dimers (15)
@) if Jy = @, and both the time series [V(), t E T]and [SQ + N@,t€ 7)
are normal, then Ps, and Py are orthogonal.
‘We next apply these criteria under the following assuinptions.
The noise process [N()), t 7] is a normal process with zero means and
covariance kernel
P
KG, 0) = FN(s) NOL, (18)
which is positive definite in the sonse that for every finite subset 7” = {41,
+++ ,t,} of T the covariance matrix
Kh) ++ Kh, tr)
Kr = (Kt, (l= . . an
Ket Ayo Ket bn)
is nonsingular, with inverse matrix denoted
Ket = [KM 8)
(It should be noted that the assumption of positive definiteness is made only
for mathematical convenience in the present exposition; it can be omitted.) 480
158 STRUCTURAL PROBLEMS
In regard to the signal process, two cases are of most interest:
1, Sure signalease. [S(), t G Tlisa nonrandom function.
2. Stochastie signol case. [S(@), t € 1 is normal time series, independent
of the noise process, with zero means and positive definite covariance kernel
RG, ) = EIS) SO]. as)
‘To employ the criterion (15), we first need to compute the divergence
Jr», defined by (12). In this section we consider the sure signal case; the
stochastic signal case is considered in Section 3,
In the sure signal case
log pr: = (X, 8). — HS, 8) x2 20)
where we define for any functions f and g on 7
Goer = FY MK, 0 0. et)
er
Consequently
Jr = Beyal(X, S)x.r] — Ew(X, Sr] = (8,8) (22)
and
Jr < if and only if lim (8, S)x,27 <9. 23)
In words, in the sure signal case, Ps,.y is absolutely continous with respect to
Py if and only if (S, S)x,r- approaches a limit as 7” tends to 7. Fortunately
it is possible to characterize those functions S(-) that have this property.
To do so, we introduce the notion of a reproducing kernel Hilbert space,
2, REPRODUCING KERNEL HILBERT SPACES
Let K(s, t) be the covariance kernel of time series [X(, 6G 7}. For each
tin 7, let K(-, 2) be the function on T whose value at sin Tis equal to K(s, é).
It may be shown [sce Aronszajn (1950)] that there exists a unique Hilbert
space, denoted H(K; 7), with the following properties:
1. The members of H(K; 7) are real-valued functions on T [if K(s, #) wore
complex-valued, they would be complex-valued functions).
2. For every tin 7
KG,) € H(K; 1). @
3, For every tin T and f in H(K; 7)
$0 = GF KG Or ay
where the inner product between two funetions f and g in H(K; 7) is written
ax.
Frample 1. Suppose T = (1,2, . . . , n) for some positive integer m and
that the covariance kernel K is givon by a symmetric positive definite matrix
[Kj] with inverse [K]. ‘The corresponding reproducing kernel space H(K; f) 481
PROBABILITY FUNCTIONALS AND HILBERT SPACES 159
consists of all n-dimensional vectors f = (fy * * « ya) with inner product
Gere = 5 1k en
To prove (24) we need only to verify that the reproducing property holds for
eens
Goer 3 ptm, = 3 1a69 =
‘The inner product may also be written as a ratio of determinants:
Kao Ke fl iy oo ry
E@xer= — (25)
To prove (25), we again need only to verify the reproducing property. When
the covariance matrix K is singular, we may define the corresponding reproduc-
ing kernel inner product in terms of the pseudo-inserse of the matrix K.
Example 2. Let T = [isa < ¢ € blandlet [N(@, a < ¢ < b] be the Wiener
process; that is, it has independent increments and covariance function
K(s, ) = 9? min (s, 0) (26)
for some parameter c%, Consider the Hilbert spaces H(K; T) consisting of all
funetions fon a & t & b of the form
$0 =f) + [17 du en
for some square integrable measurable function f” on a < t < b [which can be
called the Lederivative of f], with inner product defined by
.
Gdee = 2 [Loam + ['reore eu} 8)
Tf we define
a1 if a