A likelihood approximation for locally stationary processes
by Rainer Dahlhaus
Universitat Heidelberg
Abstract
A new approximation to the Gaussian likelihood of a multivariate locally sta- tionary process is introduced. It is based on an approximation of the inverse of the covariance matrix of such processes. The new quasi-likelihood is a generalisation of the classical Whittle-likelihood for stationary processes. For parametric models asymptotic normality and eciency of the resulting estimator are proved. Since the likelihood has a special local structure it can be used for nonparametric inference as well. This is briey sketched for dierent estimates.
1 Introduction
Suppose we observe data X1::: XT from some nonstationary process and we want to t a parametric model to the data. An example is an autoregressive process with time varying coecients where we model the coecient functions by polynomials in time. If the process is Gaussian we can write down the exact likelihood function which, in the case of mean zero, takes the form
L
(e)T () := ;1
T Gaussian log likelihood
= 12 log(2)+ 1
2T logdet + 12TX0;1X (1.1) with X = (X1::: XT)0 (the assumption of a zero mean is given up later on).
However, for most of the time varying models the calculations needed for the min- imisation of this function are too time consuming. Suppose for example we want to t a time varying AR-model to the data where the coecient functions are polynomials in
0AMS 1991 subject classications. Primary 62M10 secondary 62F10.
0Key words and phrases. Locally stationary processes, Whittle-likelihood, local likelihood, preperi- odogram, generalized Toeplitz matrices.
1
time and where the AR-model order and the polynomial orders have to be determined by a model selection criterion. Such a model can be written in state space form with time varying system matrices (cf. Dahlhaus, 1996b) and - in principle - the minimisation could be done as in the stationary case by using the prediction error decomposition, the Kalman lter and a numerical optimisation routine (cf. Harvey, 1989, Section 3.4). However, we usually have a high dimensional parameter space, time dependent system matrices and a large number of models at hand, which make the calculations practically impossible.
To overcome these problems we suggest in this paper an approximation to the above likelihood which is a generalisation of Whittle's approximation in the stationary case (cf.
Whittle, 1953, 1954). In the stationary case is the Toeplitz matrix of the spectral density. Whittle had suggested to approximate ;1 by the Toeplitz matrix of the inverse of the spectral density leading with the Szego formula (cf. Grenander and Szego, 1958, Section 5.2) to the Whittle likelihood
L
(W)T () := 14 Z;
log42f() + IT() f()
d where
IT() = 12Tj
T
X
t=1Xtexp(;it)j2 is the periodogram.
In this paper we derive a similar approximation for processes that only show locally some kind of stationary behaviour. More precisely we consider locally stationary pro- cesses as dened in Dahlhaus (1996a,b 1997), i.e. processes with a time varying spectral representation as in (2.1) (the exact denition is given in Section 3). For an introduction to univariate locally stationary processes we refer to Dahlhaus (1996c).
In Section 2 we motivate the approximation and discuss its benets in a simplied setting (univariate processes, mean zero). In Section 3 we introduce multivariate locally stationary processes and the generalisation of the Whittle likelihood for such processes.
We then investigate the properties of the resulting parameter estimate.
2
Technically the approximation is based on a special generalisation of Toeplitz matrices (see (2.2)). The behaviour of norms and matrix products of such matrices is investigated in the appendix.
2 A motivation for the likelihood approximation
In this section we use a simplied setting to introduce and motivate the likelihood ap- proximation and to discuss its applications. Furthermore, we compare it to the Whittle approximation in the stationary case.
Suppose the observed process has a time varying spectral representation of the form XtT =Z
;exp(it)A
t T
d() (t = 1::: T) (2.1)
where() is a stochastic process on ;] with mean zero and orthonormal increments.
As e.g. in nonparametric regression the time parameter u = t=T in A is rescaled for a meaningful asymptotic theory (this is a special case of a locally stationary process as dened in Denition 3.1 below). We obtain for the variance covariance matrix
rs = cov(XrTXsT)
= Z
;expfi(r;s)gA
r T
A
s T
d:
In the stationary case where A(Tr) = A() does not depend on time this is equal to
Z
;expfi(r;s)gf()d
where f() = jA()j2 is the spectral density of the process. In the derivation of the Whittle approximation ;1 is approximated by the Toeplitz matrix
1 42
Z
;expfi(r;s)gf();1d
rs=1:::T :
In the nonstationary case we have forrs close to each other and for a function A which is smooth in time
rs
Z
;exp(i(r;s))f
r + s 2T
d 3
where f(u) :=jA(u)j2 is the time-varying spectral density of the process.
This suggests to use now
( 1 42
Z
;expfi(r;s)gf
r + s 2T
;1
d
)
rs=1:::T
as an approximation of ;1 in the nonstationary case. Since it leads to a slightly nicer criterion we use instead UT(412f;1) where
UT( ) =
Z
;expfi(r;s)g
1 T
r + s 2
d
rs=1:::T (2.2)
and x] denotes the smallest integer larger or equal to x. Note that UT(412f;1) is the classical Toeplitz/Whittle-approximation if f is constant over time (stationary case).
Using this approximation, i.e.
;1 UT
1 42f;1
and a generalization of Szego's formula to the nonstationary case (see Proposition 3.4 below), nameley
T logdet1 1 2
Z 1 0
Z
;log2f(u)]ddu
we obtain the following likelihood function as an approximation of the exact Gaussian likelihoodL(e)T ()
L
(`)T () = 14 1 T
T
X
t=1
Z
;log 42f
t T
d + 182T X0UT(f;1)X:
The subsitution r+s2 ] =t, r;s = k yields X0UT(f;1)X = XT
rs=1XrTXsT
Z
;expfi(r;s)gf
1 T
r + s 2
;1
d
= XT
t=1
Z
;f( tT);1Xk X t+k=2]TX t;k=2]Texp(ik)d 4
where the second sum is over all k such that 1 t + k=2]t;k=2] T and x] is the largest integer smaller or equal to x. Thus,
L
(`)T () = 14 1 T
T
X
t=1
Z
;
nlog42f( tT)+ ~IT(Tt) f(Tt)
od (2.3)
where
~IT(u) := 121 uT+k=2] uTXk;k=2]TX uT+k=2]TX uT;k=2]Texp(ik):
~IT(Tt) may be regarded as a local version of the periodogram at timet. It was introduced by Neumann and von Sachs (1997) as a starting point for a wavelet estimate of the time- varying spectral density. We will call ~IT(Tt) the preperiodogram at time t.
There exist several nice relations between the preperiodogram and the ordinary peri- odogram and the above likelihood and the Whittle-likelihood: We have
IT() = 12Tj
T
X
r=1XrTexp(;ir)j2
= 12 k=;TX(T;1;1)
T1
0
@
TX;jkj
t=1 XtTXt+jkjT
1
Aexp(ik) (2.4)
= 1T Xt=1T 1 2
X
1 t+k=2] t;kk=2]TX t+k=2]TX t;k=2]Texp(ik)
= 1T Xt=1T ~IT( tT)
i.e. the periodogram is the average of the preperiodogram over time. (2.4) means that the periodogram IT() is the Fourier transform of the covariance estimator of lag k over the whole segment while the preperiodogram ~IT(Tt) just uses the pair X t+k=2]X t;k=2] as a kind of \local estimator" of the covariance of lagk at timet (note that t+k=2];t;k=2] = k). For this reason Neumann and von Sachs also called ~IT(Tt) the localized periodogram.
A classical kernel estimator of the spectral density of a stationary process at some frequency0 therefore can be regarded as an average of the preperiodogram over all time
5
points and over the frequencies in the neighbourhood of 0. It is therefore plausible that averaging the preperiodogram around some frequency0 and around some time-pointt0
gives an estimate of the time-varying spectrumf(tTo).
For a locally stationary process the preperiodogram is asymptotically unbiased. How- ever, its variance explodes as T tends to innity. Therefore, smoothing over time and frequency is essential to make a consistent estimate out of it. This smoothing is im- plicitely contained in the likelihood L(`)T (). Instead of using ~IT(Tt) in (2.3) one could think of using the classical periodogram over some small segment of data around t. Such a likelihood was studied in Dahlhaus (1997). The preperiodogram has advantages over such an estimate since a classical periodogram always contains some implicit smoothing over time (even if it is calculated over a small segment) which in the case of time varying spectra means that some information is getting lost. For this reason the preperiodogram is a valuable raw estimate, e.g. in (2.3) or for wavelet smoothing as in Neumann and von Sachs (1997). Another advantage in the context of Whittle estimation is that no segment length (e.g. for a periodogram) has to be selected.
The above likelihood L(`)T () coincides with the Whittle likelihood in the stationary case: If a stationary model is tted, then f(u) = f() is constant over time and the likelihood becomes
L
(`)T () = 14Z;flog42f() + T1
PT
t=1 ~IT(Tt) f() gd
= L(W)T ():
For that reason the results on the asymptotic behaviour of the minimizer of L(`)T () contain most of the results on the classical Whittle estimate as a special case (apart from our restriction to Gaussian processes). Among the large number of papers we mention the results of Dzhaparidze (1971) and Hannan (1973) for univariate time series, Dunsmuir (1979) for multivariate time series and Hosoya and Taniguchi (1982) for misspecied mul- tivariate time series which follow as a special case from Theorem 3.8 below. A general overview over Whittle-estimates for stationary models may be found in the monograph of
6
Dzhaparidze (1986). We also mention the results of Kluppelberg and Mikosch (1996) on Whittle estimates for linear processes where the innovations have heavy tailed distribu- tions, of Fox and Taqqu (1986) on Whittle estimates for long range dependent processes and of Robinson (1995) on semiparametric Whittle estimates for long range dependent processes. These results however are not a special case of Theorem 3.8.
There is another important aspect of the above likelihood approximation: The likeli- hood is of the form
L
(`)T () = 1T Xt=1T `T( tT) with
`T( tT) = 1 4
Z
;flog42f( tT)+ ~IT(Tt) f(Tt)gd
i.e. L(`)T () has a similar form as the negative log-likelihood function of iid observations where `T(Tt) is the negative log-likelihood at time point t. In the present dependent situation`T(Tt) may still be regarded as the negative log-likelihood at time pointt which now in addition contains the full information on the dependence (correlation) structure of XtT with all the other variables.
To illustrate this we give two examples:
1. Suppose we have the situation of nonparametric regression with heteroscedastic errors, i.e. our model is
XtT =m( tT)+(t
T )"t "t iid N(01)
with m(u) = m(u), (u) = (u). This process is locally stationary in the sense of Denition 3.1 below. Since the mean is di erent from zero, the preperiodogram in
`(Tt) contains an extra term (see (3.7) below). It is easy to show that in this case
`T( tT) = 1
2 log22( tT)+ 1
22(Tt)(XtT ;m( tT))2 which is exactly the Gaussian log-likelihood.
7
2. Suppose
XtT =a( tT)Xt;1T +( tT)"t "tiid N(01)
with a(u) = a(u), (u) = (u). Then XtT is locally stationary with time varying spectrum
f(u) = 2(u)
2 j1;a(u)eij;2 leading to
`T( tT) = 1
2 log 22( tT)+ 1
22(Tt)(XtT ;a( tT)Xt;1T)2 +rt
with
rt =a( tT)2(X2tT ;X2t;1T) i.e. PTt=1rt=Op(1).
The fact that`T(Tt) can be seen as the local likelihood of the process at timet opens the door for various nonparametric estimation methods. In this situation the model is parametrized by one or several curves in time (eg. as in Example 3.2).
Recall that several nonparametric estimation techniques can be written as the solution of a least squares problem, for example for the simple nonparametric regression problem
XtT =m( tT)+"t a) a kernel estimate can be written as
m(u) = argmin^ m 1 bTT
X
t K
u;t=T bT
fXtT ;mg2 where K is the kernel and bT is some bandwidth
8
b) a local polynomial t can be written as
^c(u) = argminc 1 bTT
X
t K
u;t=T bT
(
XtT ; d
X
j=0cj( tT ;u)j
)2
where c = (c0::: cd)0 are the coecients of the tted polynomial at time u c) an orthogonal series estimator (e.g. wavelets) can be written as
"
= argmin 1 T
X
t
(
XtT ; J
X
j=1jj( tT)
)2
together with some shrinkage to obtain the nal estimator ^. Here the j() (j = 1::: J) denote some orthonormal functions. J usually increases with T.
Note that the f:::g-brackets always contain the negative log likelihood of the param- eters up to some constants.
Suppose now we have a locally stationary model which is parametrized by one or several curves in time. By using the local likelihood we may dene completely analogous to above
a) a kernel estimate by
^(u) = argmin 1 bTT
T
X
t=1K
u;t=T bT
`T
tT
b) a local polynomial t by
^c(u) = argminc 1 bTT
T
X
t=1K
u;t=T bT
`T
Xd
j=0cj( tT ;u)j tT
!
c) an orthogonal series estimator (e.g. wavelets) by
"
= argmin 1 T
T
X
t=1 `T
XJ
j=1jj( tT) t T
!
together with some shrinkage of ".
9
In case of several parameter curves (a vector of curves) , the cj and the j are also vectors. In case of a multivariate process or a process with mean di erent from zero the denition (3.6) of `T(Tt) has to be used.
It is obvious that the properties of these estimators have to be investigated in detail.
However, this is quite complicated and would exceed the scope of this paper. We only want to demonstrate that the likelihood representation may have important applications in nonparametric estimation as well.
In the next section we prove that L(`)T () indeed is a good approximation of the exact Gaussian likelihoodL(e)T (). Furthermore, we consider parametric models and prove that the resulting parameter estimates are consistent, asymptotically normal and ecient.
We do this for a larger class of processes than discussed in this section. In particular, we study multivariate locally stationary processes and allow the mean to be a function di erent from zero which introduces extra terms into the above expressions.
3 Asymptotic properties of parameter estimates
We start with the denition of a multivariate locally stationary process.
(3.1) Denition
A sequence of multivariate stochastic processes XtT = (XtT(1):::XtT(d))0 (t = 1:::T) is called locally stationary with transfer function matrix Ao and mean function vector if there exists a representationXtT =
t T
+
Z
;exp(it)AotT()d() (3.1)
where
(i)() is a stochastic vector process on ;] with a() = a(;) and cumfda1(1):::dak(k)g=
Xk j=1j
!
ha1:::ak(1:::k;1)d1:::dk
10
where cumf:::gdenotes the cumulant of k;th order, ha = 0, hab() = ab,
jha1:::ak(1:::k;1)j constk for all a1:::ak 2 f1:::dg and () =P1j=;1( + 2j) is the period 2 extension of the Dirac delta function.
(ii) There exists a constant K and a 2-periodic matrix valued function A : 01]R!
Cdd with A(u) = A(u;) and
supt jAotT()ab ;A
t T
abjKT;1 (3.2)
for allab = 1::: d and T 2N. A(u) and (u) are assumed to be continuous in u.
f(u) := A(u)A(u)0 is the time varying spectral density matrix of the process.
Processes with an evolutionary spectral representation were introduced and investi- gated by Priestley (1965, 1981). The above denition is the multivariate generalization of the denition of univariate local stationarity as given in Dahlhaus (1997). This approach to local stationarity may be regarded as a setting which allows for a meaningful asymptotic theory for processes with an evolutionary spectral representation. The classical asymp- totics for stationary sequences is contained as a special case (if and A do not depend on t). A detailed discussion of this denition and a comparison to Priestley's approach can be found in Dahlhaus (1996c). Another denition of local stationarity has recently been given by Mallat, Papanicolaou and Zhang (1998). We remark that the methods presented in this paper do not depend on the special denition of local stationarity. In some sense the above denition is only a framework for investigating the asymptotic properties of the estimates.
Examples of locally stationary processes in the univariate case can be found in Dahlhaus (1996a). For the multivariate case we give the following examples.
(3.2) Examples
(i) Suppose Yt is a multivariate stationary process, () is a vector function and () is a matrix function. ThenXtT =( tT)+( t T )Yt 11
is locally stationary. If Yt is an iid sequence we have the situation of multivariate non- parametric regression.
(ii) Suppose XtT is a time varying multivariate ARMA-model, that isXtT is denied by the di erence equations
p
X
j=0#j( tT)
Xt;jT ;(t;j T )
=Xq
j=0$j( tT)(t;j T )"t;j
where"tare iid with meanzero and variance-covariance matrixIdand #o(u) $o(u) Id. Under regularity conditions on the coecient functions #j(u) and $j(u) it can be shown similarly to the univariate case (Dahlhaus, 1996a, Theorem 2.3) that these di erence equations dene a locally stationary process of the form (3.1). The time varying spectral density of the process is
f(u) = 12#(u);1$(u)(u)$(u;)0#(u;)0;1
where #(u) = Ppj=0#j(u)eij and $(u) = Pqj=0$j(u)eij. We omit details of the derivation. However, we remark that in this case the functions AotT() and A(t=T) do not coincide. They only fulll (3.2).
In the following we look at parametric locally stationary models. An example is the case where the curves in the above examples are parametrized in time, e.g. by polynomials (for an example see Dahlhaus, 1997, Section 6).
Let X = (X1T0 ::: XTT0 )0, = ((T1)0::: (TT)0)0, and let the dT dT-matrices T(AB) and UT( ) be dened by
T(AB)rs=
Z
;exp(i(r;s))AorT()BosT(;)0d (3.3) and
UT( )rs =
Z
;exp(i(r;s))
1 T
r + s 2
d (3.4)
12
(rs = 1:::T) where AorT(), BorT() and (u) are dd-matrices. Then the exact Gaussian likelihood is
L
(e)T () := d2 log(2)+ 1
2T logdet + 12T(X;)0;1(X ;) (3.5) where = T(AA) and E(X ;)(X;)0= T(AA) with A from Denition 3.1.
We now proceed as in the univariate case (Section 2) to nd a local likelihood approx- imation. We use a generalisation of the multivariate Szego identity (see Proposition 3.4 below) and UT(412f;1) as an approximation of ;1 to obtain
LT() := L(`)T () := 141 T
T
X
t=1
Z
;log(2)2ddetf( tT)]d + 182T (X;)0UT(f;1)(X;)
= 1T Xt=1T 1 4
Z
;
log
(2)2ddetf( tT)
+ tr
f( tT);1~IT( tT) d
=: 1T Xt=1T `T( tT) (3.6)
where
~IT(u)ab:= 12 1 Xk
uT+k=2] uT;k=2]T
X uT+k=2]T(a) ;(a)
uT + k=2]
T
X uT(b);k=2]T ;(b)
uT;k=2]
T
exp(;ik) (3.7) is the multivariate version of the preperiodogram.
In the univariate case and for = 0 this is the likelihood we have already discussed in Section 2. We call`T(Tt) the local likelihood at timet. If the mean is not zero and one is not interested in modelling the mean one may use ~IT^(u) instead of ~IT(u) where ^ is the arithmetic mean or some kernel estimate (if the mean is not believed to be constant over time).
Before investigating the asymptotic properties of the minimizer of LT() we prove some results on the likelihood approximation itself. First we state two results which show
13
that UT(f42fg;1) and LT() are approximations of ;1 and L(e)T respectively. We also show that
L() := 14
Z 1 0
Z
;
logdetf(u) + trf(u);1f(u)]ddu + 14
Z 1
0 ((u);(u))0f;1(u0)((u);(u))du (3.8) is the limit of LT() and L(e)T ().
The technical parts of the following proofs consist of the derivation of properties of products of matrices T(AB), T(AA);1 and UT( ). These properties are derived in the appendix. In particular Lemma A.1, A.5 and A.8 are of relevance for the following proofs.
For convenience we refer in the following proposition to Assumption A.3 in the ap- pendix concerning the smoothness of the transfer function and the mean. These conditions are fullled under Assumption 3.6 below. BykAkand A we denote the spectral norm and the Euclidean norm of a matrixA (cp. (A.1) and (A.2)). kvk2 is the Euclidean norm of a vector.
(3.3) Proposition
Suppose the matrices A and fulll the smoothness conditions of Assumption A.3 (i) - (iii) (appendix) with existing and bounded derivatives @u@22@@ A(u)aband eigenvalues of (u) which are bounded from below uniformly in u and . Then we
have 1
T T(AA);1;UT(f42A "A0g;1) 2 =O(T;1ln3T) (3.9)
and 1
T UT( );1 ;UT(f42 g;1) 2 =O(T;1ln;23T):
Proof. Let T = T(AA) and UT = UT(f42A "A0g;1). We obtain with Lemma A.1 (b,c) and Lemma A.5
T 1 ;T1 ;UT 2
T I1 ;1=2T UT1=2T 2k;T1k2
K(d; 2
T trfUTTg+ 1TtrfUTTUTTg):
14
Lemma A.7 (i) now implies the result. The second result is obtained in the same way with Lemma A.8.
We now state the generalisation of the Szego identity (cf. Grenander and Szego, 1958, Section 5.2) to multivariate locally stationary process.
(3.4) Proposition
Suppose A fullls Assumption A.3 (i), (ii), with bounded derivatives@2
@u2 @
@A(u)ab. Then we have with f(u) = A(u)A(u;)0 T log det1 T(AA) = 12
Z 1 0
Z
;log(2)ddetf(u)]ddu + O(T;1ln11T):
If A = A depends on a parameter and fullls the smoothness conditions of Assumption 3.6 (iii), (iv), then the O(T;1ln11T) term is uniform in .
Proof. The proof can be found in A.9 of the appendix.
From now on we set ri = @@i and r2ij = @@i@2 j.
(3.5) Theorem
Suppose XtT is a locally stationary Gaussian process with transfer func- tion matrix Ao and mean function vector and we t a locally stationary model with transfer function matrix Ao and mean function vector . Suppose further that all eigen- values of f(u) = A(u)A(u)0 are bounded from below uniformly in u and and the components of A, A, , are dierentiable with uniformly bounded derivatives@2
@u2 @
@A(u)ab, @u@22@@ A(u)ab, @u@ (u)a, @u@ (u)a respectively. Then we have (i)
LT();L(e)T () = OP(T;1ln11T):
(ii) If in addition the rst derivatives rjA(u)ab and rj(u)a fulll the above smooth- ness properties, we also have
rjLT();rjL(e)
T () = OP(T;1ln23T):
15
(iii) Furthermore,
L() = limT
!1
EL(e)T () = limT
!1
ELT() and
L
(e)T ()!P L() LT() !P L():
A similar result also holds for the higher order derivatives of the likelihoods. We conjecture that also a uniform result (in) holds and that the log-terms and the Gaussian assumption can be dropped. However, a uniform result requires muchmore e ort. In order not to blow up the paper we omit these generalisations.
Proof. (i) We obtain with Proposition 3.4 andBT := T(AA);1;UT(f42A "A0g;1)
LT();L(e)T () = 12T(X;)0BT(X ;) +O(T;1ln11T):
Since
T (X1 ;)BT(X ;)
= 1T(X ;)BT(X;) (3.10)
+ 2T(X ;)0BT(;) + 1T(;)0BT(;) we obtain with Lemma A.8 and = T(AA)
EfLT();L(e)T ()g = 12TtrfBTg+ 12T(;)0BT(;) +O(T;1ln11T)
= O(T;1ln11T) and
varfLT();L(e)T ()g = 12T2trfBTBTg+ 1T2(;)0BTBT(;)
= O(T;2ln23T) 16
which implies the result.
(ii) We obtain with Lemma A.8,BT as above and CT :=;T(AA);1fT(rjAA)+
T(ArjA)gT(AA);1;UT(rjf42A "A0g;1)
rjLT();rjL(e)
T () = 12T(X ;)0CT(X ;); 1
T (rj)0BT(X ;) +O(T;1ln11T):
Analogously to above we obtain with Lemma A.8 E(rjLT();rjL(e)
T ()) = O(T;1ln23T) and
var(rjLT();rjL(e)
T ()) = O(T;2ln47T) which gives the result.
(iii) follows similarly to (i) (use e.g. BT = T(AA);1 in the above derivation).
Theorem 3.5 (iii) basically gives the asymptotic Kullback-Leibler-information diver- gence of two multivariate locally stationary processes: If XtT( ~XtT) are multivariate lo- cally stationary with spectral densities f = A "A0( ~f = ~A"~A0), mean functions (~) and Gaussian densities g(~g), then we obtain for the information divergence
D( ~f ~f) = limT
!1
T E1 glog g~g
= 14
Z 1 0
Z
;flogdet ~f(u)f(u);1] + tr ~f(u);1f(u);I]gddu + 14
Z 1
0 (~(u);(u))0~f(u0);1(~(u);(u))du:
This is the time average of the Kullback-Leibler divergence in the stationary case (cf.
Parzen, 1983, for the univariate stationary case with mean zero).
We now study the behaviour of
^T := argmin
2 LT():
17
Furthermore, let
0 := argmin
2 L():
The results are proved under the following assumptions.
(3.6) Assumption
(i) We observe a realisation X1T:::XTT of a d-dimensional sta- tionary Gaussian process with true mean function vector and transfer function matrixAo and t a class of locally stationary Gaussian processes with mean function vector and transfer function matrixAo, 2%Rp, % compact.(ii) 0= argminL() exists uniquely and lies in the interior of %.
(iii) The components ofA(u) are di erentiable in u and with uniformly continuous derivativesr2ij @@u22@@ A(u)ab.
(iv) All eigenvalues of f(u) = A(u)A(u;)0 are bounded from below by some constant C > 0 uniformly in , u and .
(v) The components of A(u) are di erentiable in u and with uniformly bounded derivatives @u@ @@A(u)ab.
(vi) The components of (u), (u), ri(u) and r2ij(u) are di erentiable in u with uniformly bounded derivatives.
In the case where the model is correctly specied, i.e. A(u) = A(u) and (u) = (u) with some 2% one can show that 0 =.
(3.7) Theorem
Suppose that Assumption 3.6 holds. Then^T P
!0:
Proof. The basic idea is taken from Walker (1964), Section 2. In Theorem 3.5 (iii) we have proved that
LT() !P L():
18
Since 0 is assumed to be unique it follows that for all 1 6= 0 there exists a constant c(1)> 0 with
Tlim!1P(LT(1);LT(0)< c(1)) = 0:
Furthermore, we have with a mean value "
LT(2);LT(1) = (2;1)0rLT(") where (cp. (3.10))
riLT() = 14 1 T
T
X
t=1
Z
;tr
f( tT)rif( tT);1
d
+ 182T (X;)0UT(rif;1)(X ;) (3.11)
;
412T (ri)0UT(f;1)(X ;)
= 182T (X ;)0UT(rif;1)(X ;) + 142Tri
n(;)0UT(f;1)o(X ;) + const: (3.12) with a constant independent ofX (but dependent on and T). With the Cauchy-Schwarz inequality and Lemma A.1(h) we get
T (1 ri)0UT(f;1)(X ;)
T1
n(ri)0UT(f;1)(ri)(X ;)0UT(f;1)(X ;)o1=2
1
Tkrik22
1=22
TkXk22+ 2Tkk22
1=2
kUT(f;1)k which by Assumption 3.6 and Lemma A.5 is uniformly bounded by
K + K 1TkXk22:
Similarly, we can estimate the other terms in (3.11) leading to 19
22supU(1)jLT(2);LT(1)jK(1 + 1TX0X)
with some constant K. Since ET1X0X = T1kk22 ! PTa=1R01a(u)2du and varT1X0X =
T22trf2g T2kk2 KT (Lemma A.1 and A.5)T;1X0X is bounded in probability. Thus there exists for all 1 6=0 a c(1)> 0 and a = (1) with
Tlim!1P( inf
2
2U(1)LT(2);LT(0)c(1)=2)
1;Tlim
!1
P(LT(1);LT(0)< c(1));Tlim
!1
P( sup
22U(1)jLT(2);LT(1)jc(1)=2)
= 1:
A compactness argument as in Walker (1964) implies the result.
(3.8) Theorem
Suppose that Assumption 3.6 holds. Then we havepT(^T ;0)!D N(0;;1V ;;1) with
;ij = 14
Z 1 0
Z
;tr(f;f0)rijf;01ddu; 1 4
Z 1 0
Z
;tr(rif0)(rjf;01)ddu + 14
Z 1
0 r2ij((u);0(u))0f;01(u0)((u);0(u))du and
Vij = 14
Z 1 0
Z
;trf (rif;1)f (rjf;1)ddu + 12
Z 1 0
Z
;
ri((u);0(u))0f;01(u0)f(u0)rjf;01(u0)((u);0(u))du:
Proof. We obtain with the mean value theorem
riLT(^T);riLT(0) =fr2LT((i)T )(^T;0)gi
20
withjT(i);0j j^T;0j(i = 1:::p). If ^T lies in the interior of % we haverLT(^T) = 0.
If ^T lies on the boundary of %, then the assumption that 0 is in the interior implies
j^T;0j for some > 0, i.e. we obtain P(pTjrLT(^T)j")P(j^T;0j) !0 for all" > 0. Thus, the result follows if we prove
(i) r2LT(T(i));r2LT(0)!P 0 (ii)r2LT(0)!P ;
(iii) pTrLT(0)!D N(0V ):
We now obtain from (3.11)
r2ijLT() = ; 1 4 1
T
T
X
t=1
Z
;tr
f( tT)r2ijf( tT);1
d (3.13)
;
41 1 T
T
X
t=1
Z
;tr
rif( tT)rjf( tT);1
d + 182T (X;)0UT(r2ijf;1)(X ;)
+ 142T (ri)0UT(f;1)(rj)
;
412T (ri)0UT(rjf;1)(X ;)
;
412T (rj)0UT(rif;1)(X ;)
;
412T (r2ij)0UT(f;1)(X ;):
To prove (i) we have to consider the above terms separately. The assertion is obvious for the rst and second term. LetT =(i)T . The remaining terms of (3.13) can all be written as sums of expressions of the form
T X1 0UX 1T0UX or 1T01U2 (3.14) withU being equal toUT(f;1),UT(rif;1) orUT(r2ijf;1). Lemma A.5(iii) implieskUT;
U0k!0 in probability. Furthermore, T1kT ;0k22 !0 in probability. This implies for example with the Cauchy-Schwarz inequality
21
