Definition 4.2.1. Let x(t) = µ nt
+ξ(t) (t = 1, . . . , n) as before. Then, the least squares estimator θˆof θ is defined by the equation
Ln θˆn
= inf
τ∈Θ
Ln(τ), where L(τ) = Pn
t=1
x(t)−µ nt, τ2
and Θ =Rν ×Sl.
To improve the asymptotic variance, we consider the more general weighted least squares estimator θˆwn which is defined by
Lwn θˆnw
= inf
τ∈Θ
Lwn(τ), where Lwn(τ) = Pn
t=1w n+1t x(t)−µ nt, τ2
. Here, w : (0,1)→ R∗+ denotes a contin-uously differentiable weight function satisfying the regularity assumptions
Z 1 0
|w(s)|2ds <∞ (4.4) and
1 n
n
X
t=1
w n+1t
h nt n→∞
−−−→
Z 1 0
h(t)w(t) dt <∞ ∀h∈ C([0,1]) (4.5)
where the integral the integral on the right-hand side in (4.5) is assumed to exists in the sense of Lebesgue. Moreover, we assume that w satisfies the assumption of theorem 3.2.14.
4.2.2. As said above, we consider a fully parametric regression setting. To avoid a poten-tial confusion of this approach with related nonparametric spline models, we will briefly recall the basic ideas of these conceptual different techniques.
A popular nonparametric method are the so-called smoothing splines as introduced by Whittacker [Whi23], Schoenberg [Sch64] and Reinsch [Rei67]: let
yi =f(ti) +εi (i= 1, . . . , n)
with 0 < t1 < . . . < tn < 1 denoting fixed design points, f : [0,1] → R a real function such that R1
0
f(m)
2(t) dt < ∞ and εi some stochastic noise. Then, for fixed λ > 0 the smoothing spline sλ is defined as the solution of the problem
s:[0,1]→minR, R1
0 |s(m)|2(t) dt<∞
(1 n
n
X
i=1
(yi −s(ti))2+λ Z 1
0
s(m)
2(t) dt )
. (4.6)
The parameter λ controls the tradeoff between smoothness and goodness-of-fit. If λ →
∞, the solution sλ converges to a polynomial of order m. If λ → 0, sλ will almost interpolate the data. Although this optimisation problem is initially defined over an infinite dimensional vector space, one can show that in the case n > m, the unique solution of this problem is given by a natural spline of order 2m with knots at t1, . . . , tn, see e.g. [Eub88, lemma 5.2, page 283]. Recall, that a spline of order 2m with knots at t1, . . . , tn is called natural spline if it is polynomial of order m outside [t1, tn]. One can can show that the space of all natural splines of order 2m with knots at t1, . . . , tn has dimension n. Hence, a solution to (4.6) can be given explicitly: denote by xi, . . . , xn a basis of the vector space of the natural splines and define
Xn = [xj(ti)]i,j=1,...,n ∈Rn×n. Furthermore, define
Ω = Z 1
0
x(m)i (t)x(m)(t)j dt
i,j=1...,n
. Then, the solution to (4.6) can be written as sλ = Pn
i=1cixi where (c1, . . . , cn) satisfies the equation system
XtnXn+nλΩ
c=Xtny
which can be considered as a ridge regression problem. The parameterλis usually referred to as penalising, smoothing or regularisation parameter. The estimated trend function can now be written as
sλ(t) =
n
X
i=1
yiKn(t, ti;λ) with
Kn(t, s;λ)→ e−|t−s|/
√
λ+e−2/
√ λe|t−s|/
√
λ +e−(t−s)/
√
λ+e(t+s+2)/
√ λ
2√
λ(1−e−2/
√
λ) +O (nλ)−1
uniformly insandtforλ →0 andnλ→ ∞, see [Eub88, theorem 5.4, page 247]. Based on this asymptotic kernel representation, asymptotic values of the integrated mean squared error can be derived. Note that λ corresponds to the bandwidth of this asymptotic kernel. For more information, including estimation procedures for λ see e.g. [Eub88] and in particular [Wah92]. For smoothing splines with autocorrelated short-memory errors, see [OWY01, Wan98, RKW92].
Agarwal and Studden [AS80, AS78] consider the regression model yi =g(xi) +εi
where g : [0,1] → R is d times differentiable and the εi are iid with finite variance. To estimate g, they fit a spline of orderdto the data. Based on the asymptotic mean squared error, the optimal number and placement of the knots are determined.
4.2.3. In our context, weighted regression is used to improve the asymptotic variance of the estimator. It is well-known that the usual least squares estimator is asymptotically not efficient if applied to long-memory data, see [Yaj88, Yaj91]. Dahlhaus [Dah95] introduced the following class of weight function which lead in the case of polynomial regression to an asymptotically efficient estimator: let ξ(·) be either a long-memory or antipersistent process in the sense of definition 1.1.5. Define
wd(x) = x−d(1−x)−d
where d = 1−2α with α as in definition 1.1.5. Then, wd satisfies the conditions stated in definition 4.2.1. In fact, (4.4) follows by definition. As for (4.5), note that
1 n
n
X
t=1
wd n+1t h nt
= 1 n
bn/2c
X
t=1
wd n+1t h nt
+ 1 n
n
X
dn/2e+1
wd n+1t h nt
+o(1).
Define the sequence (fn)n∈N byfn(u) = Pbn/2c t=1 1ht−1
n+1, t n+1
h(u)wd n+1t h nt
, u ∈[0,1/2].
Then fn(u) → wd(u)h(u) for all u ∈ (0,12) and fn ≤ khk∞wd if d > 0 and fn ≤ khwdk∞
if d <0. Thus, by Lebesgue’s theorem, 1
n
bn/2c
X
t=1
wd n+1t h nt
→ Z 12
0
wd(u)h(u) du.
Likewise,
1 n
1
X
dn/2e+1
wd n+1t h nt
→ Z 1
1 2
wd(u)h(u) du,
which proves (4.5). The assumptions of theorem 3.2.14 are more demanding to prove.
Assume that ξ(·) has long memory. Then, we need to show that 1
n2
n
X
s,t=1 s6=t
w n+1t
w n+1s t−sn
−β n→∞
−−−→
Z 1 0
Z 1 0
w(s)w(t)
|s−t|β dsdt <∞ ∀β ∈(0,1). (4.7)
where the integral on the right-hand side in (4.7) is defined in the sense of Lebesgue. To verify (4.7), observe at first that the integral
Z 1 0
Z 1 0
s−d(1−s)−d(1−t)−dt−d
|s−t|β dsdt
exists for all d∈(−0.5,0.5) andβ ∈(0,1). Indeed, by substitution s=ut, Z 1
0
t−d Z t
0
s−d(t−s)−βdsdt = Z 1
0
t1−2d−β Z 1
0
u−d(1−u)−βdudt <∞.
By symmetry,
Z 1 0
Z 1 0
s−dt−d
|s−t|β dsdt <∞ and
Z 1 0
Z 1 0
(1−s)−d(1−t)−d
|s−t|β dsdt <∞.
In particular,
Z 12
0
Z 12
0
s−d(1−s)−d(1−t)−dt−d
|s−t|β dsdt <∞ and
Z 1
1 2
Z 1
1 2
s−d(1−s)−d(1−t)−dt−d
|s−t|β dsdt <∞.
Finally, we have Z 1
1 2
Z 12
0
(1−s)−ds−dt−d(1−t)−d
|s−t|β dsdt≤K Z 1
1 2
(1−t)−d Z 12
0
s−d(12 −s)−βdsdt <∞
which proves our claim. To show convergence of the Riemann sums in (4.7), define for u, v ∈[0,1/2] the function gn(u, v) by
gn(u, v) =
bn/2c
X
s,t=1 s6=t
1hs−1 n+1, s
n+1
h(u)1ht−1 n+1, t
n+1
h(v)wd n+1t
wd n+1s t−sn
−β.
Then
|gn(u, v)| ≤wd(u)wd(v)
12(u−v)
−β
and thus 1 n2
bn/2c
X
s,t=1 s6=t
w n+1t
w n+1s t−sn
−β =(n+ 1)2 n2
Z 12
0
Z 12
0
gn(u, v) dudv
−→
Z 12
0
Z 12
0
w(s)w(t)
|s−t|β dsdt
Extending this argument to the whole summation index, gives the desired result, i.e.
1 n2
n
X
s,t=1 s6=t
w n+1t
w n+1s t−sn
−β −→
Z 1 0
Z 1 0
w(s)w(t)
|s−t|β dsdt
We conclude the long-memory case by noting that the argument still holds if w in (4.7) is replaced by 1.
If ξ(·) is antipersistent, then d <0 and and w is absolutely continuous by definition.
Furthermore, we need to show that
− 1 n2
n
X
t=1
f n+1t
w n+1t
n
X
s=1,s6=t
g n+1s
w n+1s
−g n+1t
w n+1t
s−tn
β
−→ − Z 1
0
f(t)w(t) Z 1
0
g(s)w(s)−g(t)w(t)
|s−t|β dsdt
(4.8)
for all β ∈(1,2) and all (not necessarily continuous) piecewise polynomialsf, g : [0,1]→ R.
To verify (4.8), we show at first that the integral on the right-hand side is indeed well-defined. To start with, consider the special case g =f = 1 and integration over the
set [0,12]×[0,12]. Since wd0(u)≤const·s|d|−1 for all 0< s≤u≤t < 12, we find that Z 12
0
wd(t) Z 12
0
|wd(s)−wd(t)|
|s−t|β dsdt=2 Z 12
0
wd(t) Z t
0
wd(t)−wd(s) (t−s)β dsdt
=2 Z 12
0
wd(t) Z t
0
Rt
sw0(u) du (t−s)β−1 dsdt
≤const Z 12
0
wd(t) Z t
0
s|d|−1
(t−s)β−1 dsdt
<∞,
by the same arguments used to establish (4.7). Likewise, Z 1
1 2
wd(t) Z 1
1 2
|wd(s)−wd(t)|
|s−t|β dsdt <∞ and
Z 12
0
wd(t) Z 1
1 2
|wd(s)−wd(t)|
|s−t|β dsdt ≤ Z 12
0
wd(t) Z 1
1 2
1
(s−t)β dsdt
= Z 12
0
wd(t)(1−t)1−β−(12 −t)1−β
β−1 dsdt
<∞.
So for f =g = 1, the integral in (4.8) is indeed finite. Now, let g(s) = (s−η)k+ for some η ∈[0,1) and some integer k≥0. Then
Z 1 0
wd(t) Z 1
0
g(s)wd(s)−g(t)wd(t)
|s−t|β dsdt
= Z 1
0
wd(t) Z 1
0
g(s)wd(s)−g(s)wd(t)
|s−t|β dsdt+ Z 1
0
w2d(t) Z 1
0
g(s)−g(t)
|s−t|β dsdt
=I1+I2, say. But
I1 ≤ kgk∞
Z 1 0
wd(t) Z 1
0
|wd(s)−wd(t)|
|s−t|β dsdt <∞ and, for k ≥1,
I2 ≤const Z 1
0
wd2(t) Z 1
0
|s−t|1−βdsdt <∞.
On the other hand, we obtain for k= 0 I2 =−
Z η 0
w2d(t) Z 1
η
1
|s−t|β dsdt+ Z 1
η
wd2(t) Z η
0
1
|s−t|β dsdt <∞.
By linearity, we conclude that Z 1
0
wd(t) Z 1
0
|g(s)wd(s)−g(t)wd(t)|
|s−t|β dsdt <∞ for all piecewise polynomials g. It follows that
Z 1 0
f(t)wd(t) Z 1
0
g(s)wd(s)−g(t)wd(t)
|s−t|β dsdt
≤kfk∞ Z 1
0
wd(t) Z 1
0
|g(s)wd(s)−g(t)wd(t)|
|s−t|β dsdt <∞.
for all piecewise polynomials f. To show that the Riemann sums in (4.8) converge, we argue similarly as in the proof of (4.7): Let g(u) = (u−η)k+ be as above and let f be a piecewise polynomial. Define for u, v ∈[0,1/2] the function hn(u, v) by
hn(u, v) =
bn/2c−1
X
s,t=1 t>s
1hs−1 n+1, s
n+1
h(u)1h t n+1,t+1
n+1
h(v)f n+1t
wd t n+1
×
"
g n+1s
wd n+1s
−g n+1t
wd n+1t
t−sn
β
# . Then, for s, t= 1. . .bn/2c with t > s,
g n+1s
wd n+1s
−g n+1t
wd n+1t
t−sn
β
≤g n+1s wd t n+1
−wd s n+1
t−sn
β +wd n+1t |g n+1t
−g n+1s
|
t−sn
β .
Therefore, for all u∈(0,12) and v ∈(u,12) hn(u, v)≤ kfk∞kwdk∞kgk∞
wd(v)−wd(u)
1
2|u−v|β
+kfk∞kwdk2∞
g(v)−g(u)
1
2|u−v|β
and at the same time
hn(u, v)→f(v)wd(v)
g(u)wd(u)−g(v)wd(v)
|u−v|β
Thus, by Lebesgue’s theorem and symmetry,
− 1 n2
bn/2c−1
X
t=1
f n+1t
w n+1t
bn/2c−1
X
s=1,s6=t
g n+1s
w n+1s
−g n+1t
w n+1t
s−tn
β
−→ − Z 12
0
f(t)w(t) Z 12
0
g(s)w(s)−g(t)w(t)
|s−t|β dsdt
.
Similar arguments apply to other subsets of the summation index and/or area of integra-tion, consider for example the functions
˜hn(u, v) =
n
X
s,t=dn/2e+1 t>s
1hs−1 n+1, s
n+1
h(u)1h t n+1,t+1
n+1
h(v)f n+1t
wd n+1t
×
"
g n+1s
wd s n+1
−g n+1t wd t
n+1
t−sn
β
# . and
h¯n(u, v) =
n
X
t=dn/2e+1
bn/2c−1
X
n+1−t<ss=1
1h s n+1,s+1
n+1
h(u)1h t n+1,t+1
n+1
h(v)f n+1t wd t
n+1
×
"
g n+1s
wd n+1s
−g n+1t
wd n+1t
t−sn
β
# .
Finally, for the antipersistent case we need to show that that w ∈ I|d|,p(R) and w ∈ I|2d|,q(R) for some 1< p < |d|1 , 1< q < |2d|1 such that 1p+1q = 1+|d|withd= 1−α2 . This can be done by successively applying the results of section 2.5: define ˜wd(x) = 1]0,1/2](x)xd. Applying the reflexion operator Q and proposition 2.5.5 on example 2.5.1 (a= 0, b= 1), shows that
˜
wd∈I|2d|,p for all p≤ 1 d
Since the restriction ofh(x) = (1−x)|d|on ]0,12] can be extended toRsuch that it satisfies the assumptions of proposition 2.5.4, we conclude that
wd∈I|2d|,p for all p≤ 1 d
Similar arguments for the interval [1/2,1] lead to wd ∈I|2d|,p for all p≤ d1. Likewise, we can show that wd∈I|d|,p for all p≥1.
As in the long-memory case, note that all arguments still hold ifw is replaced by 1.
4.2.4. If we replace the general h in (4.5) by a (not necessarily continuous) piecewise polynomialµ(·, θ), then the assertion holds (in some sense) uniformly inθ. More precisely, setνn= Pn 1
t=1w(n+1t ) Pn
t=1w n+1t t
n with t
n denoting the Dirac measure at nt. By (4.5), Z 1
0
h(s)dνn(s)→ 1 R1
0 w(s)ds Z 1
0
h(s)w(s) ds
for all h∈ C([0,1]), i.e. (νn)n∈N converges weakly to the measure ν defined by ν(]a, b]) :=
Z b a
1[0,1](s) 1 R1
0 w(s) dsw(s) ds.
Denote by FB = {µ(·, θ) :θ ∈B} a family of (not necessarily continuous) piecewise polynomials for some bounded set B ⊂Θ. It then follows from [BT67, theorem 1, p. 2]
that
limn sup
θ∈B
Z
µ(θ, s) dνn(s)− Z
µ(θ, s) dν(s)
= 0, (4.9)
which entails
limn sup
θ∈B
1 n
n
X
t=1
w n+1t
µ nt, θ
− Z
µ(s, θ)w(s) ds
= 0. (4.10)
4.2.5. We need to fix some additional notation. Define mj,kn ∈Rn by mj,kn =h
t
n −ηkbj,k
+
in
t=1 (j = 1, . . . , pk, k = 0, . . . , l). Given a set of knots k, the design matrix Mn,l,(pk),(bj,k)(k) is then defined by
Mn,l,(pk),(bj,k)(k) =
m1,0n , . . . ,mpn0,0,m1,1n , . . . ,mpn1,1, . . . ,m1,ln , . . . ,mpnl,l
∈Rn×ν. To avoid a too cumbersome notation, we often omit the subscript and write only Mn(k) or Mn. Moreover, we denote
Wn = diag w n+11
, . . . , w n+1n
∈Rn×n
and
hx, yiRn,w =
n
X
t=1
xtw n+1t
yt =x0Wny.
If w is the identity mapping, we just write hx, yiRn. It follows from (4.10) that for each k∈Sl, there exists N(k) such that Mn(k)0WnMn(k) is invertible for all n > N(k).
Fora, b∈[0,1] such that a < b and p, n∈N\ {0}, define the matrix Fa,bp,n by Fa,bp,n=
"
t
n −bancn j−1
1„banc n ,bbnc
n – t
n
#
t=1,...,n,j=1...,p
∈Rn×p.
The columns will be denoted by fj,na,b(j = 1, . . . , p). Moreover, define
va,bj,n =
fj,na,b
kfj,na,bk if fj,na,b 6= 0, 0 otherwise.
We define the matrices Va,bp,n by Va,bp,n=h
va,b1,n, . . . ,va,bp,ni
∈Rn×p.
Note that banc< t ≤ bbnc if and only if a < t/n ≤b and dane ≤t < dbne if and only if a≤t/n < b.
Forq =bbnc − banc ≥1, define Ua,bn by
Ua,bn = [δtj]t=1,...,n;j=banc+1,...,bbnc ∈Rn×q. Then, for a < b≤c < d, the following orthogonality relations hold:
Ua,bn ⊥Fc,dp,n, Uc,dn ⊥Fa,bp,n, Ua,bn ⊥Uc,dn , and Fa,bp,n ⊥Fc,dp,n.
For a given matrix V ∈Rn×s, denote by sp (V) the corresponding column space and by prV,w the orthogonal projection on sp (V) with respect toh·,·i
Rn,w. IfV has full rank, prV,w may be written as prV,w =V(V0WnV)−1V0Wn, see [Har97, page 260].
Lemma 4.2.6. Let µbe identifiable and let ∆>0. Denote B(k,∆) =n
k˜ ∈Sl:
k˜−k
≤∆o . Then, there exists a δ >0 such that
lim inf
n
˜ inf
k/∈B(k,∆)
n−1
µn(θ)−prM
n(˜k),wµn(θ)
2 Rn,w
> δ.
Proof. We shall prove this lemma by contradiction: Assume there exists ∆>0 such that lim inf
n
inf
˜k/∈B(k,∆)
n−1
µn(θ)−prM
n(˜k),wµn(θ)
2 Rn,w
= 0.
In this case, we can find k∞ ∈Sl and a subsequence (kn(k))k∈N such that limkkn(k) =k∞
and |kn(k)−k|>∆ and limk
n−1(k)
µn(k)(θ)−prM
n(k)(kn(k)),wµn(k)(θ)
2 Rn(k),w
= 0.
By Pythagoras’ theorem, this is equivalent to limk
n−1(k)
µn(k)(θ)
2
Rn(k),w−n−1(k) prM
n(k)(kn(k)),wµn(k)(θ)
2 Rn(k),w
= 0 and so, by (4.10),
limk n−1(k) prM
n(k)(kn(k)),wµn(k)(θ)
2
Rn(k),w = Z 1
0
µ2(s, θ)w(s) ds. (4.11) Denote byηnk,j thej-th component ofknk (j = 1, . . . , l) and defineηnk,0 = 0 andηnk,l+1 = 1 for all k. Then, for b = maxlk=0maxpj=1k bj,k+ 1,
sp Mn(k) kn(k)
⊂
l
M
ν=0
sp
Fηn(k),bn(k),ν,ηn(k),ν+1
and thus
1 n(k)
prM
n(k)(kn(k)),wµn(k)(θ)
2 Rn,w
≤ 1 n(k)
l
X
ν=0
pr
Fηn(k),ν ,ηn(k),ν+1
n(k),b ,wµn(k)(θ)
2 Rn,w
(4.12)
Assume for the moment that n−1(k)
prFηn(k),ν ,ηn(k),ν+1
n(k),b ,wµn(k)(θ)
2
Rn,w
→
prFη∞,ν ,η∞,ν+1
∞,b ,wµ(·, θ)
2
L2([0,1],w)
.
(4.13)
where prFη∞,ν ,η∞,ν+1
∞,b ,wµ(·, θ) denotes the orthogonal projection withinL2([0,1], w) ofµ(·, θ) onto the subspace
Fη∞,b∞,ν,η∞,ν+1 =
sp s01]η∞,ν,η∞,ν](s), . . . , sb1]η∞,ν,η∞,ν](s)
if η∞,ν < η∞,ν+1,
0 if η∞,ν =η∞,ν+1.
It then follows from (4.12) and (4.13) that kµ(·, θ)k2L2([0,1],w)≤
pr⊕l
ν=0Fη∞,ν ,η∞,ν+1
∞,b ,wµ(·, θ)
2
L2([0,1],w)
, But k∞6=kand thus µ(·, θ)∈/ Ll
ν=0Fη∞,b∞,ν,η∞,ν+1, a contradiction.
It remains to prove (4.13): Since k∞ ∈ Sl, we have either |ηn(k),ν −ηn(k),ν+1| → 0 or η∞,ν < η∞,ν+1 (ν = 0, . . . , l). If |ηn(k),ν−ηn(k),ν+1| →0, then
n−1(k)
prFηn(k),ν ,ηn(k),ν+1
n(k),b ,wµn(k)(θ)
2
Rn,w
≤n−1(k)
bηn(k),ν+1n(k)c
X
t=bηn(k),νn(k)c
µ2
t n(k), θ
w n+1t
→0.
If η∞,ν < η∞,ν+1, thenFηn(k),bn(k),ν,ηn(k),ν+1 has full column rank fork large enough. For suchk we obtain
prFηn(k),ν ,ηn(k),ν+1
n(k),b ,wµn(k)(θ) =Fηn(k),bn(k),ν,ηn(k),ν+1an(k) with
an(k)=h
Fηn(k),bn(k),ν,ηn(k),ν+10Wn(k)Fηn(k),bn(k),ν,ηn(k),ν+1i−1
Fηn(k),bn(k),ν,ηn(k),ν+10Wn(k)µn(k)(θ) By 4.2.4 an(k) → a∞ with a∞ denoting the regression coefficients of µ(·, θ) onto the subspace Fη∞,b∞,ν,η∞,ν+1 within L2([0,1], w). Thus
n−1(k)
prFηn(k),ν ,ηn(k),ν+1
n(k),b ,wµn(k)(θ)
2
Rn,w
=n−1(k)
Fηn(k),bn(k),ν,ηn(k),ν+1an(k)
2 Rn,w
→
prFη∞,ν ,η∞,ν+1
∞,b ,wµ(·, θ)
2
L2([0,1],w)
.