Supplements to the Asymptotic Theory of
2.1 Mean Square Error Solution
2.1.5 Continuity Properties of Lagrange Multipliers
The Lagrange multipliers and hence the asymptotic MSE, the trace of the asymp-totic variance and the standardized asympasymp-totic bias of the MSE solution ˜ηr = ˜η are continuous in r∈(0,∞) .
Proposition 2.1.9 Let η˜r be the MSE solution for D=Ik and radius r∈(0,∞) derived in Theorem1.3.11and (rn)n∈N⊂(0,∞) with rn→r as n→ ∞. Then,
n→∞lim trArn= trAr lim
n→∞brn=br lim
n→∞crn=cr (2.1.85) In case Ar and ar are unique, then also
n→∞lim Arn=Ar and lim
n→∞arn=ar (2.1.86) Proof To simplify the notation, we dropr as an index. Let ˜η be the MSE solution for D = Ik and radius r ∈ (0,∞) provided by Theorem 1.3.11 and (rn)n∈N ⊂ (0,∞) with rn →r as n→ ∞. By the construction given at the beginning of the proof of Proposition2.1.7, we obtain
maxMSE(˜ηn, rn) = E|˜ηn|2+r2nω∗(˜ηn)2 (2.1.87) is uniformly bounded in n∈N. That is, by Proposition 2.1.1, trAn is uniformly bounded in n∈N. In case (∗=v) we have An ∈R which immediately implies An is uniformly bounded. In case (∗=c) we get by the boundedness of trAn and the positive definiteness and symmetry of An that the operator norm of An is,
||An||op= sup
|x|≤1
|Anx|= sup
|y|≤1
|GnAnGτny|= max
j=1,...,kλn,j ≤trAn<∞ (2.1.88) where λn,j are the eigenvalues of An and Gn are orthogonal matrices such that GnAnGτn = diag (λn,1, . . . , λn,k) (spectral decomposition). Thus, An is bounded uniformly in n. In addition, the uniform boundedness of maxMSE entails bn and cn∈(−bn,0) are bounded uniformly in n. Finally, an is uniformly bounded in n by bound (2.1.45). Consequentially, there is a subsequence (m) ⊂(n) such that Am→A0, bm→b0, cm→c0 and am→a0. We now define
χ=χ(Λ) = (A0Λ−a0) min
1, b0
|A0Λ−a0|
(∗=c) (2.1.89) respectively
χ=χ(Λ) =c0∨A0Λ∧(c0+b0) (∗=v) (2.1.90) Since ˜ηm → χ and |˜ηm| is uniformly bounded in m, we obtain by dominated convergence
0 = lim
m→∞E ˜ηm= Eχ (2.1.91)
Furthermore,
|AmΛ−am| −bm
+≤ |AmΛ−am| ≤ ||Am||op|Λ|+|am| (2.1.92) where
E
||Am||op|Λ|+|am|
≤ ||Am||op
E|Λ|21/2 +|am|
=||Am||op
√
trI +|am| (2.1.93)
30 Supplements to the Asymptotic Theory of Robustness
is uniformly bounded in m, respectively
cm−AmΛ)+≤Am|Λ|+cm (2.1.94) where
E
Am|Λ|+bm
≤Am
Em|Λ|21/2
+cm=Am
√
trI +cm (2.1.95) is uniformly bounded in m. Hence, an application of dominated convergence yields
0 = lim
m→∞E |AmΛ−am| −bm
+−r2mbm= E |A0Λ−a0| −b0
+−r2b0 (2.1.96) respectively
0 = lim
m→∞E cm−AmΛ)+−r2mbm= E c0−A0Λ)+−r2b0 (2.1.97) Moreover,
|˜ηmΛτ| ≤ |(AmΛ−am)Λτ| ≤ kAmkop|Λ|2+|am| |Λ| (2.1.98) where
E
kAmkop|Λ|2+|am| |Λ|
=kAmkoptrI+|am|√
trI (2.1.99) is uniformly bounded in m. That is, we can once again apply dominated conver-gence and get
Ik = lim
m→∞E ˜ηmΛτ = EχΛτ (2.1.100) Thus, χ is the unique solution to the MSE problem (1.3.5) for radius r; i.e., χ=η in L2(P) . In particular, we obtain b0 =b, c0 =c and trA0 = trA by the uniqueness of b, c and trA. Hence, b, c and trA are the unique accumulation points of the sequencesbn, cn and trAn and the sequences converge. If, in addition A and a are unique, then also A0=A and a0=a where the accumulation points
are unique; i.e., (2.1.86) holds. ////
Remark 2.1.10 (a) The proof of the previous proposition is similar to the proof of the subsequent Theorem 2.1.11, but easier. We can argue with dominated con-vergence, since L(Λ) is independent of n, whereas in Theorem 2.1.11we have to invoke uniform integrability.
(b)As a direct consequence of Proposition2.1.9, we obtain
E|˜ηrn|2= trArn−r2nb2rn−→trAr−r2b2r= E|η˜r|2 (2.1.101) as n→ ∞; i.e., the trace of the asymptotic covariance is continuous in r∈(0,∞) . (c)Rieder(1994) proves the uniqueness and continuity of ar on (0,∞) in case k= 1 and med(Λθ) unique; confer Lemma C.2.4 (ibid.).
(d)A generalization of Proposition2.1.9to arbitrary D∈Rp×k with rkD=p
is given inRuckdeschel(2005b). ////
Under some additional assumptions the Lagrange multipliers and hence the asymp-totic MSE, the trace of the asympasymp-totic variance and the standardized asympasymp-totic bias of the MSE solution ˜ηθ= ˜η are continuous in the parameter θ∈Θ . Moreover, the minimum bias ωmin∗ (∗=c, v) is continuous in θ∈Θ .
2.1 Mean Square Error Solution 31
Theorem 2.1.11 Let D=Ik and Aθ, aθ, bθ and cθ be the Lagrange multipliers contained in the solution η˜θ to the MSE problem (1.3.5) for radiusr∈(0,∞)given in Theorem1.3.11. Further assume
trIθn−→trIθ as n→ ∞ (2.1.102) and
LPθn(Λθn)−→w LPθ(Λθ) as n→ ∞ (2.1.103) where (θn)n∈N⊂Θ is some sequence such that θn→θ as n→ ∞.
(a)Then,
n→∞lim trAθn= trAθ lim
n→∞bθn=bθ lim
n→∞cθn =cθ (2.1.104) In case Aθ and aθ are unique, then also
n→∞lim Aθn =Aθ and lim
n→∞aθn=aθ (2.1.105) (b)It holds for the minimum bias ωmin∗,θ given in Theorems1.3.7(b) and1.3.9(b), respectively
n→∞lim ω∗,θminn =ωmin∗,θ (∗=c, v) (2.1.106) Proof To simplify the notation, we omit θ as an index and identify θn by n.
(a)Let r∈(0,∞) be fixed. We have E|Λn,iΛn,j| ≤
En|Λn,i|21/2
En|Λn,j|21/2
≤En|Λn|2= trIn→trI <∞ (2.1.107) for all i, j= 1, . . . , k. Moreover, if we fix some ε >0 , there exists some δ(ε)>0 such that for any An∈Bk: Pn(An)< δ(ε) implies
R
An
|Λn,iΛn,j|dPn≤ R
An
|Λn,i|2dPn1/2 R
An
|Λn,j|2dPn1/2
≤ R
An
|Λn|2dPn≤ε (2.1.108) for all i, j = 1, . . . , k. That is, (ΛnΛτn) is uniformly integrable by Theorem 4.5.3 of Chung (2000). Moreover, Ln(ΛnΛτn) −→w L(ΛΛτ) by the continuous mapping theorem and we therefore can apply Vitali’s theorem (cf. Corollary A.2.3 ofRieder (1994)) which yields In → I, where I 0 . Thus, by the continuity of the determinant, there is some N ∈ N such that for all n ≥ N, In 0 . In the sequel, we argue similarly to the proof of Proposition 2.1.7; i.e., we want to apply the construction given on p 197 of Rieder (1994). Therefore, we first verify that there is some M ∈(0,∞) such that
EnΛnΛτnI |In−1Λn| ≤M
(2.1.109) is regular for sufficiently large n ∈ N. Since Ln(Λn) −→w L(Λ) , we get by the continuous mapping theorem for all M ∈ (0,∞) satisfying P(|I−1Λ| = M) = 0 that
EnΛnΛτnI |In−1Λn| ≤M
→E ΛΛτI |I−1Λ| ≤M
(2.1.110)
32 Supplements to the Asymptotic Theory of Robustness
Therefore, by In→ I we also have EnΛnΛτnI |In−1Λn|> M
→E ΛΛτI |I−1Λ|> M
(2.1.111) We can now choose M ∈ (0,∞) subject to P(|I−1Λ| = M) = 0 so large that the right hand side of (2.1.111) becomes arbitrarily small (e.g., in operator norm).
Thus, by the continuity of the determinant there is some (sufficiently large) M such that
EnΛnΛτnI |In−1Λn| ≤M
0 (2.1.112)
for all n≥N1 (≥N). That is, for n≥N1 we can define the following ICs, χn=
EnΛnΛτnJn−1
ΛnJn−EnΛnJn] Jn := I |In−1Λn| ≤M
(2.1.113) which are bounded uniformly in n. Hence, the maximum asymptotic MSE
maxMSE(χn, r) = En|χn|2+r2supPn|χn|2 (2.1.114) is bounded uniformly in n. Thus, for n≥N1 the corresponding optimally robust ICs ˜ηn must have a uniformly bounded maximum asymptotic MSE; i.e., by Propo-sition2.1.1, trAn is bounded for n≥N1. In case (∗=v) we have An ∈R which immediately implies An is bounded. In case (∗=c) we get by the boundedness of trAn and the positive definiteness and symmetry of An that the operator norm of An is,
||An||op= sup
|x|≤1
|Anx|= sup
|y|≤1
|GnAnGτny|= max
j=1,...,kλn,j ≤trAn<∞ (2.1.115) where λn,j are the eigenvalues of An and Gn are orthogonal matrices such that GnAnGτn = diag (λn,1, . . . , λn,k) (spectral decomposition). Thus, An is bounded uniformly in n. By the uniform boundedness of maxMSE , this immediately implies that bn and cn ∈(−bn,0) are bounded uniformly in n. Finally, an is uniformly bounded in nby bound (2.1.45). Consequentially, there is a subsequence (m)⊂(n) such that Am→A0, bm→b0, cm→c0 and am→a0. We now define
χ=χ(Λ) = (A0Λ−a0) min
1, b0
|A0Λ−a0|
(∗=c) (2.1.116) respectively
χ=χ(Λ) =c0∨A0Λ∧(c0+b0) (∗=v) (2.1.117) By assumption (2.1.103) and as ˜ηm(um) → χ(u) for um → u, Theorem 5.5 of Billingsley(1968) yields Lm(˜ηm)−→w L(χ) and therefore we obtain by the uniform boundedness of ˜ηm and χ
0 = lim
m→∞Emη˜m= Eχ (2.1.118)
Moreover,
Em |AmΛm−am| −bm
+ ≤Em|AmΛm−am| ≤ ||Am||opEm|Λm|+|am|
≤ ||Am||op
Em|Λm|21/2
+|am|
=||Am||op
ptrIm +|am|
→ ||A0||op
√
trI +|a0|<∞ (2.1.119)
2.1 Mean Square Error Solution 33
respectively
Em cm−AmΛm)+≤AmEm|Λm|+cm≤Am
Em|Λm|21/2 +cm
=Amp
trIm +cm→A0√
trI +c0 <∞ (2.1.120) i.e., (|AmΛm−am|−bm
+
, respectively (cm−AmΛm)+
is uniformly integrable.
In addition, also (˜ηmΛτm) is uniformly integrable by Theorem 4.5.3 ofChung(2000) which may be shown analogously to the uniform integrability of (ΛnΛτn) at the beginning of this proof. Furthermore, Lm(˜ηmΛτm)−→w L(χΛτ) and
Lm (|AmΛm−am| −bm)+
−→w L (|A0Λ−a0| −b0)+
(2.1.121) respectively
Lm (cm−AmΛm)+
−→w L (c0−A0Λ)+
(2.1.122) by a combination of the Cram´er-Wold device, Slutzky’s lemma and the continuous mapping theorem. That is, we may apply Vitali’s theorem (cf. Corollary A.2.3 of Rieder(1994)) and obtain
Ik= lim
m→∞Emη˜mΛτm= EχΛτ (2.1.123) 0 = lim
m→∞Em |AmΛm−am| −bm
+−r2bm= E |A0Λ−a0| −b0
+−r2b0(2.1.124) respectively
0 = lim
m→∞Em cm−AmΛm)+−r2bm= E c0−A0Λ)+−r2b0 (2.1.125) Thus, χ is the unique solution to the MSE problem (1.3.5); i.e., χ=η in L2(P) . In particular, we obtain b0 = b, c0 = c and trA0 = trA by the uniqueness of b, c and trA. Hence, b, c and trA are the unique accumulation points of the sequences bn, cn and trAn and the sequences converge. If, in addition A and a are unique, then also A0 =A and a0 = a where the accumulation points are unique; i.e., (2.1.105) holds.
(b) ∗ = c: Without restriction we can rewrite the minimum bias given in Theorem1.3.7(b) as
ωcmin= max
trA E|AΛ−a|
a∈Rk, A∈Rk×k\ {0}, kAkop= 1
(2.1.126) Let a∈Rk and A∈Rk×k with kAkop= 1 be fixed such that
trA
E|AΛ−a| =ωminc (2.1.127)
Then,
ωc,nmin≥ trA
En|AΛn−a| (2.1.128)
34 Supplements to the Asymptotic Theory of Robustness
Since
lim sup
n→∞
En|AΛn−a| ≤ kAkoplim sup
n→∞
En|Λn|+|a| (2.1.129)
≤lim sup
n→∞
ptrIn +|a|=√
trI +|a|<∞(2.1.130) and Ln(Λn)−→w L(Λ) by assumption (2.1.103), we can apply Vitali’s theorem and obtain
n→∞lim En|AΛn−a|= E|AΛ−a| (2.1.131) Consequentially,
lim inf
n→∞ ωc,nmin≥ lim
n→∞
trA
En|AΛn−a| = trA
E|AΛ−a| =ωminc (2.1.132) Now, let an∈Rk and An ∈Rk×k with kAnkop= 1 such that
trAn
En|AnΛn−an| =ωc,nmin (2.1.133) for n∈N. We have,
En|AnΛn| ≤ kAnkopE|Λn| ≤p
trIn <∞ (2.1.134) Moreover,
En|AnΛn−an| ≤En|AnΛn| (2.1.135) as an is a solution to
trAn
En|AnΛn−an| = max! (2.1.136) Thus, using the triangular inequality,
|an| ≤ |AnΛn−an|+|AnΛn| (2.1.137) and taking expectations, we obtain
|an| ≤En|AnΛn−an|+ En|AnΛn| ≤2 En|AnΛn| ≤2p
trIn <∞ (2.1.138) That is, an and An are bounded uniformly in n. Hence, there is a subsequence (m)⊂(n) such that am→a0∈Rk and Am→A0 ∈Rk×k. Since
lim sup
m→∞
Em|AmΛn−am| ≤ kAmkoplim sup
m→∞
Em|Λm|+|am|
(2.1.139)
≤lim sup
m→∞
ptrIm +|am|
(2.1.140)
=
√
trI +|a0|<∞ (2.1.141) we obtain by Vitali’s theorem
m→∞lim
trAm
Em|AmΛm−am| = trA0
E|A0Λ−a0| ≤ωminc (2.1.142)