Dependence of the involved terms on the sample size and cardinality of the param-

Hb⁻¹

n

g_ig^>_i −IE h

g_ig^>_i io

Hb⁻¹ ≤δ_v^2∗, where

g_i ^def=

∇_θ`<sub>i,1</sub>(θ<sup>∗</sup><sub>1</sub>)<sup>></sup>, . . . ,∇<sub>θ</sub>`_i,K(θ^∗_K)^>>

∈IR^psum, Hb^{2 def}= Xn

i=1IE n

g_ig^>_i o

, psum

def= p1+· · ·+pK.

(Eb) The i.i.d. bootstrap weights u_i are independent of Y , and for all i= 1, . . . , n it holds for some constants gk >0, νk ≥1

IEui = 1, Varui = 1,

logIEexp{λ(u_i−1)} ≤ν₀²λ²/2, |λ| ≤g.

5.3 Dependence of the involved terms on the sample size and cardinal-ity of the parameters’ set

Here we consider the case of the i.i.d. observations Y1, . . . , Yn and x=Clogn in order to specify the dependence of the non-asymptotic bounds on n and p. In the paper by Spokoiny and Zhilova(2014) (the version of 2015) this is done in detail for the i.i.d. case, generalized linear model and quantile regression.

Example 5.1 inSpokoiny(2012a) demonstrates that in this situation gk=C√ n and ωk=C/√

n. then Z_k(x) =C√

pk+x for some constant C≥1.85 , for the function Z_k(x) given in (B.3) in Section B.1. Similarly it can be checked that g_2,k(r) from condition

(ED₂) is proportional to √

n: due to independence of the observations logIEexp

λ ωk

γ^>₁D⁻¹_k ∇²_θζ_k(θ)D⁻¹_k γ₂

=Xn

i=1logIEexp λ

√n 1 ω_k√

nγ^>₁d⁻¹_k ∇²_θζi,k(θ)d⁻¹_k γ₂

≤nλ²

nC for|λ| ≤g_2,k(r)√ n,

where ζ_i,k(θ) ^def= `<sub>i,k</sub>(θ)−IE`_i,k(θ) , d²_k ^def= −∇²_θIE`_i,k(θ^∗_k) and D²_k = nd²_k in the i.i.d.

case. Function g_2,k(r) denotes the marginal analog of g2,k(r) .

Let us show, that for the value δ_k(r) from the condition (L0) it holds δ_k(r) = Cr/√

n. Suppose for all θ∈Θ_0,k(r) and γ ∈IR^pk :kγk= 1 kD_k⁻¹γ^>∇³_θIEL_k(θ)D⁻¹_k k ≤ C, then it holds for some θ∈Θ_0,k(r) :

kD_k⁻¹D²(θ)D⁻¹_k −Ipkk=kD⁻¹_k (θ^∗_k−θ)^>∇³_θIELk(θ)D_k⁻¹k

= kD⁻¹_k (θ^∗_k−θ)^>DkD⁻¹_k ∇³_θIELk(θ)D⁻¹_k k

≤ rkD_k⁻¹kkD⁻¹_k γ^>∇³_θIELk(θ)D_k⁻¹k ≤Cr/√ n.

Similarly Cm,k(r)≤Cr/√

n+C in condition (L0m).

The next remark helps to check the global identifiability condition (Lr) in many situations. Suppose that the parameter domain Θ_k is compact and n is sufficiently large, then the value bk(r) from condition (Lr) can be taken as C{1−r/√

n} ≈ C. Indeed, for θ:kD_k(θ−θ^∗_k)k=r

−2{IEL_k(θ)−IEL_k(θ^∗_k)} ≥ r² n

1−rkD⁻¹_k kkD⁻¹_k γ^>∇³_θIEL_k(θ)D_k⁻¹ko

≥ r²(1−Cr/√ n).

Due to the obtained orders, the conditions (B.1) and (B.9) of TheoremsB.1 and B.5 on concentration of the MLEs eθ_k,eθ_k^ab require r0,k ≥C√

p_k+x.

A Approximation of the joint distributions of `

₂

-norms

Let us previously introduce some notations:

1_K ^def= (1, . . . ,1)^>∈IR^K;

k · k is the Euclidean norm for a vector and spectral norm for a matrix;

k · k_max is the maximum of absolute values of elements of a vector or of a matrix;

k · k₁ is the sum of absolute values of elements of a vector or of a matrix.

Consider K random centered vectors φ_k ∈ IR^pk for k = 1, . . . , K. Each vector equals to a sum of n centered independent vectors:

φ_k=φ_k,1+· · ·+φ_k,n, IEφ_k=IEφ_k,i= 0 ∀1≤i≤n.

(A.1) Introduce similarly the vectors ψ_k∈IR^pk for k= 1, . . . , K:

ψ_k=ψ_k,1+· · ·+ψ_k,n, IEψ_k=IEψ_k,i= 0 ∀1≤i≤n,

(A.2)

with the same independence properties as φ_k,i, and also independent of all φ_k,i. The goal of this section is to compare the joint distributions of the `2-norms of the sets of vectors φ_k and ψ_k, k= 1, . . . , K (i.e. the probability laws L(kφ₁k, . . . ,kφ_Kk) and L(kψ₁k, . . . ,kψ_Kk) ), assuming that their correlation structures are close to each other.

Denote

pmax def

= max

1≤k≤Kp_k, psum

def= p1+· · ·+pK, λ²_φ,max ^def= max

1≤k≤KkVar(φ_j)k, λ²_ψ,max^def= max

1≤k≤KkVar(ψ_j)k, zmax

def= max

1≤k≤Kzk, zmin

def= min

1≤k≤Kzk, δ_z,max ^def= max

1≤k≤Kδ_zk, δ_z,min^def= min

1≤k≤Kδ_zk, let also

∆_ε ^def=

p³_max n

1/8

log^9/16(K) log^3/8(npsum)z_min^1/8 (A.3)

×max{λ_φ,max, λ_ψ,max}^3/4log^−1/8(5n^1/2).

The following conditions are necessary for the PropositionA.1

(C1) For some g_k, ν_k,c_φ,c_ψ >0and for all i= 1, . . . , n, k= 1, . . . , K sup

γ_k∈IRpk, kγ_kk=1

logIEexp n

λ√

nγ^>_kφ_k,i/cφ

o

≤ λ²ν_k²/2, |λ|<gk,

sup

γ_k∈IR^pk, kγ_kk=1

logIEexpn λ√

nγ^>_kψ_k,i/c_ψo

≤ λ²ν_k²/2, |λ|<g_k,

where c_φ≥Cλφ,max and c_ψ ≥Cλφ,max. (C2) For some δ²_Σ ≥0

1≤kmax1, k2≤K

Cov(φ_k1,φ_k2)−Cov(ψ_k1,ψ_k2)

max≤δ²_Σ. (A.4) Proposition A.1(Approximation of the joint distributions of `₂-norms). Consider the centered random vectors φ₁, . . . ,φ_K and ψ₁, . . . ,ψ_K given in (A.1), (A.2). Let the conditions(C1) and (C2) be fulfilled, and the values z_k≥√

p_k+∆ε and δz_k ≥0 be s.t.

Cmax{n^−1/2, δ_z,max} ≤∆_ε ≤Cz_max⁻¹ , then it holds with dominating probability IP

[K

k=1{kφ_kk> zk}

−IP

[K

k=1{kψ_kk> zk−δzk}

≥ −∆_`2, IP

[K

k=1{kφ_kk> zk}

−IP

[K

k=1{kψ_kk> zk+δzk}

≤ ∆`2

for the deterministic non-negative value

∆_`2≤12.5C p³_max

n 1/8

log^9/8(K) log^3/8(npsum) max{λ_φ,max, λ_ψ,max}^3/4

+ 3.2Cδ²_Σ p³_max

n 1/4

pmaxz^1/2_minlog²(K) log^3/4(npsum) max{λ_φ,max, λψ,max}^7/2

≤25C p³_max

n 1/8

log^9/8(K) log^3/8(npsum) max{λ_φ,max, λ_ψ,max}^3/4, where the last inequality holds for

δ²_Σ ≤ 4C n

p¹³_max 1/8

log^−7/8(K) log^−3/8(npsum) (max{λ_φ,max, λ_ψ,max})^−11/4.

Remark A.1. The approximating error term ∆`2 consists of three errors, which cor-respond to: the Gaussian approximation result (Lemma A.2), Gaussian comparison (Lemma A.7), and anti-concentration inequality (Lemma A.8). The bound on ∆<sub>`2</sub> above implies that the number K of the random vectors φ₁, . . . ,φ_K should satisfy logK (n/p³_max)^1/12 in order to keep the approximating error term ∆_`2 small. This condition can be relaxed by using a sharper Gaussian approximation result. For instance, using in LemmaA.2the Slepian-Stein technique plus induction argument from the recent paper by Chernozhukov et al.(2014b) instead of the Lindeberg’s approach, would lead to the improved bound: C

_p3 max

n

1/6

multiplied by a logarithmic term.

A.1 Joint Gaussian approximation of `₂-norm of sums of independent vectors by Lindeberg’s method

Introduce the following random vectors from IR^psum: Φ^def=

φ^>₁, . . . ,φ^>_K>

, Φ_i ^def=

φ^>_1,i, . . . ,φ^>_K,i>

, i= 1, . . . , n, Φ=Xn

i=1Φ_i, IEΦ=IEΦ_i = 0.

(A.5)

Define their Gaussian analogs as follows: Lemma A.2 (Joint GAR with equal covariance matrices). Consider the sets of ran-dom vectors φ_j and φ_j, j = 1, . . . , K defined in (A.1), and (A.5)– (A.8). If the conditions of Lemmas A.4 are A.5 are fulfilled, then it holds for all ∆, β > 0, z_j ≥ max

∆+√

pj,2.25 log(K)/β with dominating probability IP

Proof of Lemma A.2.

IP Let us approximate the max1≤j≤K function using the smooth maximum:

h_β({x_j})^def= β⁻¹log The indicator function 1I{x > 0} is approximated with the three times differentiable function g(x) growing monotonously from 0 to 1 :

g(x) ^def=

Therefore

where the last inequality holds for zj ≥2.25 log(K)/β. Denote z^def= (z₁, . . . , z_K)^>∈IR^K, z_j >0.

Then by (A.10) and (A.11) IP LemmaA.6checks that F_{∆, β}(·,z) admits applying the Lindeberg’s telescopic sum device (seeLindeberg(1922)) in order to approximate IEF_{∆, β}(Φ,z) with IEF_{∆, β} Φ,z

The difference F_{∆, β}(Φ,z)−F_{∆, β} Φ,z

can be represented as the telescopic sum:

F_{∆, β}(Φ,z)−F_{∆, β} Φ,z and the same bound holds for IEmax1≤j≤K

kS_j,i+φ_j,ik^{6 1/2}. Denote

probability ≥1−6 exp(−x) as follows

The derived bounds imply:

IP

The next lemma is formulated separately, since it is used for a proof of another result.

Lemma A.3(Smooth uniform GAR). Under the conditions of Lemma A.2it holds with dominating probability for the function F∆, β(·,z) given in (A.12):

1.1. IP

Proof of Lemma A.3. The first inequality 1.1 is obtained in (A.16), the second inequality 1.2 follows similarly from (A.14) and (A.15). The inequalities 2.1 and 2.2 are given in (A.13) and (A.14).

Lemma A.4. Let for some c_φ,g1, ν₀>0and for all i= 1, . . . , n, j= 1, . . . , psum

logIEexpn λ√

n|φ^j_i|/c_φo

≤λ²ν₀²/2, |λ|<g1,

here φ^j_i denotes the j-th coordinate of vector φ_i. Then it holds for all i= 1, . . . , n and m, t >0

IP

1≤j≤pmaxsum

|φ^j_i|^m > t

≤ exp (

−nt^2/m

2c²_φν₀² + log(psum) )

.

Proof of Lemma A.4. Let us bound the maxj|φ^j_i| using the following bound for the maximum:

1≤j≤pmax_sum|φ^j_i| ≤lognXpsum

j=1 exp |φ^j_i|o . By the Lemma’s condition

IEexp

1≤j≤pmax λ√

n c_φ |φ^j_i|

≤ exp λ²ν₀²/2 + logpsum

. Thus, the statement follows from the exponential Chebyshev’s inequality.

Lemma A.5. If for the centered random vectors φ_j ∈IR^pj j= 1, . . . , K sup

γ∈IR^pj, kγk6=0

logIEexp (

λ γ^>φ_j kVar^1/2(φ_j)γk

)

≤ ν₀²λ²/2, |λ| ≤g

for some constants ν₀ >0 and g≥ν₀⁻¹max1≤j≤K

p2p_jlog(K), then IE max

1≤j≤K

kφ_jk ≤ Cν0 max

1≤j≤KkVar^1/2(φ_j)kp

2pmaxlog(K),

IE max

1≤j≤K

kφ_jk⁶ 1/2

≤ Cν₀ max

1≤j≤KkVar^1/2(φ_j)k³p

2p_maxlog(K)(p_max+ 6x), The second bound holds with probability ≥1−2e^−x.

Proof of Lemma A.5. Let us take for each j= 1, . . . , K finite ε_j-grids G_j(ε)⊂IR^pj on the (pj−1) -spheres of radius 1 s.t

∀γ∈IR^pj s.t. kγk= 1 ∃γ₀∈Gj(ε) : kγ−γ₀k ≤ε, kγ₀k= 1.

Then Hence, by inequality (A.9) and the imposed condition it holds for all

0< µ <g/max1≤j≤KkVar^1/2(φ_j)k:

For the second part of the statement we combine the first part with the result of Theorem B.3 on deviation of a random quadratic form: it holds with dominating probability for

V_φ²

Proof of Lemma A.6. Denote

s(Γ)^def= XK

j=1exp βkγ_jk²−z_j² 2zj

!

, hβ(s(Γ))^def= β⁻¹log{s(Γ)}, (A.17)

then F_β,∆(Γ,z) =g ∆⁻¹h_β(s(Γ))

. Let γ^q denote the q-th coordinate of the vector Γ ∈IR^psum. It holds for q, l, b, r= 1, . . . , psum:

d

dγ^qF_β,∆(Γ,z) = 1

∆g⁰

∆⁻¹h_β(s(Γ)) d

dγ^qh_β(s(Γ)), d²

dγ^qdγ^lF_β,∆(Γ,z) = 1

∆²g⁰⁰

∆⁻¹h_β(s(Γ)) d

dγ^qh_β(s(Γ)) d

dγ^lh_β(s(Γ)) + 1

∆g⁰

∆⁻¹h_β(s(Γ)) d²

dγ^qdγ^lh_β(s(Γ)), d³

dγ^qdγ^ldγ^bF_β,∆(Γ,z) = 1

∆³g⁰⁰⁰

∆⁻¹h_β(s(Γ)) d

dγ^qh_β(s(Γ)) d

dγ^lh_β(s(Γ)) d

dγ^bh_β(s(Γ)) + 1

∆²g⁰⁰

∆⁻¹h_β(s(Γ))

( d²

dγ^qdγ^bh_β(s(Γ)) d

dγ^lh_β(s(Γ)) + d

dγ^qh_β(s(Γ)) d²

dγ^ldγ^bh_β(s(Γ)) + d

dγ^bh_β(s(Γ)) d²

dγ^qdγ^lh_β(s(Γ)) )

+ 1

∆g⁰

∆⁻¹h_β(s(Γ)) d³

dγ^qdγ^ldγ^bh_β(s(Γ)).

Let for 1≤q ≤psum j(q) denote an index from 1 to K s.t. the coordinate γ^q of the vector Γ = γ^>₁, . . . ,γ^>_K>

belongs to its sub-vector γ_j(q).

d

dγ^qhβ(s(Γ)) = 1 β

1 s(Γ)

d

dγ^qs(Γ) = 1 s(Γ)

γ^q

z_j(q)exp β

kγ_j(q)k²−z_j(q)² 2z_j(q)

! ,

d²

The following Lemma shows how to compare the expected values of a twice differentiable function evaluated at the independent centered Gaussian vectors. This statement is used

Im Dokument Simultaneous likelihood-based bootstrap confidence sets for a large number of models (Seite 25-36)