Optimal Confidence Bands for Shape-Restricted Curves

https://doi.org/10.7892/boris.70416 | downloaded: 1.2.2022

Optimal Confidence Bands for Shape-Restricted Curves

Lutz D¨umbgen

Department of Mathematical Statistics and Actuarial Science University of Bern

Sidlerstrasse 5, CH-3012 Bern, Switzerland

E-mail: duembgen@stat.unibe.ch, URL: www.imsv.unibe.ch August 2002, updated December 2013

This paper has been published inBernoulli9, 423–449 (2003).

The present version corrects two typos in the published version and contains updated references.

Abstract

LetY be a stochastic process on[0,1]satisfyingdY(t) =n^1/2f(t)dt+dW(t), wheren≥1 is a given scale parameter (“sample size”),W is standard Brownian motion andf is an unknown function. Utilizing suitable multiscale tests we construct confidence bands forf with guaranteed given coverage probability, assuming that f is isotonic or convex. These confidence bands are computationally feasible and shown to be asymptotically sharp optimal in an appropriate sense.

Running title.Confidence Bands for Shape-Restricted Curves

Keywords and phrases.adaptivity, concave, convex, isotonic, kernel estimator, local smoothness, minimax bounds, multiscale testing

1 Introduction

Nonparametric statistical models often involve some unknown functionfdefined on a real interval J. For instancef might be the probability density of some distribution or a regression function.

Nonparametric point estimators for such a curvef are abundant. The available methods are based on kernels, splines, local polynomials, or orthogonal series, including wavelets; see Hart (1997) and references cited therein. In order to quantify the precision of estimation, one often wants to replace a point estimator with a confidence band(ˆ`,u)ˆ forf. The latter consists of two functions

`ˆ= ˆ`(·,data)anduˆ = ˆu(·,data)onJ with values in[−∞,∞]such that, hopefully,`ˆ≤ f ≤uˆ pointwise. More precisely, one is aiming at a confidence band such that

(1) IP{`ˆ≤f ≤u} ≥ˆ 1−α

for a given levelα∈]0,1[, while`ˆanduˆshould be as close to each other as possible.

Unfortunately, curve estimation is an ill-posed problem, and usually there are no nontrivial bands(ˆ`,u)ˆ satisfying (1) for arbitraryf; see Donoho (1988). Therefore one has to impose some additional restrictions on f. One possibility are smoothness constraints on f, for instance an upper bound on a certain derivative off. Under such restrictions, (1) can be achieved approxi- mately for large sample sizes; see for example Bickel and Rosenblatt (1973), Knafl et al. (1985), Hall and Titterington (1988), H¨ardle and Marron (1991), Eubank and Speckman (1993), Fan and Zhang (2000), and the references cited therein.

A problem with the aforementioned methods is that smoothness constraints are hard to justify in practical situations. More precisely, even if the underlying curve f is infinitely often differ- entiable, the actual coverage probabilities of the confidence bands mentioned above depend on quantitative properties of certain derivatives off which are difficult to obtain from the data.

In many applications qualitative assumptions about f such as monotonicity, unimodality or concavity/convexity are plausible. One example are growth curves in medicine, e.g. wheref(x)is the mean body height of newborns at agex. Here isotonicity off is a plausible assumption. An- other example are so-called Engel curves in econometrics, wheref(x)is the mean expenditure for certain consumer goods of households with annual incomex. Here one expectsf to be isotonic and sometimes concave as well. Under such qualitative assumptions it is possible to construct (1−α)–confidence sets for f based on certain goodness-of-fit tests without relying on asymp- totic arguments. Examples for such procedures can be found in Davies (1995), Hengartner and Stark (1995) and D¨umbgen (1998). In particular, these papers present confidence bands(ˆ`,u)ˆ for

f such that

(2) IP{`ˆ≤f ≤u} ≥ˆ 1−α wheneverf ∈ F.

HereF denotes the specified class of functions. Given a suitable distance measure D(·,·) for functions, the goal is to find a band(ˆ`,u)ˆ satisfying (2) such that eitherD(ˆu,`)ˆ orD(ˆ`, f) and D(ˆu, f) are as small as possible. The phrase “as small as possible” can be interpreted in the sense of optimal rates of convergence to zero as the sample sizentends to infinity. The papers of Hengartner and Stark (1995) and D¨umbgen (1998) contain such optimality results.

In the present paper we investigate optimality of confidence bands in more detail. In addi- tion to optimal rates of convergence we obtain optimal constants and discuss the impact of local smoothness properties off. Compared to the general confidence sets of D¨umbgen (1998), the methods developed here are more stringent and computationally simpler. They are based on mul- tiscale tests as developed by D¨umbgen and Spokoiny (2001), who considered tests of qualitative assumptions rather than confidence bands. For further results on testing in nonparametric curve estimation see Hart (1997), Fan et al. (2001), and the references cited there.

2 Basic setting and overview

For mathematical convenience we focus on a continuous white noise model: Suppose that one observes a stochastic processY on the unit interval[0,1], where

Y(t) = n^1/2 Z t

0

f(x)dx+W(t).

Heref is an unknown function inL²[0,1],n≥1is a given scale parameter (“sample size”), and W is standard Brownian motion. In this context the bounding functions`,ˆuˆare defined on[0,1], but for notational convenience the functionf is tacitly assumed to be defined on the whole real line with values in[−∞,∞]. From now on we assume that

f ∈ G ∩L²[0,1], whereGdenotes one of the following two function classes:

G_↑ :=

n

non-decreasing functionsg:R→[−∞,∞]o , G_conv := n

convex functionsg:R→]−∞,∞]o .

The paper is organized as follows. In Section 3 we treat the caseG = G_↑ and measure the quality of a confidence band(ˆ`,u)ˆ by quantities related to the Levy distancedL(ˆ`,u). Generally,ˆ

dL(g, h) := inf n

>0 :g≤h(·+) +andh≤g(·+) +on[0,1−] o

for isotonic functionsg, h: [0,1]→[−∞,∞]. It turns out that a confidence band which is based on a suitable multiscale test as introduced by D¨umbgen and Spokoiny (2001) is asymptotically optimal in a strong sense. Throughout this paper asymptotic statements refer ton → ∞, unless stated otherwise.

In Section 4 we treat both classesG_↑andG_conv simultaneously. We discuss the construction of confidence bands(ˆ`,u)ˆ satisfying (2) such thatD(ˆ`, f) andD(f,u)ˆ are as small as possible wheneverf satisfies some additional smoothness constraints. HereD(g, h)is a distance measure of the form

D(g, h) := sup

x∈[0,1]

w(x, f)(h(x)−g(x))

for some weight functionw(·, f) ≥0reflecting local smoothness properties off. Again it turns out that suitable multiscale procedures yield nearly optimal procedures without additional prior information onf.

In Section 5 we present some numerical examples for the procedures of Section 4. The proofs are deferred to Sections 6, 7 and 8. In particular, Section 7 contains a new minimax bound for confidence rectangles in a gaussian shift model, which may be of independent interest.

As for the white noise model, the results of Brown and Low (1996), Nussbaum (1996) and Grama and Nussbaum (1998) on asymptotic equivalence can be used to transfer the lower bounds of the present paper to other models. Moreover, one can mimick the confidence bands developed here in traditional regression models under minimal assumptions; see D¨umbgen and Johns (2004) and D¨umbgen (2007).

3 Optimality for isotonic functions in terms of L´evy type distances

In this section we consider the classG_↑. For isotonic functionsg, h: [0,1]→[−∞,∞]and >0 let

D(g, h) := inf n

λ≥0 :g≤h(·+) +λandh≤g(·+) +λon[0,1−] o

.

Then the L´evy distancedL(g, h)is the infimum of all >0such thatD(g, h)≤. We use these functionalsD(·,·)in order to quantify differences between isotonic functions. Figure 1 depicts one such functiong, and the shaded areas represent the set of all functionshwithD0.05(g, h) ≤ 0.1andD_0.05(g, h)≤0.025, respectively.

The next theorem provides lower bounds forD(ˆ`,u),ˆ 0 < ≤ 1. Here and throughout the sequel the dependence of probabilities, expectations and distributions on the functional parameter

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 -0.4

-0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

-0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4

Figure 1: TwoD_0.05(·,·)–neighborhoods of some functiong.

f is sometimes indicated by a subscriptf.

Theorem 3.1. There exists a universal functionbon]0,1]withlim_↓0b() = 0such that

f∈G_↑inf∩L²[0,1] IP_f (

`ˆ≤f ≤uˆandD(ˆ`,u)ˆ < (8 log(e/))^1/2−b() (n)^1/2

)

≤ b()

for any confidence band(ˆ`,u)ˆ and arbitrary∈]0,1].

Theorem 3.1 entails a lower bound fordL(ˆ`,u). For letˆ =n:= c(log(n)/n)^1/3−δn^−1/3 with any fixedc, δ >0. Then one can show that for sufficiently largen,

(8 log(e/))^1/2−b()

(n)^1/2 =

8 3c

1/2logn n

1/3

+o(n^−1/3) ≥ , provided thatcequals(8/3)^1/3 ≈1.387.

Corollary 3.2. For eachn≥1there exists a universal constantβ_nsuch thatβ_n→0and inf

f∈G_↑∩L²[0,1] IP_f

`ˆ≤f ≤uˆandd<sub>L</sub>(ˆ`,u)ˆ <8 3

1/3logn n

1/3

−β_nn^−1/3

≤ β_n

for any confidence band(ˆ`,u).ˆ

It is possible to get close to these lower bounds forD(ˆ`,u)ˆ simultaneouslyfor all∈ ]0,1]

while (2) is satisfied. For letκαbe a real number such that IP

|W(t)−W(s)|

(t−s)^1/2 ≤Γ(t−s) +καfor0≤s < t≤1

≤ α, where

Γ(u) := (2 log(e/u))^1/2 for0< u≤1.

The existence of such a critical value κ_α follows from D¨umbgen and Spokoiny (2001, Theo- rem 2.1). With the local averages

F_f(s, t) := 1 t−s

Z t s

f(x)dx off and their natural estimators

Fˆ(s, t) := Y(t)−Y(s) n^1/2(t−s) it follows that

IP_f

Fˆ(s, t)−F_f(s, t)

≤ Γ(t−s) +κ_α

(n(t−s))^1/2 for0≤s < t≤1

≥ 1−α.

But for0≤s < t≤1,

f(s) ≤ Ff(s, t) ≤ f(t) wheneverf ∈ G_↑. This implies the first assertion of the following theorem.

Theorem 3.3. With the critical valueκαabove let

`(x)ˆ := sup

0≤s<t≤x

Fˆ(s, t)−Γ(t−s) +κ_α pn(t−s)

, ˆ

u(x) := inf

x≤s<t≤1

Fˆ(s, t) +Γ(t−s) +κα

pn(t−s)

.

This defines a confidence band(ˆ`,u)ˆ forf satisfying (2) withF = G<sub>↑</sub> ∩L<sup>2</sup>[0,1]. Moreover, in case of`ˆ≤u,ˆ

D(ˆ`,u)ˆ ≤ (8 log(e/))^1/2+ 2κ_α

(n)^1/2 for0< ≤1, d_L(ˆ`,u)ˆ ≤ 8

3

1/3logn n

1/3

+o(n^−1/3).

Proof.The preceding upper bound forD(ˆ`,u)ˆ follows from the fact that for anyx∈[0,1−], ˆ

u(x)−`(xˆ +) ≤

Fˆ(x, x+) +Γ() +κα

(n)^1/2

−

Fˆ(x, x+)−Γ() +κα

(n)^1/2

= 2Γ() + 2κ_α (n)^1/2

= (8 log(e/))^1/2+ 2κ_α (n)^1/2 .

Letting=_n= (8/3)^1/3(log(n)/n)^1/3yields the upper bound ford_L(ˆ`,u).ˆ

4 Bands for potentially smooth functions

A possible criticism of the preceding results is the fact that the minimax bounds are attained at special step functions. On the other hand one often expects the underlying curve f to be smooth in some vague sense. Therefore we aim now at confidence bands satisfying (2) with F =G ∩L²[0,1], which are as small as possible wheneverfsatisfies some additional smoothness conditions. ThroughoutGstands forG_↑orG_conv.

In the sequel let hg, hi := R∞

−∞g(x)h(x)dxandkgk := hg, gi^1/2 for measurable functions g, hon the real line such that these integrals are defined. The confidence bands to be presented here can be described either in terms of kernel estimators forf or in terms of tests. Both viewpoints have their own merits.

4.1 Kernel estimators forf

Letψ be some kernel function inL²(R). For technical reasons we assume that ψ satisfies the following three regularity conditions:

(3)







ψhas bounded total variation;

ψis supported by[−a, b], wherea, b≥0;

h1, ψi > 0.

For any bandwidthh >0and location parametert∈Rlet ψ_h,t(x) := ψx−t

h

.

Thenhg, ψ_h,ti=hhg(t+h·), ψiandkψ_h,tk=h^1/2kψk. A kernel estimator forf(t)with kernel functionψand bandwidthhis given by

fˆ_h(t) := ψY(h, t) n^1/2hh1, ψi, where

ψY(h, t) :=

Z 1 0

ψh,t(x)dY(x).

From now on suppose thatah≤t≤1−bh. Thenψ_h,tis supported by[0,1]and one may write IE ˆf_h(t) = hf, ψ_t,hi

hh1, ψi = hf(t+h·), ψi h1, ψi , Var( ˆfh(t)) = kψ_t,hk²i

nh²h1, ψi² = kψk² nhh1, ψi².

The random fluctuations of these kernel estimators can be bounded uniformly inh > 0. For that purpose we define the multiscale statistic

T(±ψ) := sup

h>0

sup

t∈[ah,1−bh]

±ψW(h, t)

h^1/2kψk −Γ((a+b)h)

= sup

h>0

sup

t∈[ah,1−bh]

±fˆ_h(t)−IE ˆf_h(t)

Var( ˆf_h(t))^1/2 −Γ((a+b)h) ,

similarly as in D¨umbgen and Spokoiny (2001). It follows from Theorem 2.1 in the latter paper, that 0 ≤T(±ψ) <∞almost surely. In particular,|fˆ_h(t)−IE ˆf_h(t)| ≤ (nh)^−1/2log(e/h)^1/2Op(1), uniformly inh >0andah≤t≤1−bh.

It is well-known that kernel estimators are biased in general. But our shape restrictions may be used to construct two kernel estimators whose bias is always non-positive or non-negative, respectively. Precisely, letψ^{(`)</sup> andψ<sup>(u)</sup> be two kernel functions satisfying (3) with respective supports[−a<sup>(`)}, b^(`)]and[−a^(u), b^(u)]. In addition suppose that

hg, ψ^{(`)</sup>i ≤ g(0)h1, ψ<sup>(`)}i for allg∈ G ∩L²[−a^{(`)</sup>, b<sup>(`)}], (4)

hg, ψ^(u)i ≥ g(0)h1, ψ^(u)i for allg∈ G ∩L²[−a^(u), b^(u)].

(5)

These inequalities imply that the corresponding kernel estimators satisfy the inequalitiesIE ˆf_h^(`)(t)≤ f(t)≤IE ˆf_h^(u)(t), and the definition ofT(±ψ)yields that

f(t) ≥ fˆ_h^(`)(t)−

kψ^(`)k

Γ(d^{(`)</sup>h) +T(ψ<sup>(`)}) h1, ψ^(`)i(nh)^1/2 , (6)

f(t) ≤ fˆ_h^(u)(t) +

kψ^(u)k

Γ(d^(u)h) +T(−ψ^(u)) h1, ψ^(u)i(nh)^1/2 . (7)

Hered^(z) := a^(z) +b^(z). Now letκα be the(1−α)–quantile of the combined statistic T^∗ :=

max

T(ψ^(`)), T(−ψ^(u))

, i.e. the smallest real number such thatIP{T^∗ ≤κ_α} ≥1−α. Then

`(t)ˆ := sup

h>0 :t∈[^{a(`)h,1−b(`)h}]

fˆ_h^{(`)</sup>(t)−kψ<sup>(`)}k(Γ(d^{(`)</sup>h) +κα) h1, ψ<sup>(`)}i(nh)^1/2

! ,

ˆ

u(t) := inf

h>0 :t∈[^{a(u)h,1−b(u)h}]

fˆ_h^(u)(t) +kψ^(u)k(Γ(d^(u)h) +κ_α) h1, ψ^(u)i(nh)^1/2

!

defines a confidence band(ˆ`,u)ˆ forf satisfying (2).

Equality holds in (2) ifG = G_↑ andf is constant, or ifG = G_conv andf is linear, provided that κα > 0. For then it follows from (4) and (5) with g(x) = ±1 or g(x) = ±x that the kernel estimators are unbiased. Thus`ˆ≤ f ≤ uˆ is equivalent to T^∗ > κ_α. Moreover, using

general theory for gaussian measures on Banach spaces one can show that the distribution ofT^∗ is continuous on]0,∞[.

Sufficient conditions for requirements (4) and (5) in general are provided by Lemma 8.1 in Section 8. The confidence band presented in Section 3 is a special case of the one derived here, if we defineψ^(`)(x) := 1{x ∈[−1,0]}andψ^(u)(x) := 1{x ∈[0,1]}and apply postprocessing as described below.

4.2 Postprocessing of confidence bands

Any confidence band(ˆ`,u)ˆ forf can be enhanced, if we replace`(x)ˆ andu(x)ˆ with ˆˆ

`(x) := infn

g(x) :g∈ G,`ˆ≤g≤uˆo

and u(x) := supˆˆ n

g(x) :g∈ G,`ˆ≤g≤uˆo , respectively. Here we assume tacitly that the set{g∈ G: ˆ`≤g≤u}ˆ is nonempty.

In case ofG=G_↑one can easily show that ˆˆ

`(x) = sup

t∈[0,x]

`(t)ˆ and u(x) =ˆˆ inf

s∈[x,1] u(s).ˆ

Note also that`ˆˆanduˆˆare isotonic, whereas the raw functions`ˆanduˆneed not be.

In case ofG =G_convthe modified upper bounduˆˆis the greatest convex minorant ofuˆand can be computed (in discrete models) by means of the pool-adjacent-violators algorithm (cf. Robertson et al. 1988). The modified lower bound`(x)ˆˆ can be shown to be

ˆˆ

`(x) = max (

sup

0≤s<t≤x

u(s) +ˆˆ `(t)ˆ −u(s)ˆˆ

t−s (x−s) , sup

x≤s<t≤1

u(t)ˆˆ − u(t)ˆˆ −`(s)ˆ

t−s (t−x) )

.

This improved bound`ˆˆis not a convex function, though more regular than the raw function `.ˆ Figure 2 depicts some hypothetical confidence band(ˆ`,u)ˆ for a function f ∈ G<sub>conv</sub> and its im- provement(`,ˆˆu).ˆˆ

4.3 Adaptivity in terms of rates

Whenever we construct a band following the recipe above we end up with a confidence band adapting to the unknown smoothness off in terms of rates of convergence. For β, L > 0 the H¨older smoothness classH_β,Lis defined as follows: In case of0< β≤1let

H_β,L :=

n

g:|g(x)−g(y)| ≤L|x−y|^βfor allx, y o

.

Figure 2: Improvement(`,ˆˆu)ˆˆ of a band(ˆ`,u)ˆ ifG =G_conv. In case of1< β≤2let

H_β,L := n

g∈ C¹ :g⁰ ∈ H_β−1,Lo .

Theorem 4.1. Suppose thatf ∈ G ∩ H_β,L, where eitherG =G_↑ andβ ≤1, orG =G_conv and 1≤β≤2. Let(ˆ`,u)ˆ be the confidence band forf based on test functionsψ<sup>(`)</sup>, ψ^(u)as described previously. Then there exists a constant∆depending only on(β, L)and(ψ^(`), ψ^(u))such that

sup

t∈[n,1−n]

ˆ

u(t)−`(t)ˆ

≤ ∆ρn

1 +κα+T(ψ^(u)) +T(−ψ^(`)) log(en)^1/2

, where_n:=ρ^1/βn and

ρn :=

log(en) n

β/(2β+1)

.

Using the same arguments as Khas’minskii (1978) one can show that for any0≤r < s≤1,

f∈G∩Hinf_β,L IP_f (

sup

t∈[r,s]

(ˆu(t)−`(t))ˆ ≤∆ρ_n )

→ 0,

provided that ∆ > 0 is sufficiently small. Thus our confidence bands adapt to the unknown smoothness off.

4.4 Testing hypotheses aboutf(t)

In order to find suitable kernel functionsψ^(`), ψ^(u)we proceed similarly as D¨umbgen and Spokoiny (2001, Section 3.2). That means we consider temporarily tests of the null hypothesis

F_o :=

n

f ∈ G ∩L²[0,1] :f(t)≤r−δ o

versus the alternative hypothesis

F_A := n

f ∈ G ∩ H_k,L :f(t)≥ro . Heret∈[0,1],r ∈RandL, δ >0are arbitrary fixed numbers, while (8) (G, k) = (G_↑,1) or (G, k) = (G_conv,2).

Note that F_o andF_A are closed, convex subsets of L²[0,1]. Suppose that there are functions f_o∈ F_oandf_A∈ F_Asuch that

Z 1 0

(fo−fA)(x)²dx = min

go∈Fo, gA∈F_A

Z 1 0

(go−gA)(x)²dx.

Then optimal tests ofF_o versusF_Aare based on the linear test statisticR1

0(f_A−f_o)dY, where critical values have to be computed under the assumptionf = fo. The problem of finding such functionsf_o, f_Ais treated in Section 8. Here is the conclusion: Let

(9) ψ^(`)(x) :=

( 1{x∈[−1,0]}(1 +x) ifG=G_↑, 1{x∈[−2,2]}

1−(3/2)|x|+x²/2

ifG=G_conv. Then the functions

(10) f_A(s) :=

r+L(s−t) ifG=G_↑ r+L(s−t)²/2 ifG=G_conv and

f_o := f_A−δψ^(`)_h,t withh:= (δ/L)^1/k

solve our minimzation problem, provided that a^{(`)</sup>h ≤ t ≤ 1−b<sup>(`)}h. Thus the optimal linear test statistic may be written asR1

0 ψ_h,tdY =ψY(h, t). Elementary considerations show that the inequality

fˆ_h^{(`)</sup>(t)−kψ<sup>(`)}k(Γ(d^{(`)</sup>h) +κα) h1, ψ<sup>(`)}i(nh)^1/2 ≤ ro

is equivalent to

ψY(h, t) ≤ n^1/2hroh1, ψ^{(`)</sup>i+h<sup>1/2</sup>kψ<sup>(`)}k(Γ(d^(`)h) +κα)

= IE_fo(ψY(h, t)) + Var(ψY(h, t))^1/2(Γ(d^(`)h) +κα).

Thus our lower confidence bound`ˆmay be interpreted as a multiple test of all null hypotheses {f ∈ G :f(t)≤ro}witht∈[0,1]andro∈R.

Analogous considerations yield a candidate forψ^(u): Let F_o :=

n

f ∈ G ∩L²[0,1] :f(t)≥r+δ o

and

F_A := n

f ∈ G ∩ H_k,L :f(t)≤ro . Then the functionf_Ain (10) and

f_o := f_A+δψ^(u)_h,t withh:= (δ/L)^1/k form a least favorable pair(f_o, f_A)inF_o× F_A, where

(11) ψ^(u)(x) :=

1{x∈[0,1]}(1−x) ifG=G_↑, 1{x∈[−2^1/2,2^1/2]}(1−x²/2) ifG=G_conv. Figures 3 and 4 depict the functionsψ^(`)in (9) andψ^(u)in (11).

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

Figure 3: Kernel functionsψ^(`), ψ^(u)forG_↑.

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0 0.2 0.4 0.6 0.8 1

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

0 0.2 0.4 0.6 0.8 1

Figure 4: Kernel functionsψ^(`), ψ^(u)forG_conv.

4.5 Optimal constants and local adaptivity

Now we are going to show that our multiscale confidence band (ˆ`,u), if constructed with theˆ kernel functions in (9) and (11), is locally adaptive in a certain sense. Precisely, we consider an arbitrary fixed functionfo ∈ G ∩ C^kwith(G, k)as specified in (8). We analyze quantities such as

k(ˆu−fo)wk⁺_r,s and k(f_o−`)wkˆ ⁺_r,s, wherewis some positive weight function on the unit interval and

kgk⁺_r,s := sup

t∈[r,s]

g(t).

The functionwshould reflect local smoothness properties offo in an appropriate way. The fol- lowing theorem demonstrates that the k–th derivative of f_o, denoted by ∇^kf_o, plays a crucial role.

Theorem 4.2. For arbitrary fixed numbers0≤r < s≤1let L := max

t∈[r,s]∇^kf_o(t).

Then for anyγ ∈]0,1[, inf

(ˆ`,ˆu)

IP_fon

kf−`kˆ <sup>+</sup><sub>r,s</sub>≥γ∆<sup>(`)</sup>L^1/(2k+1)ρ_no

≥ 1−α+o(1), inf

(ˆ`,ˆu)

IP_fo n

kˆu−fk⁺_r,s≥γ∆^(u)L^1/(2k+1)ρn

o

≥ 1−α+o(1),

where both infima are taken over all confidence bands(ˆ`,u)ˆ satisfying (2), and

∆^(z) :=

(k+ 1/2)kψ^(z)k²−k/(2k+1)

, ρn :=

log(en) n

k/(2k+1)

.

In case of G = G_↑, the critical constants are ∆^(`) = ∆^(u) = 2^1/3 ≈ 1.260. In case of G=G_conv,

∆^(`) = (3/4)^2/5 ≈ 0.891 and ∆^(u) = 3^2/5/128^1/5 ≈ 0.588.

This indicates that bounding a convex function from below is more difficult than finding an upper bound.

In view of Theorem 4.2 we introduce for arbitrary fixed >0the weight function w :=

max(∇^kfo, )

−1/(2k+1)

reflecting the local smoothness of fo. The next theorem shows that our particular confidence band(ˆ`,u)ˆ attains the lower bounds of Theorem 4.2 pointwise. Suprema such ask(f<sub>o</sub>−`)wˆ k⁺_r,s andk(ˆu−fo)wk⁺_r,sattain their respective lower bounds∆^(`),∆^(u)up to a multiplicative factor 2^k/(k+1/2)+op(1).

Theorem 4.3. Let(ˆ`,u)ˆ be the confidence band based on the kernel functions in (9) and (11). If f =f_o, then for arbitrary >0and anyt∈]0,1[,

(f_o−`)(t)wˆ (t) ≤

∆^(`)+o_p(1) ρ_n, (ˆu−f_o)(t)w(t) ≤

∆^(u)+o_p(1) ρ_n. Moreover,

k(f_o−`)wˆ k⁺_,1− ≤

2^k/(k+1/2)∆^(`)+op(1)

ρn, k(ˆu−f_o)wk⁺_,1− ≤

2^k/(k+1/2)∆^(u)+o_p(1) ρ_n.

If we used kernel functions differing from (9) and (11), then pointwise optimality would be lost, and the constants for the supremum distances would get worse.

5 Simulations and numerical examples

Here we demonstrate the performance of the procedures in Section 4. We replace the continuous white noise model with a discrete one: Suppose that one observes a random vectorY~ ∈Rⁿwith components

(12) Yi = f(xi) +i,

wherexi := (i−1/2)/n, and the random errorsi are independent with Gaussian distribution N(0, σ²). Our kernel functionsψ^(`)andψ^(u)are rescaled as follows:

ψ^(`)(x) :=

( 1{x∈[−1,0]}(1 +x) ifG =G_↑, 1{x∈[−1,1]}

1−3|x|+ 2x²

ifG =G_conv, ψ^(u)(x) :=

1{x∈[0,1]}(1−x) ifG=G_↑, 1{x∈[−1,1]}(1−x²) ifG=G_conv.

Note that nowa^{(`)</sup>, a<sup>(u)</sup>, b<sup>(`)}, b^(u) ∈ {0,1}. For convenience we compute kernel estimators and confidence bounds forf only on the gridT_n := {1/n,2/n, . . . ,1−1/n}, while the bandwidth parameterhis restricted to

H_n :=

{1/n,2/n, . . . ,1} ifG=G_↑, {1/n,2/n, . . . ,bn/2c/n} ifG=G_conv.

Letψ stand forψ^(`) orψ^(u) with support[−a, b]. Then forh ∈ Hn andt ∈ T_nwithah ≤ t ≤ 1−bhwe define

ψ ~Y(h, t) :=

n

X

i=1

ψxi−t h

Yi =

bnh

X

j=1−anh

ψj−1/2 nh

Ynt+j

and

fˆh(t) := ψ ~Y(h, t) S_nh , where S_d stands for Pd

j=1−dψ((j −1/2)/d). The standard deviation of fˆ_h(t) equals σ_h :=

σR^1/2_nh/S_nh, whereR_d := Pd

j=1−dψ((j−1/2)/d)². Tedious but elementary calculations show that in case ofG=G_↑,

S_d = d/2 and R_d = d/3−1/(12d).

In case ofG =G_conv,

S_d^{(`)</sup> = d/3−1/(3d) and R<sup>(`)}_d = 4d/15−1/(2d) + 7/(30d³), S_d^(u) = 4d/3 + 1/(6d) and R^(u)_d = 16d/15 + 7/(120d³).

Note that here S₁^{(`)</sup> = 0 = ψ<sup>(`)}Y~(1/n,·), whence the bandwidth 1/n is excluded from any computation involvingψ^(`).

As for the bias of these kernel estimators, one can deduce from Lemma 8.1 thatIE ˆf_h^(`)(t) ≤ f(t) and IE ˆf_h^(u)(t) ≥ f(t) whenever f ∈ G. Here is a discrete version of our multiscale test statistic:T_n^∗ := max

Tn(ψ^(`)), Tn(−ψ^(u))

, where Tn(±ψ) := max

h∈Hn

t∈Tn∩[ah,1−bh]max

±σ⁻¹R_nh^−1/2ψ ~E(h, t)−Γ((a+b)h)

withE~ := (i)ⁿ_i=1. Letκα,nbe the(1−α)–quantile ofT_n^∗. Then

`(t)ˆ := max

h∈H_n:t∈[a^{(`)</sup>h,1−b<sup>(`)}h]

fˆ_h^{(`)</sup>(t)−σ<sub>h</sub><sup>(`)}(Γ(d^(`)h) +κα,n)

, ˆ

u(t) := min

h∈Hn:t∈[a^(u)h,1−b^(u)h]

fˆ_h^(u)(t) +σ_h^(u)(Γ(d^(u)h) +κ_α,n) , defines a confidence band forfsuch that

IP

n`ˆ≤f ≤uˆonT_no

≥ 1−α wheneverf ∈ G.

Equality holds ifG = G_↑ andf is constant, or ifG =G_conv andf is linear. If the noise variance σ²is unknown, it may be estimated as described in D¨umbgen and Spokoiny (2001). Then, under moderate regularity assumptions onf, our confidence bands haveasymptoticcoverage probability at least1−αasntends to infinity.

Critical values. For various values ofnwe estimated several quantilesκα,n in 9999 Monte- Carlo simulations; see Table 1. One can easily show that the critical value κ_α,n converges to the corresponding quantileκα for the continuous white noise model as n → ∞. Software for the computation of critical values as well as confidence bands may be obtained from the author’s URL.

G_↑ G_conv

n κ0.5,n κ0.1,n κ0.05,n κ0.5,n κ0.1,n κ0.05,n

100 0.330 1.092 1.349 0.350 1.053 1.283 200 0.433 1.146 1.392 0.430 1.121 1.342 300 0.475 1.169 1.416 0.470 1.126 1.342 400 0.507 1.204 1.446 0.489 1.128 1.340 500 0.526 1.222 1.450 0.512 1.143 1.358 700 0.570 1.252 1.492 0.536 1.162 1.380 1000 0.585 1.250 1.483 0.552 1.178 1.393 Table 1: Some critical values for the discrete white noise model

Two numerical examples. Figure 5 shows a simulated data vectorY~ withn= 500compo- nents together with the corresponding95%–confidence band(ˆ`,u)ˆ after postprocessing, wheref is assumed to be isotonic. The latter function is depicted as well. Note that the band is compara- tively narrow in the middle of]0,1/3[, on whichf is constant. On]1/3,1]the widthuˆ−`ˆtends to inrease, as does∇f. These findings are in accordance with Theorem 4.3.

An analogous plot for a convex function f can be seen in Figure 6. Note that the deviation f−`ˆis mostly greater thanuˆ−f, as predicted by Theorem 4.3.

6 Proofs

Proof of Theorem 3.1. In order to prove lower bounds we construct unfavorable subfamilies of G_↑ similarly as Khasminski (1978). For a given integer m > 0 we defineI1 := [0,1/m]and I_j := ](j−1)/m, j/m]for1< j≤m. Then we define step functionsgandh_ξforξ∈R^mvia

g(t) := 2j−1 and h_ξ(t) := ξj fort∈Ij,1≤j≤m.

For anyδ > 0andξ ∈ [−δ, δ]^m the functionδg+h_ξ is isotonic on[0,1]. Now we restrict our attention to the parametric submodelF_o = n

δg+h_ξ : ξ ∈ [−δ, δ]^mo

of G_↑ ∩L²[0,1]. Any confidence band(ˆ`,u)ˆ forf =δg+h_ξdefines a confidence setS=S1×S2× · · · ×Smforξvia

Sj :=

h sup

t∈I_j

`(t)ˆ −δ(2j−1),inf

t∈Ij

ˆ

u(t)−δ(2j−1) i

.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 -4

-2 0 2 4 6 8

Figure 5: DataY~ and95%–confidence band forf ∈ G_↑.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

-2 0 2 4 6 8

Figure 6: DataY~ and95%–confidence band forf ∈ G_conv.

Here`ˆ≤f ≤uˆif, and only if,ξ∈S. Moreover, D(ˆ`,u)ˆ ≥ max

j=1,...,mlength(Sj) for1/(m+ 1)≤ <1/m.

However,

logdIPδg+h_ξ

dIP_δg (Y) = n^1/2 Z 1

0

h_ξdY˜ −n Z 1

0

h_ξ(t)²dt/2

=

m

X

j=1

(n/m)^1/2ξ_jX_j −(n/m)ξ_j²/2

= logdN((n/m)^1/2ξ, I) dN(0, I) (X), whereY˜(t) :=Y(t)−n^1/2Rt

0δg(s)dsandX := (Xj)^m_j=1with components Xj := m^1/2

Y˜(j/m)−Y˜((j−1)/m)

.

In case off = δg these random variables are independent and standard normal. Consequently, Xis a sufficient statistic for the parametric submodelF_owith distributionN_m((n/m)^1/2ξ, I)in case off =δg+h_ξ. In particular, the conditional distribution ofSgivenXdoes not depend on ξ. Hence lettingδ = (n/m)^−1/2c_mwithc_m := (2 logm)^1/2 it follows from Theorem 7.1 (b) in Section 7 that for1/(m+ 1)≤ <1/m,

f∈G_↑inf∩L²[0,1] IP_f

n`ˆ≤f ≤uˆandD(ˆ`,u)ˆ ≤2cm−bm

(n/m)^1/2 o

≤ min

ξ∈[−δ,δ]^m IP_ξ n

ξ∈Sand max

j=1,...,mlength(Sj)≤2cm−bm

(n/m)^1/2 o

≤ bm,

whereb1, b2, b3, . . . are universal positive numbers such thatlimm→∞bm = 0. This entails the assertion of Theorem 3.1 withlog(1/)in place oflog(e/)and

b() := (2 log(1/))^1/2−(m)^1/2(c_m−b_m) for1/(m+ 1)≤ <1/m.

Finally note thatlog(e/)^1/2 = log(1/)^1/2+o(1)as↓0.

Proof of Theorem 4.1. Instead of an upper bound foruˆ−`ˆwe prove an upper bound foruˆ−f, because analogous arguments apply tof −`. In what follows letˆ ψ =ψ^(u) with support[−a, b].

Fort∈[0,1]andh >0withah≤t≤1−bh, ˆ

u(t)−f(t) ≤ fˆh(t)−f(t) +kψk(Γ((a+b)h) +κα) h1, ψi(nh)^1/2

= D

f(t+h·)−f(t), ψE

h1, ψi + ψW(h, t)

n^1/2hh1, ψi +kψk(Γ((a+b)h) +κ_α) h1, ψi(nh)^1/2

≤ D

f(t+h·)−f(t), ψE h1, ψi +

kψk

2Γ((a+b)h) +κ_α+T(ψ) h1, ψi(nh)^1/2 . (13)

For any functiong∈ H_β,L,

g(x)−g(0)

≤ L|x|^β ifβ≤1,

g(x)−g(0)−g⁰(0)x

≤ L|x|^β if1< β≤2.

Sincef(t+h·)∈ H_β,Lhβ iff ∈ H_β,L, this implies that D

f(t+h·)−f(t), ψ E

h1, ψi ≤ Lh^βRb

−a|x|^β|ψ(x)|dx

h1, ψi ≤ ∆h^β.

Here and subsequently∆denotes a generic constant depending only on(β, L)andψ. Its value may vary from one place to another. In case of t ∈ [_n,1−_n] andh = _n/max(a, b) the right-hand side of (13) is not greater than

∆^β_n+

∆

log(en)^1/2+κ_α+T(ψ)

(nn)^1/2 = ∆ρ_n

1 +κα+T(ψ) log(en)^1/2

.

Proof of Theorem 4.2.We prove only the lower bound forf_o−`, becauseˆ uˆ−f_ocan be treated analogously. It suffices to consider the caseL >0and to show that for any fixed numberγ ∈]0,1[,

IPfo

n

kf_o−`kˆ <sup>+</sup><sub>r,s</sub>≥γ∆<sup>(`)</sup>L^1/(2k+1)ρn

o

≥ 1−α+o(1)

for arbitrary confidence bands(ˆ`,u) = (ˆˆ `n,uˆn)satisfying (2). Without loss of generality one may assume that

∇^kfo ≥ L on[r, s].

Otherwise one could increaseγ and decreaseLwithout changing γL^1/(2k+1), and replace[r, s]

with some nondegenerate subinterval. Letψstand for ψ^(`) with support [−a, b]. For 0 < h ≤ (s−r)/(a+b)and positive integersj ≤m:=b(s−r)/((a+b)hclet

tj := s+ah+ (j−1)(a+b)h and fj := fo−Lh^kψ_h,tj.

It follows from Lemma 8.4 that these functionsf_j belong toG ∩L²[0,1]. Thus (2) implies that the event

A := n

`ˆ≤f_j for somej≤mo

satisfies the inequalityIP_fj(A) ≥ 1−αfor allj ≤ m. Since kf_o−f_jk⁺_r,s ≥ δ, this entails the inequality

IP_fo n

kf_o−`kˆ ⁺_r,s≥Lh^k o

≥ IP_fo(A) ≥ 1−α−min

j≤m

IP_fj(A)−IP_fo(A)

.

Now leth := (cρn)^1/k so thatLh^k = Lcρn, wherec > 0is some number to be specified later.

For sufficiently largenthis bandwidthhis smaller than(s−r)/(a+b). Then logdIP_fj

dIP_fo(Y) = n^1/2h^k+1/2LkψkX_j−nh^2k+1L²kψk²/2, whereX_j :=h^−1/2kψk⁻¹R1

0 ψ_h,tjdY˜ andY˜(t) :=Y(t)−n^1/2Rt

0f_o(x)dx. ThusX:= (X_j)^m_j=1 is a sufficient statistic for the restricted model{f_o, f1, f2, . . . , fm}, where L_fo(X) is a standard normal distribution on R^m. Thus it follows from Theorem 7.1 (a) and a standard sufficiency argument that

n→∞lim min

1≤j≤m

IP_fj(A)−IP_fo(A)

= 0 if lim

n→∞

nh^2k+1L²kψk² 2 logm < 1.

Sincelogm= (1 +o(1)) log(n)/(2k+ 1), the limit on the right hand side is equal to c^(2k+1)/kL²kψk²(k+ 1/2)

and smaller than one ifcequalsγ∆^(`)L^−2k/(2k+1). In that case, the lower boundLh^k=Lcρ_nfor

kf_o−`kˆ <sup>+</sup><sub>r,s</sub>equalsγ∆<sup>(`)</sup>L^1/(2k+1)ρnas desired.

Proof of Theorem 4.3. Again we restrict our attention tofo−`ˆand letψ := ψ<sup>(`)</sup> with support [−a, b]. For any fixed >0and arbitraryt∈[0,1]leth_t>0and

L_t := max

s∈[t−aht,t+bht]∩[0,1]max(∇^kf_o(s), ).

In case ofah_t≤t≤1−bh_tthe inequality(f_o−`)(t)ˆ ≥L_th^k_t implies that fˆ_ht(t)−

kψk

Γ((a+b)ht) +κα

(nht)^1/2h1, ψi ≤ fo(t)−Lth^k_t. Sincef =fo, this can be rewritten as

ψW(ht, t) h^1/2_t kψk

≤ −(nht)^1/2 kψk

D

fo(t+ht·)−fo(t) +Lth^k_t, ψ E

+ Γ((a+b)ht) +κα

≤ −n^1/2Lth^k+1/2_t kψk+ Γ((a+b)ht) +κα, where the latter inequality follows from Lemma 8.4 (c). Specifically let

h_t := cw(t)²ρ^1/k_n

for some positive constantcto be specified later. By continuity of∇^kf_o, the weight functionwis bounded away from zero and infinity. Henceht→ 0andLtmax(∇^kfo(t), )⁻¹ →1, uniformly

int∈[0,1]. In particular,

Γ((a+b)h_t) ≤ (k+ 1/2)^−1/2log(en)^1/2 forn≥n_o, n^1/2L_th^k+1/2_t kψk ≥ c^k+1/2kψklog(en)^1/2,

L_th^k_t ≤ w(t)⁻¹c^k(1 +b_n)ρ_n,

wherenoandbnare positive numbers depending only onfo,andcsuch thatbn → 0. Conse- quently, forn≥n_o,

aht≤t≤1−bht and (fo−`)(t)wˆ (t) ≥ c^k(1 +bn)ρn

implies that

ψW(ht, t) h^1/2_t kψk

≤ −

c^k+1/2kψk −(k+ 1/2)^−1/2

log(en)^1/2+κα.

Wheneverc > (∆^{(`)</sup>)<sup>1/k</sup>, the right-hand side of the preceding inequality tends to minus infinity, while the random variable on the left-hand side has mean zero and variance one. Since the limit ofc<sup>k</sup>(1 +b<sub>n</sub>)can be arbitrarily close to∆<sup>(`)}, these considerations show that(f_o−`)(t)wˆ (t) ≤ (∆<sup>(`)</sup>+op(1))ρnfor any fixedt∈]0,1[.

Ifnis sufficiently large, thenah_t≤t≤1−bh_tand ψW(ht, t)

h^1/2_t kψk

≥ −T(−ψ)−Γ((a+b)ht) for allt∈[,1−]. Consequently,

sup

t∈[,1−]

(f_o−`)(t(wˆ (t) ≥ c^k(1 +b_n) implies that

T(−ψ) ≥ n^1/2L_th^k+1/2_t kψk −2Γ((a+b)h_t)−κ_α

≥

c^k+1/2kψk −2(k+ 1/2)^−1/2

log(en)^1/2−κ_α.

Wheneverc >2^1/(k+1/2)(∆^(`))^1/k, the right hand side of the preceding inequality tends to infinity.

Since the limit ofc^k(1 +b_n)can be arbitrarily close to2^k/(k+1/2)∆<sup>(`)</sup>, these considerations reveal thatk(f<sub>o</sub>−`)wˆ k⁺_,1−is not greater than

2^k/(k+1/2)∆^(`)+op(1)

ρn.

7 Some decision theory

LetX = (X_i)^m_i=1 be a random vector with distribution N_m(θ, I). In what follows we consider testsφ:R^m →[0,1]and confidence sets

S = S₁×S₂× · · · ×S_m

forθwith random intervalsSj ⊂R. The conditional distribution ofS, givenX, does not depend onθ. The possibility of randomized confidence setsS, i.e. confidence sets not just being a function ofX, has to be included for technical reasons. Unless specified differently, asymptotic statements in this section refer tom→ ∞.

Theorem 7.1. Letcm := (2 logm)^1/2. There are universal positive numbersbm with bm → 0 such that the following two inequalities are satisfied:

(a)For arbitrary testsφ,

j=1,...,mmin IE_(cm−bm)ejφ(X)−IE₀φ(X) ≤ b_m, wheree₁, e₂, . . . , e_mdenotes the standard basis ofR^m.

(b)For arbitrary confidence setsS as above, min

θ∈[−cm,cm]^m IP_θn

θ∈Sand max

j=1,...,mlength(S_j)<2(c_m−b_m)o

≤ b_m.

Proof of Theorem 7.1.Part (a) is classical and can be proved by a Bayesian argument; see for in- stance Ingster (1993) or D¨umbgen and Spokoiny (2001). In order to prove part (b) we also consider a Bayesian model: Letθhave independent components each of which is uniformly distributed on the three-point setK_m :={−κ_m,0, κ_m}, whereκ_m:=c_m−b_mwith constantsb_m∈[0, c_m]to be specified later on. LetL(X|θ) =N_m(θ, I). LetIP(·),IE(·)denote probabilities and expectations in this Bayesian context, whereasIP_θ(·),IE_θ(·) are used in case of a fixed parameterθ. For any confidence setS,

θ∈[−cminm,cm]^m IPθ

n

θ∈Sand max

j=1,...,mlength(Sj)<2κm

o

≤ IP n

θ∈Sand max

j=1,...,mlength(Sj)<2κm

o

≤ IP{θ∈S},˜ where

S˜ :=

( S if max

j=1,...,mlength(Sj)<2κm, {0} × · · · × {0} else.

The conditional distribution ofθgiven(X, S)is also a product ofm probability measures: For anyη∈K_m^m,

IP(θ=η|X, S) =

m

Y

i=1

g(ηi|Xi) with g(z|x) := exp(−(x−z)²/2) P

y∈Kmexp(−(x−y)²/2).

Since each factorS˜j ofS˜contains at most two points fromKm, IP{θ∈S}˜ = IEIP(θ∈S˜|X, S)

≤ IE max

η∈K_m^m IP(θ_i6=η_ifori= 1, . . . , m|X, S)

= IE

m

Y

i=1

1− min

z∈Km

g(z|Xi)

=

1−IE min

z∈Km

g(z|X₁)m

≤

1−3⁻¹IE min

z∈Km

exp(−(X₁−z)²/2)m

. The latter expectation can be bounded from below as follows:

3⁻¹IE min

z∈Km

exp(−(X₁−z)²/2)

≥ 3⁻¹IP{|X₁| ≤bm/2}exp(−(κ_m+bm/2)²/2)

≥ 3⁻¹IP{|θ₁|= 0,|X₁| ≤bm/2}exp(−(c_m−bm/2)²/2)

= 9⁻¹(2π)^−1/2(bm+O(b²_m)) exp(cmbm/2−b²_m/8)m⁻¹.

In case of b_m := 1{m > 1}c^−1/2_m = o(1) the latter bound is easily seen to be a_mm⁻¹ with am =am(bm)→ ∞. Thus

IP{θ∈S} ≤˜ (1−a_mm⁻¹)^m → 0.

Replacingbmwithmax{b_m,(1−amm⁻¹)^m}yields the assertion of part (b).

8 Related optimization problems

As in Section 4 let(G, k) be either(G_↑,1)or(G_conv,2). In view of future applications to other regression models we extend our framework slightly and considerhg, hi := R

gh dµ, kgk :=

hg, gi^1/2for some measureµon the real line such thatµ(C)<∞for bounded intervalsC⊂R.

Letψbe some bounded function on the real line withψ(x) = 0forx6∈[−a, b]andh1, ψi ≥0, where a, b ≥ 0. The next lemma provides sufficient conditions for one of the following two requirements:

hg, ψi ≤ g(0)h1, ψi wheneverg∈ G,1_[−a,b]g∈L¹(µ), (14)

hg, ψi ≥ g(0)h1, ψi wheneverg∈ G,1_[−a,b]g∈L¹(µ).

(15)

Lemma 8.1. LetG = G_↑ andψ ≥ 0. Thenb = 0entails condition (14), whilea = 0implies condition (15).

LetG =G_convandR∞

−∞xψ(x)µ(dx) = 0. Condition (15) is satisfied ifψ≥0. On the other hand, condition (14) is a consequence of the following two requirements: R

x^±ψ(x)µ(dx) = 0 and

ψ

≥0 on[c, d]

≤0 onR\[c, d]

for some numbersc < 0 < d, where µ([−a, c]), µ([d, b]) > 0. (Herey⁺ := max(y,0) and y⁻ := max(−y,0).)

With Lemma 8.1 at hand one can solve two mimimization problems leading to the special kernels in (9) and (11). In both cases we consider two disjoint convex sets G_o,G_A ⊂ G and construct functionsG_o∈ G_o,G_A∈ G_Asuch that

(16) kG_o−G_Ak = min

go∈G_o, gA∈G_A kg_o−g_Ak.

Theorem 8.2. LetG_o:=n

g∈ G :g(0)≤ −1o

andG_A:=n

g∈ G ∩ H_k,1 :g(0)≥0o

. In case ofG =G_↑letGA(x) :=xand

G_o(x) :=

−1 ifx∈[−1,0], G_A(x) else.

In case ofG =G_convletGA(x) :=x²/2and G_o(x) :=

−1 + (a/2 + 1/a)x⁻+ (b/2 + 1/b)x⁺ ifx∈[−a, b],

G_A(x) else,

wherea, b≥2^1/2 are chosen such thatR

x^±(GA−Go)(x)µ(dx) = 0.

Then equation (16) holds in both cases. More precisely, the functionψ:=G_A−G_osatisfies the inequalitiesh1, ψi ≥ kψk², (14) and

(17) hg, ψi ≥ kψk²− h1, ψi wheneverg∈ H_k,1, g(0)≥0.

In case ofµbeing Lebesgue measure,ψ=GA−Gocoincides with the functionψ^(`)in (9), wherea=b= 2.

Theorem 8.3. LetG_o :=

n

g ∈ G :g(0) ≥1 o

,G_A:=

n

g ∈ G ∩ H_k,1 :g(0)≤0 o

, and define G_Aas in Theorem 8.2. In case ofG=G_↑let

Go(x) :=

0 ifx∈[0,1], GA(x) else.

In case ofG =G_convsuppose thatµ(]−∞,0[), µ(]0,∞[)>0and let G_o(x) :=

1 +cx ifx∈[−a, b], G_A(x) else,

where a := −c+ (c² + 2)^1/2, b := c+ (c² + 2)^1/2, and c is chosen such that R

x(Go − G_A)(x)µ(dx) = 0.

Then equation (16) is satisfied in both cases. More precisely, the function ψ := G_o −G_A satisfies the inequalitiesh1, ψi ≥ kψk², (15) and

(18) hg, ψi ≤ h1, ψi − kψk² wheneverg∈ H_k,1, g(0)≥0.

In case ofµbeing Lebesgue measure,ψ=G_o−G_Acoincides with the functionψ^(u)in (11), wherec= 0anda=b= 2^1/2.

The following lemma summarizes essential properties of the optimal kernelsψ^{(`)</sup>andψ<sup>(u)</sup>. Lemma 8.4. Letψ<sup>(`)}andψ^(u)be the kernel functions in (9) and (11), and leth, L >0andt∈R.

(a)IfG=G_↑, thenh1, ψ^{(`)</sup>i=h1, ψ<sup>(u)</sup>i = 1/2andkψ<sup>(`)}k² =kψ^(u)k² = 1/3. Iff :R→R satisfiesf(y)−f(x)≥L(y−x)for allx < y, then

f−Lh⁻¹ψ^(`)_h,t, f +Lh⁻¹ψ_h,t^(u) ∈ G_↑.

(b)IfG = G_conv, thenh1, ψ^{(`)</sup>i = 2/3, kψ<sup>(`)}k² = 8/15,h1, ψ^(u)i = 2^2.5/3andkψ^(u)k² = 2^4.5/15. Let f :R → Rbe absolutely continuous with derivativef⁰ such thatf⁰(y)−f⁰(x) ≥ L(y−x)for allx < y. Then

f −Lh⁻²ψ_h,t^(`), f+Lh⁻²ψ^(u)_h,t ∈ G_conv. (c)In general, for any functionf ∈ H_k,L,

D

f(t+h·)−r+Lh^k, ψ^(`)E

≥ Lh^kkψ^(`)k² iff(t)≥r, D

f(t+h·)−r−Lh^k, ψ^(u)E

≤ −Lh^kkψ^(u)k² iff(t)≤r.

Proof of Lemma 8.1.The assertions forG=G_↑are a simple consequence ofg≤g(0)on]−∞,0]

andg≥g(0)on[0,∞[.

Now let G = G_conv. If ψ ≥ 0 andR

xψ(x)µ(dx) = 0, then Condition (15) follows from Jensen’s inequality applied to the probability measureP(dx) =h1, ψi⁻¹ψ(x)µ(dx).

On the other hand, suppose thatψ≥0on[c, d]andψ≤0onR\[c, d], wherec <0< dand µ([−a, c]), µ([d, b])>0. Forg∈ G_conv with1_[−a, b]g ∈L¹(µ), bothg(c)andg(d)have to be finite, and we define

˜

g(x) := g(x)−

d⁻¹(g(d)−g(0))x ifx≥0, c⁻¹(g(c)−g(0))x ifx≤0.