High-confidence nonparametric fixed-width uncertainty intervals and application...

(1)

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=lsqa20

Sequential Analysis

Design Methods and Applications

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/lsqa20

High-confidence nonparametric fixed-width

uncertainty intervals and applications to projected high-dimensional data and common mean

estimation

Ansgar Steland & Yuan-Tsung Chang

To cite this article: Ansgar Steland & Yuan-Tsung Chang (2021) High-confidence nonparametric fixed-width uncertainty intervals and applications to projected high-dimensional data and common mean estimation, Sequential Analysis, 40:1, 97-124, DOI: 10.1080/07474946.2021.1847966 To link to this article: https://doi.org/10.1080/07474946.2021.1847966

Submit your article to this journal

Article views: 159

View related articles

View Crossmark data

(2)

High-confidence nonparametric fixed-width uncertainty intervals and applications to projected high-dimensional data and common mean estimation

Ansgar Steland^a and Yuan-Tsung Chang^b

aInstitute of Statistics, RWTH Aachen University, Aachen, Germany;^bDepartment of Social Information, Mejiro University, Tokyo, Japan

ABSTRACT

Nonparametric two-stage procedures to construct fixed-width confidence intervals are studied to quantify uncertainty. It is shown that the validity of the random central limit theorem (RCLT) accompanied by a consistent and asymptotically unbiased estimator of the asymptotic variance already guarantees consistency and first-order as well as second-order efficiency of the two-stage procedures. This holds under the common asymptotics where the length of the confidence interval tends toward 0 as well as under the novel proposed high-confidence asymptotics where the confidence level tends toward 1. The approach is motivated by and applicable to data analysis from distributed big data with nonnegligible costs of data queries. The following problems are discussed: Fixed-width intervals for the mean, for a projection when observing high-dimensional data, and for the common mean when using nonlinear common mean estimators under order constraints. The procedures are investi- gated by simulations and illustrated by a real data analysis.

ARTICLE HISTORY Received 16 October 2019 Revised 29 June 2020, 5 September 2020 Accepted 9 September 2020 KEYWORDS

Big data; data science; high- dimensional data; jackknife;

sequential analysis;

sequential sampling SUBJECT CLASSIFICATIONS 62L15; 60G40; 62F12; 62F15

1. INTRODUCTION

In this article, we study fully nonparametric two-stage procedures to construct a fixed- width interval for a parameter to quantify uncertainty. Both the common high-accuracy framework, where the asymptotics assumes that the width of the interval shrinks, and a novel high-confidence framework are studied. Under high-confidence asymptotics the required uncertainty in terms of the width of the interval is fixed and the asymptotics assumes that the confidence level increases. General sufficient conditions are derived that yield consistency and efficiency for both frameworks. We study three statistical problems: Nonparametric fixed-width intervals for the mean of univariate data, which may be the most common setting; for the mean projection of high-dimensional data to illustrate the application to big data; and for the common mean of two samples as a classic statistical problem leading to a nonlinear estimator, which has not yet been

CONTACT Ansgar Steland steland@stochastik.rwth-aachen.de Institute of Statistics, RWTH Aachen University, Aachen, W€ullnerstr. 3, Aachen D-52065, Germany.

Recommended by Bhargab Chattopadhyay.

ß2021 The Author(s). Published with license by Taylor & Francis Group, LLC

This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.

2021, VOL. 40, NO. 1, 97–124

https://doi.org/10.1080/07474946.2021.1847966

(3)

treated in the literature from a two-stage sampling perspective. The focus is on two- stage procedures, because they provide a good compromise between the conflicting goals of a minimal sample size, which requires purely sequential sampling, and feasibil- ity in applications in terms of required computing resources and logistic simplicity, which is better matched by one- or two-stage sampling procedures.

Two-stage sequential sampling is a well-established approach motivated by the aim to make statistical statements with minimal samples sizes without relying on purely sequential sampling. Instead, the data are sampled in two batches, a first-stage sample and a second-stage sample, if required. In the second stage, the final sample size is determined using the information contained in the first-stage (pilot) sample. The devel- opment of such procedures was mainly motivated by the need to base statistical infer- ence on small samples in a world where large samples are not available. But this technique is also of interest in various areas including emerging ones such as data science and big data, where massive amounts of variables are collected and need to be processed and analyzed: When analyzing big data distributed over many nodes of a net- work, each single query may be associated with a high response time and substantial data transmission costs, ruling out a purely sequential sampling strategy, because the benefit of fewer required observations on average is overcompensated by the high costs for each query. In contrast, the two-stage methods proposed in this article allow efficient estimations of the means of the variables and their projections with preassigned accuracy and confidence. The general construction of the sample size rules mainly follows the established approach. However, compared to the existing literature, we use a slightly modified first-stage sample size rule that takes into account prior knowledge and historical estimates, respectively, of the data uncertainty. Our studies indicate that even if we use only three data points to get a rough guess of variability, the resulting first-stage sample size is much closer to the actually required sample size, thus avoiding oversampling at this stage. In the context of distributed data, the proposed methods with this three-observations rule need at most three database queries.

This article contributes to the existing literature on two-stage procedures (see Stein (1945), Mukhopadhyay 1980; Ghosh, Mukhopadhyay, and Sen 1997; Mukhopadhyay and Silva 2009; Steland 2015, 2017; and references therein) by proposing a concrete nonparametric procedure with the following properties: For any estimator of the mean (or a parameter h) that satisfies the random central limit theorem (CLT) and whose asymptotic variance can be estimated by a consistent and asymptotically unbiased estimator, the random sample size leading to the proposed fixed-width confidence interval is consistent and asymptotically unbiased for the optimal sample size. Further, the procedure yields the right asymptotic coverage probability and exhibits first-order as well as second-order efficiency.

Further, and more important, we go beyond the classic framework that establishes the above properties when the width of the confidence interval tends toward 0. We argue that this is to some extent counterintuitive in view of the posed problem to construct a fixed-width confidence interval. It also limits the approximate validity of the results to cases where one aims at high-accuracy estimation. But in many applications it is more appropriate to fix the width of the confidence interval and to assume that a larger number of observations is due to a higher confidence level. Therefore, we

(4)

propose a novel framework and study the construction of a fixed-width interval under high-confidence asymptotics (equivalently: low error probability asymptotics). This is also motivated by the fact that in many areas such as high-quality, high-throughput production engineering or statistical genetics and brain research, where high-dimensional data are collected, large confidence levels and small significance levels, respectively, are in order and used in practice. For example, in production engineering the accuracy is fixed by the technical specifications and not by the statistician, and in genetics as well as in brain research small error probabilities are required to reach scientific relevance and to take multiple testing into account.

It is shown that the proposed two-stage procedure is valid under high-confidence asymptotics and exhibits first- and second-order efficiency properties, as long as the parameter estimator satisfies the random central limit theorem (RCLT) and a consistent and asymptotically unbiased estimator of the asymptotic variance is at our disposal.

Having in mind big data sets with a large number of variables, we then apply the general results to projections of high-dimensional data. It is assumed that the observations are given by a data stream of (possibly) increasing dimension, which is sampled in batches by our two-stage procedure. Two-stage procedures for high-dimensional data have been studied in-depth in Aoshima and Yata (2011) assuming that the dimension p tends toward 1and the sample size is either fixed or tends toward 1as well. Here we consider a projection of high-dimensional data, where, when having sampledn observations, the projection may depend on the sample size n. The asymptotic properties (consistency and efficiency) of the fixed-width confidence interval for the mean projection hold for high-accuracy asymptotics as well as high-confidence asymptotics. The dimen- sionpmay be increasing withn in an unconstrained way.

As an interesting and nontrivial classical application, we consider the problem of common mean estimation. Here one aims at estimating the mean from two samples assuming that they have the same mean but possibly different or ordered variances.

Many of the estimators proposed and studied in the literature are given by a convex combination of the sample means with convex weights depending on the sample means and the sample variances.

This article is organized as follows. Section 2 studies nonparametric two-stage fixed- width confidence intervals for the mean under both asymptotic frameworks, starting with the usual high-accuracy approach and then discussing the novel high-confidence asymptotics. Section 3 provides the results when dealing with a projection of high- dimensional data. Common mean estimation is treated in Section 4. Results from simulations and a data example are provided inSection 5.

2. NONPARAMETRIC TWO-STAGE FIXED-WIDTH CONFIDENCE INTERVAL LetY₁,Y₂,:::be independent and identically distributed with common distribution function F (F observations with mean l and finite variance r²2 ð0,1Þ: Further, let l^n be an estimator forlusing the firstn observationsY₁,:::,Y_n: We focus on the mean as the parameter of interest, but it is easy to see that all results remain true for any univariate parameterh¼hðFÞand an estimator^hn:

(5)

The classical approach to the construction of a confidence interval is based on a sample of fixed (but large) sample size N and determines a random interval ½U_N,V_N depending on the sample(s), such that its coverage probability equals the given confidence level 1a2 ð0, 1Þ for each N1 or has asymptotic coverage 1a, as N tends toward 1: As a consequence, the length L¼V_NU_N of the interval, which represents the reported accuracy, is random.

There are, however, situations where we want to report an interval of a fixed, preassigned accuracyd, symmetric around the point estimator of l, so that L¼2d:Then the coverage probability of the resulting interval ½^lNd,l^Nþd depends on the distribution of ^lN and the sample size N becomes the parameter we may select to achieve a certain confidence level. In mathematical terms, we wish to find some N so that the interval around the estimator ^lN based on a sample of size Nhas coverage probability

Pð½l^_Nd,l^_Nþd像lÞ ¼1aþoð1Þ, (2.1) as the precision parameter d tends toward 0. The o(1) term is required as the CLT (respectively RCLT) for ^lN is used to construct a solution.

Usually, d is small and N increases when d decreases. Thus, it is reasonable to consider asymptotic properties as d tends toward 0. We shall, however, also consider the case of a fixed accuracy d, not necessarily small, and study asymptotic properties when the confidence level tends toward 1.

Suppose that the estimator ^ln satisfies the CLT; that is, ffiffiffin

p ð^lnlÞ !^d Nð0,r²_lÞ, (2.2) as n! 1, for some positive constant r²_l, the asymptotic variance of our estimator for l. Throughout the article we shall assume that we have an estimator for r²_l at our disposal, which we denote by r^²n¼r^²_l,n if it is based on the first n observations. The most common choice for ^ln is, of course, the sample average Yn¼¹_nP_n

i¼1Yi, which satisfies (2.2)with r²_l¼r²: The canonical estimator for r²_l isS²_n¼_n1¹ P_n

i¼1ðYiYnÞ²: When considering a parameter h estimated by ^hn such that the analogue of (2.2) holds—that is, ffiffiffi

pn

ð^hnhÞ!^dNð0,r²_hÞ—one formally replaces r²_l by r²_h and needs an estimator ^r²_h,n having the properties required in Assumption (E) in the next section.

For simplicity of presentation and proofs, we stick to the case of the mean, however.

Invoking the CLT for^lN, it is easy to see that the problem is solved by theasymptot- ically optimal sample sizeNopt ¼ dN_opt e, where

N_opt ¼r²_lU¹ð1a=2Þ²

d² , (2.3)

because the left-hand side of (2.1)is equal to Pðj ffiffiffiffi pN

ð^lNlÞj dÞ ¼2U ffiffiffiffi pN

d=rl

1þoð1Þ: Observe that N_opt ! 1, if d#0, which in turn justifies the application of the CLT.

If r²_l were known, then dN_opt e would solve the posed problem. The proposed two- stage procedure draws a random sample of size N₀ at the first stage, which is larger than or equal to a given minimal sample size N₀:The first-stage sample size N₀ will be larger if the required precision gets smaller. The first-stage sample is used to estimate the uncertainty of the estimator, and that (random) estimate is then used to specify the

(6)

final sample size N^_opt used in the second stage. Before discussing how one should spe- cifyN₀andN^_opt, let us summarize the basic algorithm:

Preparations: Specify the minimal sample size N0, the confidence level 1a, and the precision d.

Stage I: Draw an initial sample of sizeN0N0, in order to estimate unknowns (in our case r²_l) based on that data, yielding an estimator N^_opt forN_opt ; that is, a random sample size.

Stage II: Draw additional N^_optN₀ observations to obtain a sample of size N^_opt: Estimate lby ^lN^_opt

Solution:Output the fixed-width confidence interval h^lN^_opt d,^lN^_opt þdi :

2.1. Fixed-Width Interval under High-Accuracy Asymptotics

Let us first study the classical approach to fix the confidence level 1aand to assume that the accuracy is small suggesting to investigate approximations for d#0: This framework can be calledhigh-accuracy asymptotics.

In the sequel, we review part of the literature that focused on normal data and the associated optimal estimators. We follow the arguments developed for the Gaussian case to motivate our fully nonparametric proposal where ^ln may be an arbitrary estimator satisfying the required regularity assumptions stated below in detail.

The original Stein procedure (see Stein 1945) addresses Gaussian i.i.d. observations and estimates l by the sample mean, such that r²_l¼r² and a natural estimator for r²_l based onY1,:::,Ynis S²_n:Stein uses the rule

N¼max N0,

tðN₀1Þ²_1a=2S²_N

0

d²

þ1

( )

:

Because N₀ is fixed, the procedure turns out to be inconsistent. To overcome this issue, Chow and Robbins (1965) proposed a purely sequential rule, namely,

N ¼inffnN₀ :ntðn1Þ²₁_a=2S²_n=d²g:

Mukhopadhyay (1980) noted that one gets for small d the lower bound N U¹ð1a=2Þ and proposed to increase the variance estimate slightly by 1=n: Indeed, for small enough dwe have the lower bound NtðN01Þ_1a=2=d, if one replaces the estimateS²_n

0 by S²_n

0þ1=n, because for d1, tðn1Þ²_1a=2ðS²_nþ1=nÞ

d² ¼tðn1Þ²_1a=2S²_n

d² þtðn1Þ²_1a=2 nd² tðn1Þ²_1a=2

nd² ,

such that any nN₀ with ntðn1Þ²₁_a=2ðS²_nþ1=nÞ=d² satisfies ntðn 1Þ²_1a=2=ðnd²Þ and hence n² ntðn1Þ²_1a=2=d²: This leads to ntðN01Þ_1a=2=d for any such n, such that we obtain the lower bound NtðN₀1Þ₁_a=2=d: Therefore, the purely sequential rule

(7)

N⁰¼inffnN₀:ntðn1Þ²_1a=2ðS²_nþ1=nÞ=d²g satisfiesN⁰ maxfN₀,tðN₀1Þ_1a=2=dg:

The idea is to now to use this lower bound maxfN₀,btðN₀1Þ₁_a=2=dc þ1g, as the first-stage sample size for Gaussian data; for nonnormal samples one replaces tðN₀1Þ₁_a=2 by the corresponding quantile, U¹ð1a=2Þ, of the standard normal distribution. However, this rule does not take into account the scale of data and can lead to unrealistically large sample sizes; see the data example in Section 5.

Mukhopadhyay (1980) has proposed the modified rule maxfN₀,bðtðN₀1Þ_1a=2= dÞ^2=ð1þ^c^Þc þ1g, for some 0<c<1:ccan be selected to obtain a reasonable first-stage sample size; see the discussion and example in Mukhopadhyay and Silva (2009, p. 115).

But, indeed, Mukhopadhyay’s argument also applies when using S²_n

0þf²=n0, for some f >0, and then one gets the lower bound maxfN₀,tðN₀1Þ_1a=2f=dg and maxfN₀,U¹ð1a=2Þf=dg, respectively. It is easy to check that all of the above arguments go through as well, if we replace S²_n by any guess or pilot estimate using a (very) small sample.

Three-Observations Rule: Because frequently in applications it is possible to sample at least three observations, we propose to choosef as an estimater^l, 3 ofrl, using three additional observations, on which we condition in what follows. This leads to our proposal for the first-stage sample size, namely,

N₀¼max N₀,

U¹ð1a=2Þ^rl, 3

d

þ1

( )

: (2.4)

Note that N₀depends on the preassigned precisiondand satisfiesN₀! 1, asd#0: It is natural to estimate N_opt by N^_opt ¼^r^{^}

2

N0U¹ð1a=2Þ²

d² , and this leads to the final sample size of the procedure,

N^opt¼max N0,

^r²_N₀U¹ð1a=2Þ² d²

þ1

( )

, (2.5)

which is a random variable (depending on the first-stage data), because r^²_N₀ estimates the asymptotic variance of the estimator ^lN0 using the first-stage sample of size N0. Observe that we continue to add a in notation to indicate quantities that may not be integer valued.

Let us briefly review the following facts and considerations leading to the notions of consistency and unbiasedness: Note that N_opt ! 1 and N^_opt ! 1, in probability, for any (arbitrary) weakly consistent estimator r^²_l of the asymptotic variance r²_N₀ (based on the first-stage sample of size N0), which slightly complicates their comparison. If we only know that j^r²_lr²_lj ¼o_Pð1Þ, which follows from ratio consistency j^r²_l=r²_l1j ¼ oPð1Þ, then the difference N^_optN_opt is not guaranteed to be bounded, because

jN^_optN_opt j ¼ j^r²_lr²_ljU¹ð1a=2Þ²

d² ,

where the first factor is oPð1Þ, but the second one diverges, as d#0: For this reason, N^_opt is called consistent for the asymptotically optimal sample size N_opt , if the ratio approaches 1 in probability; that is, if

(8)

N^_opt

N_opt ¼1þo_Pð1Þ,

as d#0: Having in mind that the second-stage (final) sample size N^_opt is random whereas the unknown optimal value, N_opt , is nonrandom, the question arises as to whetherN^_opt is at least close to N_opt on average. Therefore, to address this question and going beyond consistency, N^_opt is called asymptotically first-order efficient in the sense of Chow and Robbins (1965)if

EðN^_optÞ

N_opt ¼1þoð1Þ, asd#0:

Observe that the last property allows for the case that the differenceN^_optN_opt tends toward 1even in the mean, as d tends toward 0. This typically indicates that the procedure is substantially oversampling the optimal sample size. A procedure for which the estimated optimal sample size remains on average in a bounded vicinity of the optimal truth is, of course, preferable.N^_opt is calledsecond-order asymptotically efficient if

EðN^_optN_opt Þ ¼Oð1Þ, asd#0:

The regularity assumptions we need to impose are as follows:

Assumption (E). The estimator r^²_N₀ is consistent and asymptotically unbiased for r²_l; that is,

r^²_N₀

r²_l ¼1þo_Pð1Þ, Eð^r²_N₀Þ

r²_l ¼oð1Þ, as d#0:

This assumption is not restrictive and is satisfied by many estimators. For example, the jackknife variance estimator studied in Shao (1993), Shao and Wu (1989), and Steland and Chang (2019) provides an example satisfying Assumption (E).

Further, we require the following strengthening of the CLT to hold.

Assumption (A). ^ln satisfies the RCLT; that is, for any family fNa:a>0g of stopping times for which Na=a!^P k, 0<k<1, it holds that

ffiffiffiffiffiffi N_a

p ð^lNalÞ= ffiffiffiffiffiffiffiffiffiffi r²_lak q

!^dNð0, 1Þ, as a! 1:

The validity of the RCLT is required, because we have to employ a normal approximation with the first-stage sample size, which is random by construction. Clearly, however, for i.i.d. observations following an arbitrary distribution with finite second moment and l^n the arithmetic mean, Assumption (A) is well known (see, e.g., Ghosh, Mukhopadhyay, and Sen1997, theorem 2.7.2).

The following theorem summarizes the main asymptotic first-order properties of the proposed two-stage approach to construct a fixed-width confidence interval.

(9)

Theorem 2.1. Suppose that Assumption (E) is satisfied. Then the following two assertions hold true:

i. The estimated optimal sample sizeN^_opt is consistent for N_opt ; that is, N^_opt

N_opt ¼1þoPð1Þ, as d#0:

ii. N^_opt is asymptotically first-order efficient for N_opt ; that is, EðN^_optÞ

N_opt ¼1þoð1Þ, as d#0:

If, in addition, Assumption (A) holds, then we have iii. The fixed-width confidence interval I_N_^

opt has asymptotic coverage 1a; that is, P I_N_^

opt像l

¼1aþoð1Þ, as d#0:

Remark 2.1. It is worth mentioning that the proof of Theorem 2.1 (i)–(ii) shows the following stronger properties:

(i) N^_opt is consistent forN_opt , if and only ifr^²_N₀ is consistent forr²_l:

(ii) N^opt is asymptotically unbiased for N_opt , if and only if r^²_N₀ is asymptotically unbiased forr²_l:

Let us now discuss the second-order properties of the fully nonparametric procedure.

In the literature, so far second-order efficiency for the problem at hand has been studied for parametric (Gaussian) models (see Mukhopadhyay and Duggan 1999), leading to a known distribution of N^_opt, a chi-square distribution induced by the fact that the sample variance follows a chi-square law, which converges to the normal law if d !0. To achieve second-order efficiency, the probability PðN^_opt ¼N_opt Þ that the sample size is not increased in the second stage needs to decrease faster than the first-stage sample size N₀. In a parametric setting that probability can be handled and estimated by means of appropriate Taylor expansions using properties of the known distribution function.

In a fully nonparametric framework, the exact distribution is unknown to us, and estimating the probability under the limiting law is not sufficient, because we have to take into account the error of approximation. But due to the Berry-Esseen bound, the error is of the order OðN₀¹⁼²Þ: Therefore, the following result proceeds in a different way than the proofs for parametric settings and bounds the probability PðN^opt ¼N_opt Þ using nonparametric techniques.

Theorem 2.2. Assume that Assumption (E) holds and Y1,Y2,::: are i.i.d. with EðY₁⁸Þ<1. Then the two-stage procedure given byN^_opt is second-order efficient; that is,

(10)

EðN^_optÞ N_opt ¼OðdÞ, as d#0:

2.2. Fixed-Width Interval under High-Confidence Asymptotics

Theorem 2.1 establishes the validity of the proposed sampling strategy for small-accuracy d; that is, in ahigh-accuracy framework: The (asymptotic) first-order properties hold if d tends toward zero. To some extent, this is counterintuitive because we aim at con- structing a fixed-width confidence interval and, for applications, we are then essentially limited to confidence statement whendis small.

In some applications, however, d may be not small (enough) but one aims to ensure the confidence statement that the interval covers the true parameter with high confidence. This suggests considering the case in which d is fixed and 1a tends toward 1 (or equivalently a!0). That type of asymptotics may be of particular importance in fields such as statistical genetics or brain research, where it is common to use very small significance levels (a).

Recalling formula (2.3) for the asymptotically optimal (unknown) sample size N_opt and noticing that, for fixed d N_opt ! 1 holds if 1a=2!1, again justifying the application of the CLT, the question arises as to whether consistency and efficiency can be established under this different asymptotic regime.

To begin, let us notice that the notions of consistency, asymptotically first- and second-order efficiency, and asymptotic coverage under high-confidence asymptotics can be defined analogously as under high-accuracy asymptotics by replacing the limits limd!0 by lim1a!1:

The following theorem asserts that the proposed methodology is valid without any modification ofN^_opt under high-confidence asymptotics, although the proof differs.

Theorem 2.3.

i. Assume that (E) holds. ThenN^_opt is consistent for N_opt ; that is, N^_opt

N_opt ¼1þo_Pð1Þ,

as1a!1, and asymptotically first-order efficient; that is, EðN^_optÞ

N_opt ¼1þoð1Þ, as1a!1:

ii. If Assumptions (E) and (A) hold, then I_N_^

opt has asymptotic coverage 1a; that is,

1lima!1 P I_N_^

opt像l

ð1aÞ

¼0:

Remark 2.2. The assertions of Theorem 2.3 also hold true if the first-stage sample size N0is defined as

(11)

N0 ¼max N0,

fU¹ð1a=2Þ d

_2=ð1þ_cÞ

( )

for some 0<c<1 and an arbitrary given constant f >0, as proposed in Mukhopadhyay (1980; withf¼1).

The question arises as to whether the procedure exhibits second-order efficiency under the high-accuracy regime as well. The answer is positive.

Theorem 2.4. Assume that Y1,Y2,::: are i.i.d. with EðY₁⁸Þ<1. Then the two-stage procedure given by N^_opt is second-order efficient. Precisely, we have

jEðN^_optÞ N_opt j ¼Oð1Þ, as1a!1:

2.3. Proofs

Proof of Theorem 2.1. First, notice that, by definition ofN^_opt, N^opt

^r²_N₀U¹ð1a=2Þ² d²

þ1^r²_N₀U¹ð1a=2Þ²

d² : (2.6)

It is easy to see that maxfbzc,ag zþa for all nonnegative real z and any positive constant a. Therefore, we have

N^_opt¼max N₀,

r^²N0U¹ð1a=2Þ² d²

þ1

( )

N₀þ^r²_N₀U¹ð1a=2Þ² d² þ1: Plugging in the definition ofN0, we further obtain

N^_opt max

^rl, 3U¹ð1a=2Þ d

,N₀

( )

þ^r²N0U¹ð1a=2Þ²

d² þ1

r^²N0U¹ð1a=2Þ²

d² þ^rl, 3U¹ð1a=2Þ

d þN₀þ1:

Combining the last estimate with(2.6), we arrive at r^²_N₀U¹ð1a=2Þ²

d² N^_opt r^²_N₀U¹ð1a=2Þ²

d² þ^rl, 3U¹ð1a=2Þ

d þN₀þ1, which implies, due to (2.3),

^r²N0

r²_l N^opt

N_opt r^²N0

r²_l þ d^rl, 3

U¹ð1a=2Þr²_lþ ðN₀þ1Þd²

r²_lU¹ð1a=2Þ²: (2.7)

(12)

We are led to N^opt

N_opt 1

^r²_N₀ r²_l 1

þ d^rl, 3

U¹ð1a=2Þr²_lþ ðN₀þ1Þd²

r²_lU¹ð1a=2Þ²: (2.8) Recalling thatA_nX_nB_n with A_n,B_n¼o_Pð1Þ impliesX_n¼o_Pð1Þ, because for any d>0 we have PðjXnj>dÞ PðAn dÞ þPðBn>dÞ ¼oð1Þ, as n! 1, the first assertion follows: By (2.8),N^_opt is consistent forN_opt if ^r²_N₀ is consistent forr²_l, as d#0:(2.7)also immediately yields

^r²N0

r²_l 1

N^opt

N_opt 1

,

which shows that “only if” holds as well. Next, taking expectations in (2.7) implies the result on the asymptotic unbiasedness. It remains to show that the fixed-width confidence interval has asymptotic coverage 1a:First, observe that

P I_N_^

opt像l

¼P d ffiffiffiffiffiffiffiffiffi N^_opt q

rl

ffiffiffiffiffiffiffiffiffi N^_opt

q ^lN^optl rl

d ffiffiffiffiffiffiffiffiffi N^_opt q

rl

0

@

1 A

¼P d ffiffiffiffiffiffiffiffi N_opt p

rl

ffiffiffiffiffiffiffiffiffi N_opt N^_opt s ffiffiffiffiffiffiffiffiffi

N^_opt

q l^N^optl rl

d ffiffiffiffiffiffiffiffi N_opt p

rl

0

@

1 A:

Define

H_dðxÞ ¼P

ffiffiffiffiffiffiffiffiffi N_opt Nôpt

s ffiffiffiffiffiffiffiffiffi N^_opt

q ^lN^optl rl

x 0

@

1

A, x2R:

By Assumption (A) and Slutzky’s lemma, we have supx2RjHdðxÞ UðxÞj !0,

as d#0: Now, by definition of Hd and N_opt and linearity of the function evaluationf7!fj^b_a¼fðbÞ fðaÞ(for fixed realsa,b2D) for a function defined onD R,

PðI_N_^_opt像lÞ ¼Hdj^U_U¹1^ð1ð1^a=2Þa=2Þ

¼ ðHdUþUÞj^U_U¹1^ð1ð1a=2Þ^a=2Þ

¼ ðH_dUÞj^U_U¹1^ð1a=2Þð1a=2Þþ ð1aÞ:

Therefore, we obtain P I_N_^_opt像l

ð1aÞ

2 sup

x2RjHdðxÞ UðxÞj !0,

asd#0, which completes the proof. w

Proof of Theorem 2.2. First observe that N₀! 1 if and only if d!0, so that we can show the result forN₀ ! 1:We use the refined basic inequality

^r²_N₀U¹ð1a=2Þ²

d² N^_optN₀1ðN^_opt ¼N₀Þ þr^²_N₀U¹ð1a=2Þ²

d² þ1,

(13)

which implies, in view of Assumption (E),

N_opt þoð1Þ EðN^_optÞ N0PðN^_opt¼N0Þ þN_opt þ1þoð1Þ:

This means

oð1Þ EðN^_optÞ N_opt N₀PðN^_opt¼N₀Þ þ1þoð1Þ, and the result follows if we show that

PðN^_opt ¼N₀Þ ¼OðN₀¹Þ:

We may assume that N0 is large enough to ensure that N₀¼ b^{^}^r^l^,³^U¹_d^ð1a=2Þc þ1:

Because by definition of N^_opt, N^_opt ¼N₀ ()

r^l, 3U¹ð1a=2Þ

d þ1

þ1

^rN0U¹ð1a=2Þ d²

þ1,

and using the elementary estimates bxc x1 and bxc þ1xþ1, if x0, we obtain

PðN^_opt ¼N_opt Þ P ^r²N0U¹ð1a=2Þ²

d² ^rl, 3U¹ð1a=2Þ

d þ1

!

¼P ^r²_N₀r² r²þ ^rl, 3

U¹ð1a=2Þþ d U¹ð1a=2Þ²

! d

! :

Because ð_U1^rð1a=2Þ^{^}^l^,³ þ_U1ð1^da=2Þ²Þd¼oð1Þ, asd#0, we obtain

PðN^_opt¼N_opt Þ ¼Pðr²r^²_N₀r²þoð1ÞÞ P r²^r²_N₀ r² 2

: But ifEðY₁⁸Þ<1, then

P r²r^²_N₀ r² 2

Eð^r²N0rÞ⁴

ðr²=2Þ⁴ ¼OðN₀²Þ,

see, for example, lemmas A.1 and A.2 in Steland and Chang (2019). Hence, the asser-

tion follows. w

Let us now prove the corresponding results under the high-confidence asymptotic framework.

Proof ofTheorem 2.3. Repeating the purely algebraic calculations from above we again obtain(2.8)for anyd:

N^_opt N_opt 1

r^²_N₀ r²_l 1

þ d^rl, 3

U¹ð1a=2Þþ ðN₀þ1Þd² r²_lU¹ð1a=2Þ:

Clearly, the second and third terms are o(1) if d is fixed and 1a!1()a#0: Further, noting that N0! 1 ifd is fixed and a#0, the first term isoPð1Þ if dis fixed and a#0: Taking expectations, similar arguments apply. Hence, (i) is shown. To establish (ii), first observe that if Na, a>0, is an arbitrary family of stopping times with N_a=a!^Pk2 ð0,1Þ, then N_a=ak!^P1, as a! 1, and therefore by Slutzky’s lemma and the RCLT,

(14)

ffiffiffiffiffi

pakl^Nal rl

!^dNð0, 1Þ,

as a! 1: Now consider N^opt as a family of stopping times parameterized by a¼ U¹ð1a=2Þ²: Obviously, a! 1 ()a#0: Then, using (i) and recalling (2.3), N^_opt=a!^Pk¼r²_l=d², as a! 1:Because dis fixed, we have 0<k<1: Consequently,

T_N_^

opt ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi U¹ð1a=2Þ²r²_l

d²

s ^lN^opt l rl

!^dNð0, 1Þ, (2.9)

asa! 1: BecauseTNôpt ¼Û¹^ð1a=2Þ_d ð^lNôptlÞ, it follows that P I_N_^

opt像l

¼Pðd^lN^optldÞ

¼P U¹ð1a=2Þ

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi U¹ð1a=2Þ²r²_l

d² s

l^N^ l rl

U¹ð1a=2Þ 0

@

1 A

¼H_aj^U_U¹1^ð1a=2Þð1a=2Þ,

where Ha denotes the distribution function of T_N_^

opt: By (2.9) and Polya’s theorem, we obtain

a!1lim sup

x2RjHaðxÞ UðxÞj ¼0: Now we can conclude that

P IN^opt像l

ð1aÞ

ðH aUÞ^U_U¹1^ð1ð1a=2Þ^a=2Þ 2 sup

x

jHaðxÞ UðxÞj !0, asa! 1, which completes the proof. w

Proof of Remark 2.2. The proof of the consistency needs a minor modification. Plugging in the new definition ofN₀, we obtain

N^_optmax

fU¹ð1a=2Þ d

_2=ð1þcÞ

,N₀

( )

þ^r²_N₀U¹ð1a=2Þ²

d² þ1

^r²N0U¹ð1a=2Þ²

d² þ fU¹ð1a=2Þ d

_2=ð1þcÞ

þN₀þ1:

Combining the last estimate with(2.6), we arrive at

^

r²_N₀U¹ð1a=2Þ²

d² N^opt^r²_N₀U¹ð1a=2Þ²

d² þ fU¹ð1a=2Þ d

_2=ð1þcÞ

þN0þ1, which implies, due to (2.3),

(15)

^ r²_N₀

r²_l N^_opt N_opt ^r²_N₀

r²_l þf^2=ð1þ^c^Þ

r²_l d^2c=ð1þcÞU¹ð1a=2Þ^2c=ð1þcÞþ ðN₀þ1Þd²

r²_lU¹ð1a=2Þ²: (2.7) We are led to

N^opt

N_opt 1

r^²N0

r²_l 1

þf^2=ð1þ^c^Þ

r²_l d^2c=ð1þ^c^ÞU¹ð1a=2Þ^2c=ð1þ^c^Þþ ðN₀þ1Þd² r²_lU¹ð1a=2Þ²:

(2.8) Because c>0, the second term is oPð1Þif 1a!1: ^w Proof of Theorem 2.4. Observing that N₀! 1 if and only if 1a!1 and OðN₀¹Þ ¼ OðU¹ð1a=2ÞÞ, the proof of Theorem 2.2 carries over and provides the bound

jEðN^_optÞ N_opt j N₀OðN₀²Þ þ1¼OðN₀¹Þ þ1:

w

3. APPLICATION TO A PROJECTION OF HIGH-DIMENSIONAL DATA

An interesting application is to study the construction of a fixed-width confidence interval for the mean projection of high-dimensional data. As an example, we may ask for how long we need to observe the asset returns of stocks associated to a portfolio given by portfolio vector wn in order to set up a ð1aÞ-confidence interval for the mean portfolio return w⁰_nlwith a precision d. Such uncertainty quantification of a mean projection also arises when projecting a data vector to reduce dimensionality, with calculat- ing a projection w⁰_nXn of a pn-dimensional data vectorXn being a common approach to handle multivariate data when the dimension of the observed vectors is large. Widely used methods are principal component analysis, where one projects onto eigenvectors of the (estimated) covariance matrix of the data, sparse principal component analysis yielding sparse projection vectors, or LASSO (least absolute shrinkage and selection operator) regressions. For the latter approach, recall that a LASSO regression determines a sparse weighting vector, the regression coefficients, such that the associated projection of the regressors provides a good explanation of the response variable.

Of particular interest is the situation in which the dimension gets larger when the sample size increases, in order to mimic the case of large dimension when relying on asymptotics where the sample size increases. Then the weighting vectors depend on the sample size. We show that the general assumptions established in the previous section apply under mild uniform integrability conditions onw⁰_nXn,n1:

3.1. Procedure

Suppose that we are given a series of random vectors of increasing dimension, such that at time instantn we have at our disposaln i.i.d. random vectors

Xn1,:::,Xnn ðl_n,RnÞ,

of dimension p¼p_n, where p_n! 1, as n! 1 is allowed. Here the notation nn ðl_n,RnÞ means that the random vector nn follows a distribution with mean vector l_n

(16)

and covariance matrix Rn: We are interested in an asymptotic fixed-width confidence interval for the asymptotic projected mean vector

h¼ lim

n!1hn, hn¼w⁰_nl_n,

that is, for a preassigned accuracy d>0 and a given confidence level 1a we aim at finding a sample size N such that the confidence interval ½TNd,T_Nþd has asymptotic confidence 1a,

PðTNdw⁰_Nl_NT_NþdÞ ¼1aþoð1Þ,

as d#0: Here, for any (generic) sample size n, Tn is an estimator of w⁰_nl: For simplicity, we consider the unbiased estimator T_n¼w⁰_nXn whereXn¼_n¹P_n

i¼1Xni:

The proposed two-stage procedure is as in the previous section: In the first stage, draw

N₀¼max N₀,

U¹ð1a=2Þ^rin

d

þ1

( )

: (3.1)

observations, where N₀ is a given minimal sample size and ^rin is an initial estimate of the standard deviation of the projections using independent pilot data, such as the three-observations rule estimator. Next, we estimate the variance of the projections from the first-stage sample by^r²N0 and calculate the final sample size

N^_opt ¼max N₀,

^r²N0U¹ð1a=2Þ² d²

þ1

( )

: (3.2)

In practice, one considers a certain number of variables, so we assume that these variables are observable for the relevant sample sizesN₀andN^_opt as well as that the projection vectors only have nonzero entries for those variables of interest. The mathematical framework allowing for an increasing dimension is used to justify the procedure when the number of variables (respectively dimension) is large compared to the sample sizes in use.

Let us assume that

n!1lim w⁰_nRnwn¼r²_w >0, jhhnj ¼oðn¹⁼²Þ (3.3) and forr2 f1, 2g,

sup

n1Eðjw⁰_nXn1j^r1ðjw⁰_nXn1j^k>cÞÞ !0, c! 1, k¼1,:::,r: (3.4) Assumption (3.3) is mild and rules out cases where the variance of the projection vanishes asymptotically. The uniform integrability required in(3.4)is a crucial technical condition to ensure that the weak law of large numbers and the CLT apply. In many cases the condition can be formulated in terms of the Xn: For that purpose, suppose additionally that the norms W_n¼ jjwnjj₂ are bounded by constant W<1: Then, using the simple fact that jw⁰_nXnj WjjXnjj₂, condition(3.4)holds, if theXn satisfy

(17)

sup

n1EðjjXn1jj^r₂1ðjjXn1jj^k₂>cÞÞ !0, c! 1, k¼1,:::,r: (3.5) A simple sufficient condition for (3.5)is the moment condition

sup

n1EjjXnjj^rþd₂ <1 (3.6)

for some d>0:

We need to verify Assumptions (A) and (E). The verification of the RCLT is some- what involved, because the projection statistic of interest is a weightedsum with weights depending on the sample size. DenoteT_n¼ ðTnhÞ=rw

Theorem 3.1. Suppose that (3.3) and (3.4) hold. If sa, a>0, is a sequence of integer- valued random variables with sa=a!k, in probability, for some 0<k<1, then the statistic Tn¼w⁰_nXn satisfies the RCLT; that is,

T_s_a!^d Nð0, 1Þ, as a! 1:

Define the estimator

^r²_n¼1 n

Xⁿ

i¼1

w⁰_nXniw⁰_nXn

₂

: (3.7)

The following result verifies that Assumptions (E) holds for this estimator under weak technical conditions.

Theorem 3.2. If(3.3), then

(i) ^r²_nw⁰_nRnwn!^P0and^r²_n!^Pr²_w, as n! 1:

(ii) Eð^r²_nÞ w⁰_nRnwn!0and Eð^r²_nÞ !r²_w, as n! 1:

The above two theorems imply that the proposed two-stage procedure is asymptotically consistent as well as first-order and second-order efficient.

3.2. Proofs

3.2.1. Preliminaries on the RCLT.According to the general theoretical results of the previous section, it suffices to establish Assumption (A), the validity of the RCLT, and Assumption (E) for the statistic of interest; that is, the projected data. As a preparation, let us briefly review Anscombe’s RCLT (see, e.g., Ghosh, Mukhopadhyay, and Sen 1997) and sufficient conditions. Consider a sequence X₁,X₂,:::of i.i.d. random variables with mean l and finite variance r²>0 and a family sa, a>0, of integer-valued random variables, often but not necessarily stopping times, with sa=a!c, as a! 1, for a finite constantc. The RCLT asserts that

S_s

a ¼S_s_affiffiffiffisal sa

p r !^dNð0, 1Þ,

(18)

asa! 1, where forn2N

S_n¼Xⁿ

i¼1

X_i, S_n¼Snffiffiffinl pn

r :

This means that the sample size n can be replaced by sa. The basic idea why this holds is as follows (cf. Ghosh, Mukhopadhyay, and Sen 1997): The approximation sa

ac suggests thatS_s

a should have the same limiting distribution as S_n

a, where n_a¼ bacc:

The CLT gives S_n_a !Nð0, 1Þ, as a! 1, in distribution. Now, for any e>0 and d>0,

PðjS_s_aS_n

aj eÞ P sa

n_a1

>d

!

þP max

jnnajdna

jS_nS_n

aj e

, (3.8)

because in the event jsa=na1j d we have jS_s

aS_n

aj max_jnn_a_jdn_ajS_nS_n

aj: The second term on the right-hand side of (3.8) can be made arbitrarily small if the sequence fS_n:n1g is uniformly continuous in probability (u.c.i.p.). In general, a sequence fYn:n1g is called u.c.i.p. if for any e>0 there exists some d>0 so that for large enough n,

P max

0knd

jYnþkYnj e

<e:

The u.c.i.p. property of S_n can be shown using Kolmogorov’s maximal inequality, because S_n is a sum of i.i.d. random variables.

3.3. Proofs

Let us first show the sufficiency of (3.6)for(3.5). Observe that

EðjjXn1jj^r₂1ðjjXn1jj^k₂>cÞÞ ¼E jjXn1jj^rþd₂ jjXnjj^d₂ 1ðjjXnjj^d₂ <c^1=qkÞ

c^1=qkEjjXnjj^rþd₂ :

As a preparation for the following proofs, observe that we may choosec large enough to ensure that

Eðw⁰_nXn1Þ²c²þsup

n1Eððw⁰_nXn1Þ²1ðjw⁰_nXn1j>cÞÞ 2c² (3.9) holds for all n1:Therefore, the sequence of second moments of w⁰_nXn1 is bounded.

This implies

w⁰_nRnwn¼Var ðw⁰_nXn1Þ ¼Oð1Þ, (3.10)

Ejw⁰_nXn1j ¼Oð1Þ: (3.11)

For technical reasons, we first show Theorem 3.2.

Proof of Theorem 3.2. Observe that the random variables Z_ni¼ ðw⁰_nXniÞ²,i¼1,:::,n, are independent and nonnegative with mean EðZn1Þ ¼w⁰_nMnwn, where Mn1¼EðXn1X⁰_n1Þ:

Hence, by the strong law of large numbers for arrays due to Gut (1992) under the uniform integrability condition (e) given there,