Asymptotic Statistical Theory for Long Memory Volatility Models

(1)

Asymptotic Statistical Theory for Long Memory Volatility Models

Dissertation

zur Erlangung des akademischen Grades

des Doktors der Naturwissenschaften (Dr. rer. nat.) an der

Universit¨ at Konstanz

Mathematische-Naturwissenschaftliche Sektion Fachbereich Mathematik und Statistik

vorgelegt von

Martin Sch¨ utzner

Referenten:

Prof. Dr. Jan Beran, Universit¨at Konstanz Prof. Dr. Siegfried Heiler, Universit¨at Konstanz

Tag der m¨ undlichen Pr¨ ufung: 16. Juni 2009

Konstanzer Online-Publikations-System (KOPS) URN: http://nbn-resolving.de/urn:nbn:de:bsz:352-opus-82995

URL: http://kops.ub.uni-konstanz.de/volltexte/2009/8299/

(2)

To Bianca

(3)

Acknowledgment

This work was financially supported by the German Research Foundation (DFG).

I am deeply grateful to my supervisor Jan Beran for his guidance and for the in- teresting and demanding topic of this thesis which turned out to be very fruitful and surprisingly manifold. Moreover, I would like to thank him for many ideas, perfect working conditions and the opportunity to travel to many international conferences. My special thanks also go to Siegfried Heiler for his support since the beginning of my studies and for refereeing this PhD thesis.

Konstanz, June 2009 Martin Sch¨utzner

(4)

Abstract

In this thesis, statistical theory for time series with conditional heteroskedasticity and long memory in volatility is studied. We present appropriate models and consider several problems regarding parametric estimation. First, we discuss the question whether the asymptotic properties of M-estimators of location are affected by slowly decaying autocorrelations in squares. It turns out that under certain symmetry assumptions, consistency and the usual central limit theorem still hold. On the other hand, deviations from these assumptions can lead to non-standard behavior, in particular non-gaussian limiting distributions. For the asymptotic analysis, a connection to Appell polynomials and linear long memory processes is derived. Furthermore, we focus on the parametric LARCH model and investigate a modified conditional maximum likelihood estimator. Consis- tency and asymptotic normality are derived. The proofs differ substantially from the case of related models such as ARCH(∞), since the volatility of a LARCH process is not separated from zero. Moreover, the long memory property leads to additional difficulties, for instance a slower rate of convergence. Consequently, we discuss the question how more efficient estimators can be defined for models with slowly decaying autocorrelations. Therefore, we exploit the result that long memory can be explained by contemporaneous aggregation. Based on a panel scheme of random AR(1) processes, a new estimator for the long memory parameter of the aggregated process is introduced and asymptotic properties are proven. The results indicate that the described procedure could lead to improved statistical methods, in particular for heteroskedastic models with long memory.

(5)

Zusammenfassung

In der vorliegenden Arbeit werden statistische Methoden für Zeitreihen mit be- dingter Heteroskedastizität und langfristigen Abhängigkeiten in der Volatilität untersucht. Es werden entsprechende stochastische Prozesse vorgestellt und ver- schiedene Probleme im Zusammenhang mit Parameterschätzung behandelt. Zu- nächst studieren wir die Frage, ob langsam fallende Autokorrelationen die asymp- totischen Eigenschaften von M-Schätzern für Lokationsparameter beeinflussen können. Es stellt sich heraus, dass dies nicht der Fall ist, wenn bestimmte Symmetriebedingungen erfüllt sind. Andererseits ergibt sich bei Abweichungen von diesen Bedingungen ein unterschiedliches Verhalten, insbesondere muss die Grenzverteilung nicht mehr normalverteilt sein. Für die Herleitung der asymp- totischen Ergebnisse wird ein Zusammenhang zu Appell-Polynomen und linearen Prozessen mit langfristigen Abhängigkeiten hergeleitet. Desweiteren konzentri- eren wir uns auf das parametrische LARCH-Modell und untersuchen bedingte Maximum-Likelihood-Schätzer. Konsistenz und asymptotische Normalität werden gezeigt, wobei sich die Beweise wesentlich von den Methoden für ähnliche Modelle wie ARCH(∞) unterscheiden. Der Grund dafür liegt hauptsächlich in der Tatsache, dass die Volatilität eines LARCH-Prozesses beliebig klein werden kann.

Ferner führen die langfristigen Abhängigkeiten zu weiteren Schwierigkeiten wie etwa einer langsameren Konvergenzrate. Dadurch stellt sich die Frage, wie sich ef- fizientere Methoden für solche Zeitreihen definieren lassen. Dabei greifen wir das Ergebnis auf, dass langfristige Abhängigkeiten durch Aggregation einfacherer Ba- sisprozesse erzeugt werden können, und betrachten ein Panel-Schema mit AR(1)- Prozessen und zufälligen Koeffizienten. In diesem Modell wird basierend auf Schätzern für die Basisprozesse eine neue Schätzmethode für den aggregierten Prozess eingeführt. Die Ergebnisse zeigen, dass das beschriebene Vorgehen zu ef- fizienteren Schätzern führen kann, insbesondere bei heteroskedastischen Modellen mit langfristigen Abhängigkeiten.

(6)

Chapter 1 Introduction

Many financial time series such as log-returns of asset prices or exchange rates seem to be stationary and uncorrelated. On the other hand, they simultaneously possess time-varying volatility and strong dependencies measured by the autocovariances of some non-linear transformation such as the absolute values or squares.

To model this kind of behavior, Engle (1982) proposed in his path-breaking paper the autoregressive conditional heteroskedastic (ARCH) process. Since then an enormous number of related models have been introduced in financial time series analysis and have become essential tools in practice, for instance in prediction, risk assessment, risk management, option pricing and portfolio management. The innovative feature of ARCH is conditional heteroskedasticity, i.e. time varying (and stochastic) conditional variance, which improved econometric methods in such a way that Engle was awarded in 2003 with the Nobel Memorial Prize in Economic Sciences. In its simplest form the ARCH(1) process can be written as

Xt=εtσt, σt=β0+β1Xt−1, t ∈Z,

where β0 > 0, 0 ≤ β1 < 1 are model parameters and the so-called innovations (εt)t∈Z are an i.i.d. sequence with zero means and unit variances. Thus, for a causal stationary solution (in the sense that Xt is measurable with respect to F^t =σ{εs, s≤t}), one gets

E[Xt|Ft−1] = 0, var(Xt|Ft−1) =σ_t² and further

cov(X_t, X_s) = 0, cov(X_t², X_s²) =β₁^|t−s|var(X₀²)6= 0. (1.1) 3

(9)

CHAPTER 1. INTRODUCTION 4

1995 2000 2005

−0.10−0.050.000.05

DAX Nov. 1990 − June 2008, daily log−returns

time

diff(log(DAX))

1995 2000 2005

0.0000.0020.0040.0060.0080.010

squared log−returns

time

diff(log(DAX))^2

2e−04 1e−03 5e−03 2e−02 1e−01

1e−101e−081e−06

periodogram of squared log−returns (log−log scale)

freq

spec

0 5 10 15 20 25 30 35

0.00.20.40.60.81.0

Lag

ACF

acf of log−returns

0 10 20 30 40

0.00.20.40.60.81.0

Lag

ACF

acf of squared log−returns

5 10 20 50 100 200 500

2e−083e−084e−086e−088e−08

variance plot for squared log−returns

sample size

variance

slope=−0.287 slope=−1

Figure 1.1: Daily log-returns of DAX from November 1990 until June 2008: series, acf, squared series, acf of squares, log-log-plot of periodogram and variance plot in log-log-scale. The latter has been generated by dividing the whole series into subseries of decreasing lengths whereas for each length the sample variance of the means has been calculated.

Obviously, the conditional standard deviation sd(Xt|Ft−1) =σt (named volatility) is time varying and together with (1.1) the basic properties of financial time series are captured by ARCH. Bollerslev (1986) introduced thegeneralized ARCH (GARCH(p, q)) model given by the equations

X_t =ε_tσ_t, σ²_t =β₀+ Xp

i=1

α_iσ²_t₋_i+ Xq

j=1

β_jX_t²₋_j, (1.2) where β0 >0,αi, βj ≥0 andp, q ≥0. Though GARCH allows for rather flexible and parsimonious modelling, it can be shown (see section 3.1), that the autocovariances of the squares, if they exist, decay exponentially fast analogously to (1.1). This is in contrast to several studies which investigate the observation, that often empirical autocorrelations in squared log-returns persist over long stretches of time (see Dacorogna et al. 1993, Ding et al. 1993, Baillie et al. 1996, Breidt et al. 1998, Liu 2000, Andersen et al. 2001, Beran and Ocker 2001, Mikosch and Starica 2003). Compare, for illustration, the correlogram of log-returns and

(10)

CHAPTER 1. INTRODUCTION 5 squared log-returns of the DAX stock index series in figure 1.1: While there is no apparent autocorrelation in the original series, the autocorrelations of the squares decay very slowly, in particular much slower than exponentially decreasing autocorrelations of squared GARCH processes. Further, the log-log-plot of the periodogram exhibits a negative slope near the origin while the variance plot shows a similar behavior, both indicating long memory in squares. By long memory, we mean that the autocovariances of a second order stationary process are not absolutely summable. For instance, let

cov(X_t², X_t+k² )∼bk^2d−1 as k → ∞ (1.3) with constants b >0 and d >0, then

X

k∈Z

|cov(X_t², X_t+k² )|=∞. (1.4) During the last 15 years or so, modifications of ARCH and GARCH have been introduced to include the possibility of slowly decaying autocovariances in squares or volatility, so-called long memory volatility models. Examples are ARCH(∞), FIGARCH, FIEGARCH, LARCH, quadratic ARCH(∞) and stochastic volatility models (LM-SV) - though, as it turned out, not all of these models exhibit long memory in the sense (1.4), whereas their correlations may decay hyperbolically.

Note that (1.3) with d ∈ 0,¹₂

(a value which is empirically often observed in financial data) leads to 2d−1∈(−1,0) and thus to long memory (1.4), whereas d <0 leads to summable autocovariances.

The problem of statistical inference for short memory volatility models is exten- sively studied, see e.g. Straumann (2004). On the other side, statistical analysis for long memory models is still a topic with many open research problems.

This thesis mainly studies three of such problems. First, we consider sequences Y_t=µ+X_t, whereX_t=ε_tσ_t is a volatility model satisfying (1.3) without further parametric specifications. The question is then, whether there is an effect of (1.3) on M-estimators of the location parameter µ, compared to the case of indepen- dence or short memory inX_t². It turns out that the answer essentially depends on symmetry of the innovations εt. While symmetric εt imply the usual asymptotic properties of M-estimators, asymmetry can lead to non-standard convergence rates, non-gaussian asymptotic distributions and thus to different confidence in- tervals and tests. In this context, a connection to Appell polynomials and limit

(11)

CHAPTER 1. INTRODUCTION 6 theorems for linear long memory processes is derived.

Secondly, we focus on the parametric LARCH process, which has been introduced by Robinson (1991) to allow for long memory in volatility and asymmetries such as the leverage effect. The latter describes the empirically observed property, that volatility is more sensitive to falling markets than to raising markets, which can not be replicated by the classical GARCH model whereσtis a function of absolute past returns. Though theoretical properties of LARCH have been investigated in the recent literature, there is a lack of estimation methods and corresponding asymptotic theory. We show in this thesis, that standard methods lead to difficulties and propose modified parameter estimators for which consistency and asymptotic normality are derived. As a result, the usual √

n-normalization still holds for short memory versions of the model, whereas for general LARCH processes the rate of convergence is affected by the long memory property. There are two reasons for the emphasis on LARCH. On the one hand, the long memory property can be derived easily. Thus, the probabilistic aspects can be treated rigorously. On the other hand, there are no latent processes in the definition of LARCH and direct estimation of the model parameters is possible. Compare e.g.

to stochastic volatility models, where long memory is generated by an unobserv- able process.

The findings lead to the question how more efficient estimators can be defined for LARCH processes in particular, and for long memory models in general. Within this context, we should recall, that a possible explanation of long memory is given by contemporaneous aggregation in the sense of summing or averaging across different micro-level short memory processes. It is then possible that the aggregated process has long memory, whereas the long memory parameter is characterized exclusively by the distribution of the parameters of the micro-level processes.

Thus, inference for the long memory parameter can be based on estimators for the short memory processes, which often have a faster rate of convergence compared to corresponding estimators for long memory models, e.g. as it is the case for LARCH. We will investigate the basic problem, whether the described procedure works in the elementary case of aggregating random AR(1) processes. In a panel set-up, where the AR(1)-parameters have a Beta-type distribution, we introduce the new parameter estimator for the (possibly) long memory aggregated process and derive the main asymptotic properties.

Additionally to the mentioned statistical problems, we present related models and

(12)

CHAPTER 1. INTRODUCTION 7 results as prerequisites or supplements to our study. The thesis is structured as follows: In chapter 2we state central and non-central limit theorems for gaussian and linear long memory processes. Moreover, we introduce Hermite respectively Appell polynomials, whereas the connection of the latter to limit theorems is described in detail. The results will be used in chapter 4. In chapter 3 we give a brief overview of available volatility models and discuss the existence of long memory in the different processes. The problem of M-estimation is studied in chapter 4, where we also present a standard method for parametric estimation of ARCH(∞) processes, the conditional maximum likelihood estimator. Chapter 5 gives an introduction to the LARCH model. Subsequently, chapter 6 considers the asymptotic study of the modified maximum likelihood estimator together with a result regardingM-estimation for LARCH processes. Finally, the problem of aggregation and estimation is described in chapter 7. The thesis is completed with concluding remarks in chapter8, a bibliography and an appendix containing the R program code used for the simulations in section 6.2.4.

(13)

CHAPTER 1. INTRODUCTION 8

(14)

Chapter 2 Long memory and limit theorems

In this section, we collect some well-known results that will be used later in this thesis. The main focus lies on the presentation of limit theorems for sequences of long memory processes. After introducing some standard notations in section 2.1, we will consider in section 2.2 the case of functionals of gaussian processes, where the asymptotic behavior of partial sums essentially depends on the connection to Hermite polynomials, and further we will present the generalizations for linear processes and corresponding Appell polynomials. The results are used, for instance, in section 4.1.2 for the derivation of similar limit theorems in the context of volatility models. The necessary theory of Appell polynomials, which can be used for the proof of the central limit theorem for linear long memory processes, is described in section 2.3. There, we also outline the problem of Ap- pell polynomial expansions of entire functions, which will also be used in section 4.1.2. For more detailed descriptions of long memory we refer to Beran (1994), Doukhan et al. (2003) and Robinson (2003).

2.1 Basic concepts

We start with a definition what we mean by long memory:

Definition 2.1 Let (Xt)t∈Z be a stochastic process.

(a) (Xt)t∈Z is called strictly stationary, if for all (t1, . . . , tk)^T ∈ Z^k, k ≥1, the joint distributions of (Xt1+t, . . . , Xtk+t) do not depend on t ∈Z.

(b) Define µ(t) = E[Xt] and γ(t, s) = cov(Xt, Xs). Then, (Xt)t∈Z is called second-order stationary, if E[X_t²] < ∞ for all t ∈ Z, µ(·) is constant and

9

(15)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 10 γ(t, s) only depends on |t−s|. In this case, denote

γX(k) :=γ(0, k), called to autocovariance function of (Xt)t∈Z.

(c) A second-order stationary process (Xt)t∈Z has long memory (or long range dependence), if

X

k∈Z

|γ(k)|=∞. Otherwise, (Xt)t∈Z has short memory.

As example, we consider a linear process exhibiting long memory. Therefore, let ξt, t = 1,2, . . .be an i.i.d. sequence of random variables withE[ξt] = 0,E[ξ_t²]<∞ and coefficients

bj =cj^d⁻¹, j = 1,2, . . . , where c >0 and d∈ 0,¹₂

. Then, define the linear process Xt=

X∞ j=1

bjξt−j, t∈Z.

Thus, the autocovariances are given by E[X₀X_k] =

X∞ j=1

b_jb_j+k =c² X∞

j=1

j^d⁻¹(j+k)^d⁻¹

= k^2d⁻¹c² X∞

j=1

j k

d−1 1 + j

k d−1

1

k =:k^2d⁻¹Λ(k).

Note that the following truncated sum is a Riemann approximation for the corresponding integral,

[N k]

X

j=1

j k

d−1 1 + j

k d−1

1 k →

Z N 0

x^d−1(1 +x)^d−1dx, as k → ∞, where convergence is uniform in N >0. This leads to

t→∞lim lim

N→∞

[N k]

X

j=1

j k

d−1 1 + j

k

d−1 1

k = lim

N→∞ lim

t→∞

[N k]

X

j=1

j k

d−1 1 + j

k

d−1 1 k

= Z _∞

0

x^d⁻¹(1 +x)^d⁻¹dx <∞.

(16)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 11 In particular, Λ(·) is a slowly varying function, in the sense that Λ(·) is positive with lim_j→∞ ^Λ(aj)_Λ(j) = 1 for alla >0. Hence, by Karamata’s theorem,

XM k=1

γX(k) = XM k=1

Λ(k)k^2d−1 ∼Λ(M) 1

2dM^2d → ∞ as M → ∞.

Here, ”∼” means that the ratio of the expressions on the left and right hand side converges to one. Consequently, Xt has long memory.

Finally, we introduce some notations which will be used throughout this thesis:

Let (Yn)n≥1 and Y be random variables. Then Yn

→d Y, Yn

→p Y, Yn L²

→ Y stand as usual for convergence in distribution, in probability respectively in the L²-norm. Moreover, weak convergence in the space D[0,1] equipped with the Skorohod metric of a continuous time process Yn(u), u∈ [0,1] to Y(u), u∈[0,1]

is denoted by Yn(u)→^D Y(u). Incidentally, the Landau symbols o(·) andO(·) are used.

2.2 Limit theorems

The asymptotic behavior of partial sumsPn

t=1Y_tfor stationary processes Y_twith linear long-range dependence is well known (see e.g. Rosenblatt 1961, Taqqu 1975, 1979, Dobrushin and Major 1979, Surgailis 1981, 1982, Giraitis 1983, 1985, Giraitis and Surgailis 1985, 1986, 1999, Avram and Taqqu 1987, Dehling and Taqqu 1989, Arcones and Yu 1994, Ho and Hsing 1996, 1997). One should already mention that asymptotic theory for processes that exhibit long-range dependence in volatility, which will be defined later, is much less developed. Two recent references are, for instance, Berkes and Horv´ath (2003) and Giraitis et al. (2000b), where certain limit theorems for LARCH processes are derived. The results of the latter two papers are summarized in section 5.4.

Here, we will describe limit theorems for Yt = f(Xt), where (i) Xt is gaussian and f admits a Hermite expansion and (ii) Xt is linear and f admits an Appell expansion. Start with case (i) and let (Xt)t∈Z be a stationary gaussian process with E[Xt] = 0, E[X_t²] = 1 and covariances such that

γX(t) =E[X0Xt]∼t⁻^qΛ(t), t→ ∞ (2.1) for some 0 < q <1 and a slowly varying function Λ(t) (in fact Λ may take negative values as well). For a function H with E[H(X₀)] = 0 and E[H(X₀)²] <∞, we

(17)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 12 derive the asymptotic behavior of

Sn,H(u) :=

X[nu]

t=1

H(Xt).

(We will also use the notation Sn,H :=Sn,H(1).) Therefore, define Hermite polynomials as follows:

Definition 2.2 For m≥0, the Hermite polynomials H_m are defined by H_m(x) =e^x²^/2 d^m

dx^me^−x²^/2.

Since (Hm)_m≥0 constitutes a complete orthogonal system in the space{H :R→ R : E[H(X0)] = 0, E[H²(X0)] < ∞}, see e.g. Abramowitz and Stegun (1972), the following expansion holds:

H(Xt) = X∞ m=0

cmHm(Xt), where P_∞

m=0c²_mm!<∞, that means the series converges in L² since the variance of Hm(Xt) is given by var(Hm(Xt)) =m!(γX(0))^m, see (2.2). Moreover, Hermite polynomials are uncorrelated and thus the coefficients can be calculated by cm =

1

m!E[H(X0)Hm(X0)]. The lowest integer m_∗ ∈N with cm∗ 6= 0 is then called the Hermite rank ofH. DenoteγH(t) the covariance function ofH(Xt) and note that the (cross-)covariances of Hermite polynomials are given by

E[Hk(X0)Hj(Xt)] =δk,jk!(γX(t))^k. (2.2) (The latter can be shown, e.g., by the diagram formula in theorem 2.5 below.) Thus

γH(t) =

X∞ n,m=m∗

cncmE[Hn(Xt)Hm(X0)]

= γX(t)^m^∗ X∞ m=m∗

c²_mm!γX(t)^m⁻^m^∗. (2.3) For m_∗ >1/q, the covariances of Hm∗(Xt) are absolutely summable, since

X∞ t=1

|γX(t)|^m^∗ = X∞

t=1

t⁻^qm^∗Λ^m^∗(t)<∞.

(18)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 13 Hence, (2.3) leads to

X∞ t=1

|γH(t)| ≤c X∞

t=1

|γX(t)|^m^∗, where the constant c is such that c ≥ P_∞

m=m∗c²_mm!γ_X(t)^m⁻^m^∗ for all t ∈ Z (therefore note that γX(t) →0), i.e. one gets summability of the covariances of the process H(Xt).

The other way around, (2.3) and γX(t)→0 imply

|γH(t)| ≥ |γX(t)|^m^∗c²_m_∗m_∗!/2, for t large. Consequently, we have the equivalence

X∞ t=0

|γ_H(t)|<∞ ⇐⇒

X∞ t=0

|γ_X(t)|^m^∗ <∞,

meaning that a function H with Hermite rank m_∗ >1/q leads to short memory in the process H(Xt). Indeed, Giraitis and Surgailis (1985) proved the following theorem:

Theorem 2.1 Let(Xt)t∈Z be a zero mean gaussian process for which (2.1) holds.

If P_∞

t=0|γ_H(t)|<∞ (i.e. m_∗ >1/q) and σ² :=P_∞

t=0γ_H(t)>0, then n^−1/2S_n,H(u)→^D σB(u),

where B(u) is a standard Brownian motion.

Next, we consider the case m_∗ <1/q. First, we derive the order of growth of the variance ofSn,m:=Sn,Hmin the simple caseγ(t) =ct⁻^q for a constant 0< c <∞:

var(Sn,m) = nvar(Hm(X0)) + 2n

n−1

X

t=1

1− t

n

E[Hm(X0)Hm(Xt)]

= nvar(H_m(X₀)) + 2cn

n−1

X

t=1

1− t

n

m!γ_X(t)^m

= nvar(Hm(X0)) + 2cn

n−1

X

t=1

m!t⁻^qm−2cn

n−1

X

t=1

t

nm!t⁻^qm. (2.4) Thus, for n→ ∞,

n−1

X

t=0

t⁻^qm =n¹⁻^qm

n−1

X

t=0

t n

−qm

1

n ∼n¹⁻^qm Z 1

0

x⁻^qmdx,

(19)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 14 since the sum is a Riemann approximation of the latter integral. Analogously, we get for the latter term in (2.4) that

Xn−1 t=1

t¹⁻^qm1

n =n¹⁻^qm Xn−1

t=1

t n

1−qm

1

n ∼n¹⁻^qm Z 1

0

x¹⁻^qmdx,

and hence limn→∞var(Sn,m)/n²⁻^qm = const. Consequently, since 2−qm_∗ > 1, the usual central limit theorem with standard √

n-scaling can not hold for sums of Hm∗(Xt). Indeed, even a non-normal limiting distribution can arise and the same is true for Sn,H(u), as the next theorem, which goes back to Taqqu (1979) and Dobrushin and Major (1979), states.

Theorem 2.2 Let m_∗ be the Hermite rank of H und (Xt)_t∈^Z a zero-mean gaussian process for which (2.1) holds with 0 < q < 1/m_∗. Further denote An = n¹⁻^m^∗^q/2Λ^m^∗^/2(n), then

1 An

Sn,H(u) = 1 An

X[nu]

t=1

H(Xt)→^D cm∗Hm∗(u), as n → ∞, where cm∗ is the m_∗-th coefficient in the Hermite expansion of H.

Here, the Hermite process H^k(u) of orderk ≥1 is defined by H^k(u) = Cq

Z _′

R^k

Z u 0

Yk j=1

(s−yj)⁻^(1+q)/2

!

dsdB(y1)· · ·dB(yk),

where B(·) is a standard Brownian motion and C_q is a positive constant (see e.g.

Taqqu 1979). Fork = 1,Hk(u) is fractional Brownian motion, and thus gaussian, while Hk(u) has non-normal marginal distributions for k ≥2.

The preceding theorem can be understood as a reduction principle, in the sense that the asymptotic properties of Sn,H(u) only depend on the Hermite rank m_∗ of H. In particular, the sum ofH(Xt) and the sum ofcm∗Hm∗(Xt) have the same limiting distribution.

Giraitis (1985), Surgailis (1982) and Avram and Taqqu (1987) showed that this reduction principle, together with the central limit theorem2.1, can be generalized for linear processes (our case (ii)), which will be described in the following. Let Xt be given by

Xt=X

s∈Z

b(t+s)ξs, t ∈Z (2.5)

(20)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 15 with coefficients

b(t) = Λ(|t|)|t|^d⁻¹, t∈Z, (2.6) where Λ is a slowly varying function and d ∈ (0,¹₂). (Here, the reason for the two-sided moving average representation of X_t is a more convenient notation in section 2.3.2.) Moreover, ξs, s ∈ Z are i.i.d. with E[ξs] = 0, E[ξ_s²] = 1 and E[|ξs|^k] < ∞ for all k ≥ 0. The covariance function is then given by γX(t) = E[X0Xt] = Λ1(t)t^2d⁻¹. Due to the more general marginal distributions of Xt, the Hermite polynomials have to be replaced by Appell polynomials defined as follows:

Definition 2.3 Let X be a random variable with finite moments up to order M, i.e. E[|X|^M] < ∞. Then, the corresponding Appell polynomials Am are defined by A0(X) = 1 and for m= 1, . . . , M recursively by

d

dxAm(x) =mAm−1(x), E[Am(X)] = δ0,m.

In the next section, we will give a more detailed description of the polynomials Am. However, one immediately sees that eachAm is of orderm form≥0. Thus, every polynomial Gof order M can be uniquely expanded as

G(Xt) = XM m=0

cmAm(Xt).

Correspondingly to the gaussian case, the lowest m with cm 6= 0 is called Appell rank and is denoted by m_∗. Write Sn,G(u) := P[nu]

t=1 G(Xt) and let γG(t) be the covariance function ofG(Xt). Then, the following theorem, due to Giraitis (1985), is the (partial) analogon to theorem 2.1.

Theorem 2.3 Let (2.5) and (2.6) hold and denotem_∗ the Appell rank of G. For m_∗ ≥2, the conditions

X∞ t=0

|γ_G(t)|<∞ and X∞

t=0

|γ(t)|^m^∗ <∞ (2.7) are equivalent. Moreover, if one of them holds and σ² :=P_∞

t=0γG(t)>0, then n⁻^1/2Sn,G(u)→^D σB(u). (2.8)

(21)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 16 Hence, if m_∗ > ₍₁₋¹_2d), (2.7) is fulfilled and the central limit theorem holds. The next theorem considers the case m_∗ < ₍₁₋¹_2d). The result was primarily proven by Surgailis (1982), whereas the connection to Appell polynomials was carried out in Avram and Taqqu (1987), see also Surgailis (2003).

Theorem 2.4 For1≤m <1/(1−2d), letAmdenote them-th Appell polynomial corresponding to the linear process (2.5) and define

Sn,Am(u) :=

X[nu]

t=1

Am(Xt).

Then, B_n,m² :=E[S_n,A² _m(1)]∼bmΛ^m(n)n^{2−m(1−2d)} with a constant bm >0 and B_n,m⁻¹ Sn,Am(u)→ H^D ^m(u),

as n tends to infinity.

Thus, given a polynomial function G with Appell rank m_∗ < (1−2d)⁻¹, one gets var(Sn,Am(u)) = o(B_n,m² _∗) for m > m_∗ and the leading term in the Appell expansion of G(Xt) is cm∗Am∗(Xt). Therefore, the preceding non-central limit theorem also holds for G(Xt).

Finally, note that we only considered polynomials G until now since the Appell expansion of more general functions is rather complicated, see the next section.

However, one should mention that Giraitis (1985) also proved that an expansion G(Xt) =

X∞ m=m∗

cmAm(Xt)

still holds for entire functionsGsatisfying some growth condition (see theorem2.7 below) by which theorem 2.3 can be proven forG(X_t) ifm_∗ >(1−2d)⁻¹. Thus, if m_∗ < (1−2d)⁻¹, theorem 2.4 can also be applied to G(Xt) by decomposing G=G1+G2, where G1 is a polynomial with Appell rankm⁽¹⁾_∗ <(1−2d)⁻¹ and G2 is an entire function with rank m⁽²⁾_∗ >(1−2d)⁻¹. Then, theorem 2.4 can be applied to G1(Xt), while the variance of Sn,G2(u) is of order O(n), and hence of smaller order than var(Sn,G1(u)), leading to asymptotic negligibility of Sn,G2(u).

2.3 Appell polynomials

For detailed descriptions of Appell polynomials and their role in the context of limit theorems for linear processes see Avram and Taqqu (1987), Giraitis and

(22)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 17 Surgailis (1986), Giraitis (1985), Surgailis (2003) and Sch¨utzner (2006). Here, we will give the definition and some examples in section 2.3.1, the connection to cumulants (the so-called diagram formula) and the application to the central limit theorem in section 2.3.2, and describe the problem of Appell polynomials expansions in section 2.3.3.

2.3.1 Definition and examples

We start with an alternative definition of Appell polynomials: First assume that the moment-generating function MX(z) = E[e^zX] exists for |z| < r with r > 0.

Then MX(0) = 1 and 1/MX(z) is analytic in a neighborhood of 0 leading to a power series in the variable z for exp(zx)/MX(z) with fixed x∈R.

Definition 2.4 Let X be a random variable with MX(z) = E[e^zX] < ∞ for

|z|< r. Then the Appell polynomials A_m(x) are defined by X∞

m=0

z^m

m!A_m(x) = exp(zx)

E[e^zX] . (2.9)

The function ^exp(zx)_E[ezX] is called generating function of (Am)m≥0.

Note that for this definition, the moment-generating function actually need not exist as analytical object. By matching of coefficients with respect to the generating function

exp(zx) PM

m=0 z^m

m!E[X^m],

it suffices that moments of X up to order M ≤ ∞ are finite. Consequently, if M <∞, Appell polynomials Am can only be defined for m≤M.

For the first derivative _dx^dAm(x) =A^′_m(x), we have X∞

m=1

z^m

m!A^′_m(x) = d dx

exp(zx)

E[e^zX] = zexp(zx) E[e^zX] =

X∞ m=0

z^m+1

(m+ 1)!(m+ 1)Am(x).

Thus

A^′_m(x) =mAm−1(x). (2.10)

Moreover, by applying the expected value in (2.9), we get X∞

m=0

z^m

m!E[Am(X)] =E

exp(zX) E[e^zX]

= 1,

(23)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 18 yielding

E[Am(X)] = δ0,m. (2.11)

However note that the justification of this argument relies on converge of the series in L²(X) (or at least in L¹(X)). On the other hand, (2.11) can also be shown by combinatorial arguments, see e.g. Sch¨utzner (2006) where also equivalence of definitions 2.4 and 2.3 is derived.

We will now calculate (Am)m≥0 in two examples. First, let X be distributed according to an exponential distribution with density

f(x) =λe⁻^λx, x >0, λ >0.

Then M_X(z) is given by

MX(z) = 1

1−λ⁻¹z, |z|< λ.

Thus we have to compare the coefficients in the following equation:

X∞ m=0

z^m

m!Am(x) = exp(zx)(1−λ⁻¹z) = X∞ m=0

z^m m!x^m−

X∞ m=0

z^m+1 λm!x^m

= 1 + X∞ m=1

z^m m!

x^m− 1

λmx^m−1

. Hence,

A₀(X) = 1

Am(X) = X^m− 1

λmX^m⁻¹, m≥1.

Observe, that E[X^m] = _λ^m!m impliesE[Xⁿ−_λ¹nXⁿ⁻¹] = 0 and thus definition 2.3 is also fulfilled.

Next, we show that Appell polynomials are indeed a generalization of Hermite polynomials. Therefore, let X be standard-normally distributed leading to the moment generating function MX(z) = exp(z²/2), and consider

X∞ m=0

z^m

m!Am(x) = exp

zx− z² 2

. Thus

Am(x) = d^m dz^mexp

x² 2 − 1

2(x−z)²

z=0

= e^x

2

2 d^m

dz^me⁻¹²^(x⁻^z)²

z=0

= (−1)^me^x

2

2 d^m

dx^me⁻¹²^(x−z)²

z=0

=Hm(x),

(24)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 19 compare to definition 2.2.

Finally, we will show that H_m are the only Appell polynomials that are orthogonal. I.e. assuming E[An(X)Am(X)] = 0 for n 6=m, we have to show thatX is normally distributed. Denote µm =E[X^m] and note that

φ(z) = ln X∞ m=0

z^m m!µm

!

= X∞ m=1

z^m m!χm

is the cumulant generating function and χ_m are the cumulants of X. Since Am(x) = _dz^d^mme^zx−φ^χ^(z)

z=0, one can inductively show that A_m+1(X) = XA_m(X)−

Xm k=0

m k

χ_m−k+1A_k(X). (2.12)

Let m ≥ 2. Then, by orthogonality, we have E[A1(X)Am(X)] = E[(X − µ₁)A_m(X)] = 0, which together with E[A_m(X)] = 0 implies E[XA_m(X)] = 0.

Thus taking expectation on both sides in (2.12) leads to 0 = E

" _m X

k=0

m k

χm−k+1Ak(x)

#

=χm+1.

Thus, X has moment-generating function exp(φ_χ(z)) = exp(χ₁z +χ₂z²) corresponding to the normal distribution.

Wick products

There are several more general versions of Appell polynomials in the multivariate case. Here, we will introduce the so-called Wick products which are useful for the presentation of the diagram formula in the next section.

Definition 2.5 For random variablesX1, . . . , Xmthe Wick product:X1, . . . , Xm: is defined as :∅:= 1 for m= 0 and then recursively for m≥1 by

∂

∂Xi :X1, . . . , Xm: = :X1, . . . , Xi−1, Xi+1, . . . , Xm: E[:X1, . . . , Xm:] = δ0,m.

The Appell polynomials are then given by Am(X) =:X, . . . , X

| {z }

m−times

:.

(25)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 20

2.3.2 Diagrams and the central limit theorem

Now, we want to outline the connection of Appell polynomials to the central limit theorem given in theorem 2.3. The link is given by the diagram formula, for which the introduction of joint cumulants and diagrams is needed.

Cumulants

We start with some notations: For an arbitrary index set I, let (X_i)_i∈I be a system of random variables with finite moments. Given a finite subset W ⊂ I, denote X^W = Q

i∈WXi and X^′W = {Xi, i ∈ W}. For X^′W with |W| = m, the joint cumulants are defined by

χ(X^′^W) := χ(Xi, i∈W) := ∂^m

∂z₁· · ·∂z_m lnEh

e^P^m^j=1^z^j^X^iji

z1=···=zm=0. If Xi = X for all i ∈ W, then χm(X) := χ({X, . . . , X

| {z }

m

}) is the m-th cumulant of X. Thus the cumulants are the coefficients of the joint cumulant generating function. If the latter is not finite, it can be replaced by the logarithm of the joint characteristic function. The definition immediately leads to the following properties:

(i) Multilinearity: If Xik = P_∞

j=1cikjYj with random variables Yj, j = 1,2, . . . and constants cikj ∈R, W ={ik:k = 1, . . . , m} ⊂I, then:

χ(X^′W) = X∞ j1,...,jm=1

Ym k=1

cikjk

!

χ({Yj1, . . . , Yjm}).

(ii) If there is a partitionW =W1∪W2for which{Xi, i∈W1}and{Xi, i∈W2} are independent, then χ(X^′^W) = 0.

(iii) If X^′^W are jointly normally distributed and |W| ≥3, thenχ(X^′^W) = 0.

The aim is now to express the joint cumulants of Appell polynomials, or more general of Wick product χ(:X^′^W:), in terms of χ(X^′^U), U ⊂W. This will be the contents of the diagram formula. Later, we will see thatχ(X^′^U) can be calculated explicitly, if Xi corresponds to a linear process.

(26)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 21 Diagram formalism

We now present the diagram formula for Wick products, which is the key for the proof of central limit theorems for linear processes. Though this formula provides a very elegant representation of cumulants, an extensive notation by means of so- called diagrams is necessary.

Let W = W1 ∪ · · · ∪ Wk denote a table whose rows consist of tuples Wj = {(j,1), . . . ,(j, mj)}, j = 1, . . . , k. (For intuition, W is an ordinary table whose rows Wj are arranged one apon the other and consists altogether of m := m1+

· · ·mk tuples.) A diagram γ = (V)r = (V1, . . . , Vr) is a partition of W, i.e. W = Sr

i=1V_r withV_i, V_j disjunct fori6=j. The class of all diagrams overW is denoted by ΓW. A diagram γ = (V1, . . . , Vr) is called connected, if the rows W1, . . . , Wk

can not be divided into two subgroups which are partitioned separately by γ, i.e.

if there is no division K1∪K2 ={1,2, . . . , k} withK1∩K2 =∅, K1, K2 6=∅such that for all q = 1, . . . , r we have either Vq ∈ S

j∈K1Wj or Vq ∈ S

j∈K2Wj. The class of all connected diagrams is denoted by Γ^c_W. Finally, Vq is called an edge of the diagram γ = (V₁. . . , V_r). An edge V_q is flat, if there is aj withV_q ⊂W_j, and Γ⁶⁻_W is the class of diagrams without flat edges respectively Γ⁶⁻_W^,c = Γ⁶⁻_W ∩Γ^c_W, the class of connected diagrams without flat edges.

Theorem 2.5 Let X(j,i), (j, i) ∈ W be random variables with finite moments, where W ={(j, i) :i= 1, . . . , mj and j = 1, . . . , k}. Then, for usual products we have

E

" _k Y

j=1

X^W^j

#

= X

γ=(V)r∈ΓW

χ(X^′^V¹)· · ·χ(X^′^V^r) χ(X^W¹, . . . , X^W^k) = X

γ=(V)r∈Γ^cW

χ(X^′^V¹)· · ·χ(X^′^V^r), while for Wick products

E

" _k Y

j=1

:X^W^j :

#

= X

γ=(V)r∈Γ⁶⁻_W

χ(X^′V¹)· · ·χ(X^′V^r) χ(:X^′^W¹ :, . . . ,:X^′^W^k :) = X

γ=(V)r∈Γ^6−,c_W

χ(X^′^V¹)· · ·χ(X^′^V^r).

More generally, for combinations of usual and Wick products:

χ(:X^′W¹ :, . . . ,:X^′W^s :, X^W^s+1, . . . , X^W^k) = X

χ(X^′V¹)· · ·χ(X^′V^r),

(27)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 22 where the sum is taken over all connected diagrams without flat edges inW1, . . . , Ws.

Proof: See Giraitis and Surgailis (1986).

Diagrams and limit theorems

The usefulness of the diagram formula will become clear, if one applies the following approach to weak convergence:

Theorem 2.6 Let (Yn)n≥1 be a sequence of random variables with finite moments. Further assume that there exists a random variable Y such that for all k ≥0

χk(Yn)→ χk(Y) as n→ ∞.

Then, if the cumulants (or equivalently the moments) of Y uniquely determine the distribution of Y, we have Yn

→d Y as n→ ∞.

Proof: By theorem 2.5, the moments of all orders of Yn converge to the corresponding moments of Y asn→ ∞. Thus the statement of the theorem is proven

in Gut (2007, theorem 8.6).

The proof of asymptotic normality (2.8) in theorem 2.3 can now be described as follows, compare e.g. Breuer and Major (1983), Giraitis and Surgailis (1985). We will follow the line of Surgailis (2003).

(1) By theorem 2.6, it suffices to show

χk(n⁻^1/2Sn,G)→0 as n→ ∞

for all k ≥ 3. (Note that χk(Y) = 0, k ≥ 3, if Y is normally distributed, and further that the normal distribution is uniquely determined by their moments.)

(2) Multilinearity of cumulants leads to χk(n^−1/2Sn,G) =n^−k/2

XM m1,...,mk=1

cm1· · ·cmkχ(Sn,Am1, . . . , Sn,A_mk), where Sn,A_mj :=Pn

t=1Amj(Xt) forj = 1, . . . , k.

(28)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 23 (3) The diagram formula can be applied to evaluate the cumulants of Appell

polynomials, so we get for m₁, . . . , m_k≥1:

χ(Sn,Am1, . . . , Sn,A_mk) =

Xn t1,...,tk=1

χ(Am1(Xt), . . . , Amk(Xt))

= X

γ∈Γ^6−,c_W

Jn,γ,

where

Jn,γ = Xn t1,...,tk=1

χV1. . . χVr, γ = (V1, . . . , Vr)∈ΓW.

The cumulants χV for V ⊂ W are defined as follows: For given t1, . . . , tk

and table W = W1 ∪. . .∪Wk with Wj = {(j, i) : i = 1, . . . , mj}, we fill the rows Wj with the random variables X(j,i) :=Xtj for all i = 1, . . . , mj, (1≤j ≤k). Then we denote

χV :=χ(X^′V) = χ(Xt1, . . . , Xt1

| {z }

|V∩W1|−times

, . . . , Xtk, . . . , Xtk

| {z }

|V∩Wk|−times

).

(4) Linearity of the process Xt (see (2.5) and recall that ξs are independent) leads to

χ_V = ˜χ_|_V_|X

s∈Z

b(t₁+s)^|V^∩W¹^|· · ·b(t_k+s)^|V^∩W^k^|, (2.13) where ˜χ_p = χ_p(ξ₀) is the p-th cumulant of ξ₀. Using (2.13), the product χ_V₁· · ·χ_V_r can be written (with a constant c) as

χV1· · ·χVr =cX

(sji)

Yk j=1

mj

Y

i=1

b(tj +sji)δV1· · ·δVr, (2.14) where

δVq =1{sji =sj^′i^′ for all (j, i),(j^′, i^′)∈Vq}, q = 1, . . . , r.

The sum in (2.14) is taken over all ’matrices’ (sji)j=1,...,k;i=1,...,mj with sji ∈ Z. (However, note that we only get a non-zero summand, if thesji coincide within each edge Vq,q = 1, . . . , r.) Thus, the following representation holds:

Jn,γ =X

(sji)

Xn t1,...,tk=1

Yk j=1

mj

Y

i=1

b(tj +sji) Yr q=1

δVq. (2.15)

(29)

CHAPTER 2. LONG MEMORY AND LIMIT THEOREMS 24 The problem is then to show n⁻^k/2Jn,γ →0 as n → ∞. This will be done in the remaining two steps.

(5) Consider the case P

s∈Z|b(s)| < ∞. Then one can show by combinatorial arguments that Jn,γ = O(n), for every diagram γ ∈ Γ⁶⁻_W^,c and table W consisting of k≥2 rows (see Surgailis 2003, proposition 6.1).

(6) Finally, for coefficients (2.6), one decomposesb(s) =b^<K(s)+b^>K(s), where b^<K(s) = b(s)1{|s| < K}, b^>K(s) = b(s)1{|s| ≥ K}, and applies step (5) to J_n,γ^K (here, J_n,γ^K is defined analogously to (2.15) with b(·) replaced by b^<K(s)). Then the proof is finished by proving |J_n,γ −J_n,γ^K | ≤ ǫ(K)n^k/2, where ǫ(K) is independent of n and converges to zero for K → ∞, and γ ∈ Γ⁶⁻_W^,c for a table W consisting of rows Wj with |Wj| = mj ≥ m_∗ (see Surgailis 2003, lemma 6.1). Hence, only this last step depends on the Appell rank m_∗ of G.

These steps, in particular step (5) and (6), which we did not carry out here, will be adapted for a volatility model in section 4.1.4.

2.3.3 Expansion of entire functions

Obviously, the asymptotic results for G(X_t) given in theorems2.3 and 2.4 are of limited applicability in statistics since G has to be a polynomial. For instance, the study of robust location estimators (see section4.1) directly leads to bounded G excluding polynomials. Therefore, the question whether there are functions for which an Appell expansion of infinite order holds is of high importance in statistics. In this section, we point out that entire functions satisfying some growth condition are natural candidates in this context.

Therefore note that in (2.9) we can multiply both sides with A(ω)⁻¹ := M_X(ω) and get

exp(zω) = 1 A(ω)

X∞ m=0

ω^m

m!Am(z), z ∈C. (2.16)

(Note the change of variables from P_∞

m=0 z^m

m!Am(x) to P_∞

m=0 ω^m

m!Am(z).) Thus, the definition of Appell polynomials already delivers an expansion of the entire function exp(·ω), where ω is such that the moment generating function MX(ω) is finite and greater than zero. We can then use results from complex analysis to

Asymptotic Statistical Theory for Long Memory Volatility Models