Bayesian Estimates in Time-Series Models

NOT FOR QUOTATION WITHOUT PERMISSION OF THE AUTHOR

BAYESIAN ESTIMATES I N Tm-SERIES MODELS

J u n e 1985 CP-85-29

C o Z Z a b o r a t i v e P a p e r s r e p o r t work which h a s n o t been p e r f o r m e d s o l e l y a t t h e I n t e r n a t i o n a l I n s t i t u t e f o r A p p l i e d Systems A n a l y s i s and which h a s r e c e i v e d o n l y l i m i t e d review. V i e w s o r o p i n i o n s e x p r e s s e d h e r e i n do n o t n e c e s s a r i l y r e p r e s e n t t h o s e o f t h e I n s t i t u t e ,

i t s N a t i o n a l . M e m b e r O r g a n i z a t i o n s , o r o t h e r o r g a n i - z a t i o n s s u p p o r t i n g t h e work.

INTERNATIONAL INSTITUTE FOR APPLIED SYSTEMS ANALYSIS A-2361 Laxenburg, A u s t r i a

Within the framework of the Economic Structural Change Proqram, a cooperative research activity of IIASA and the uni;ersity of Bonn,

FRG,

a project is carried out on "Statis- tical and Econometric Identification of Structural Change";

the project involves studies on the formal aspects of the

analysis of structural changes. On the one hand, they include statistical methods to detect

non-constancies,

such as sta- bility tests, detection criteria, etc., and on the other hand, methods which are suitable for models which incorporate non- constancy of the parameters, such as estimation techniques for time-varying parameters, adaptive methods, etc.

The present paper discusses the application of Bayesian estimation methods in the context of ARMA-models such as the periodic autoregression.

Anatoli Smyshlyaev Acting Leader

Economic Structural Change Program

BAYESIAN ESTIMATES I N TIME-SERIES MODELS J i r i Andel

C h a r l e s U n i v e r s i t y , S o k o Z o v s k a 8 3 , 1 8 6 0 0 Prague 8 , C z e c h o s Z o v a k i a

I N T R O D U C T I O N

A u t o r e g r e s s i v e (AR)

,

moving-average (MA)

,

and a u t o r e - g r e s s i v e - m o v i n g a v e r a g e (ARMA) models a r e v e r y p o p u l a r i n

time-series a n a l y s i s . Many problems of e s t i m a t i n g t h e i r param- e t e r s and t e s t i n g h y p o t h e s e s a r e o n l y s o l v e d a s y m p t o t i c a l l y . The d e r i v a t i o n o f a s y m p t o t i c r e s u l t s i s u s u a l l y n o t e a s y . An a l t e r n a t i v e a p p r o a c h t o s u c h problems i s t h e B a y e s i a n a p p r o a c h . I t i s assumed t h a t t h e p a r a m e t e r s o f t h e models a r e random

v a r i a b l e s . T h e r e a r e t h e o r e m s e n s u r i n g t h a t u n d e r g e n e r a l a s s u m p t i o n s t h e a s y m p t o t i c p o s t e r i o r d i s t r i b u t i o n d o e s n o t de- pend on t h e p r i o r d i s t r u b i t o n . A s t h e d e r i v a t i o n o f t h e re- s u l t s i s u s u a l l y e a s i e r i n t h e B a y e s i a n a p p r o a c h , w e c a n u s e t h i s p r o c e d u r e p a r t i c u l a r l y f o r t h e s t a t i s t i c a l a n a l y s i s of more c o m p l i c a t e d models.

PERIODIC AUTOREGRESSION

I n many a p p l i c a t i o n s w e e n c o u n t e r t i m e series w i t h a s e a s o n a l b e h a v i o r . W e may assume t h a t t h e l e n g t h o f s e a s o n p i s known ( e . g . , f o r m o n t h J y d a t a e x h i b i t i n g a one-year s e a s o n - a l i t y we have p = 1 2 ) . I n t h e c l a s s i c a l a p p r o a c h Box and J e n k i n s (1970, Ch. .9) recommend t o s t a r t w i t h t h e d i f f e r e n c e s

V X = X t - X and to apply an ARMA model to the differences

P t t-P

of the type VIVpXt. As such differencing is, however, not d D sufficient for real time series, a nonlinear transformation is sometimes applied before. In the well-known example of the number of airline passengers the log-transformation is applied at the beginning without taking into account its influence on other statistical procedures.

If the length of periodicity p = 2, we can consider the following modification of an AR model:

where {Y is a white noise. Simulations show (Andel 1983) t

that the realization of such a model is very similar to some economic time series. In some cases the model has an explosive behavior. To take account of this behavoir, we put

Then our model can be expressed in the form of

Let z l ,

z2 denote the roots of the polynomial

If l z l \ < l , 1z21<1, the process

{Et)

is stationary; other- wise

{E

) has an explosive behavior.

t

Our example was a special case of periodic autoregression.

We used a model of the second order with p = 2. Generally, the periodic autoregression is given by

If {Yt) is the usual white noise, we have a model with equal variances. If var Y 2

= Ok' 2

n+(j-l)p+k where o l ,

..., . . .

,cr2 are not all the same, we have a model with periodic variances. The periodic autoregression is P

a

special case of periodically correlated random sequences, whkch were intro- duced by Gladyshev (1961). Jones and Brelsford (1967) expanded bkl

, . . .

,bkn into a Fourier series. Pagano (1 978) considered estimators for bki obtained by modified Yule-Walker equations.

He also showed that a periodic AR can be rewritten into a multidimensional AR model. A periodic ARMA model was intro- duced by Cleveland and Tiao (1979). The problem of the

periodic AR is also treated by Troutman (19791. Tiao and

Grupe (1980) investigated the errors of nlsclassLfication when the periodic structure of an ARMA process was neglected.

Newton (1982) shows that a periodic AR can substantially simplify the numerical procedures for estimating parameters in multiple AR models. Andel (1983) presents some results of the Bayesian analysis of the periodic AR model including tests on whether the t2me series can be described by the classical AR model.

THE BAYESIAN APPROACH

If 8 ^E 9 is

a

random (multidimensional) parameter with a prior density p(0), and if a random vector X has a conditional density of ~ ( ~ 1 0 ) given 0, then according to the Bayes theorem, the posterior density ~ ( 0 1 ~ ) of 0 is given

by

where cx i s a c o n s t a n t . U s u a l l y t h e modus of t h e p o s t e r i o r d e n s i t y i s t a k e n a s a n e s t i m a t o r o f 8. T h i s i s a g e n e r a l i z a - t i o n of t h e maximum l i k e l i h o o d e s t i m a t o r , which w e would g e t f o r p ( 8 ) = c o n s t . (Another e s t i m a t o r o f 8 c o u l d be t h e

p o s t e r i o r e x p e c t a t i o n . )

The main problem w i t h u s i n g t h e BayesTan approach i s t h e c h o i c e of t h e p r i o r d e n s i t y p ( 0 )

.

The f o l l o w i n g t h r e e p o s s i - b i l i t i e s a r e most p o p u l a r :

a . C o n j u g a t e p r i o r d e n s 2 t y . L e t p(xl 0 ) b e g i v e n . A system M

P ( X ! 8 ) i s c a l l e d a c o n j u g a t e s y s t e m , i f p ( 8 [ x ) E

M

( x l

h o l d s f o r e v e r y p ( e ) E M

P

( X I

^{8 )}

.

U s u a l l y w e t a k e t h e

minimal system M

P ( X ( 8 ) w i t h t h i s p r o p e r t y , o r t h e s o - c a l l e d n a t u r a l system.

Although a c o n j u g a t e p r i o r d e n s i t y i s c o n v e n i e n t from t h e m a t h e m a t i c a l p o i n t of view, t h e r e i s no l o g i c a l r e a s o n f o r u s i n g it i n a g i v e n c a s e . Moreover, t h i s p r o c e d u r e d o e s n o t s p e c i f y which d e n s i t y from t h e system M

p ( x l

e )

s h o u l d be t a k e n . b. U n c e r t a i n t y p r i n c i p l e . I n t h e c a s e o f " f u l l i g -

norance" it i s recommended t o p u t p ( 8 ) = 1 f o r 8 ^E 52. T h i s o f t e n l e a d s t o improper ( o r vague) d e n s i t i e s , e . g . i f 52 = Rk

( E u c l i d e a n k-dimensional s p a c e ) . The a d v a n t a g e o f t h i s method i s t h a t t h e d e n s i t y p ( 8 ) = 1 can a s y m p t o t i c a l l y s u b s t i t u t e any o t h e r r e a s o n a b l e p r i o r d e n s 2 t y .

T h e o r e m (DeGroot 1970.

5

1 0 . 4 )

.

^{L e t no (8 1x1 = c p} ( x

1 0 )

and l e t . r r ( 8 ( x ) = c 1 p ( x [ 9 ) p ( 8 ) . L e t A

c

52 b e s u c h a s e t t h a t m = i n f o A ~ ( 8 ) > 0. L e t a , b , c > 0 s a t i s f y

s c p p ( 8 ) = ( l + c ) m

.

8 E R-A

Then

where

E =

max

[

(1-a) (a+b) -1 , (l+a+b+ac) -' ^(a-tbtab)

^J

⁺

In the typical situation the posterior distribution

TT

o is the normal one with a variance matrix of the order

N-'

(where N is the number of observations). Then a and

b

are nearly zero and

E

is very small.

Unfortunately, the uncertainty principle is not logically consistent. For example, let

~ ( ~ 1 8 )

= 1

ex (1-0) n-x , 0 c 0 e l .

If

8

is completely unknawn, we choose its prAor density p (8)

=

1 and compute p

(0

1x1 by means of the Bayea theorem.

-I

We find that w

= €JL

has a posterlor density of c w (X-1)/2(~-w1/2)n-X. ow ever,

1/2 n-x p(xlw)

=

wXI2 (1-w

)

If w is considered to be a completely unknown parameter with a constant prior density, then Its posterior density is c ~ w ~ / * . ( ~ - w 1/2)n-x

c. JeffreyFs principle. To avoid inconsistency it is 1 /2 necessary to choose the prtor density proport2onal to IJ(o)[ ,

where J is the Fisher information

matrix.

In practice the prior denstty is chosen by combining the

uncertainty principle and Jeffrey's principle.

SOME RESULTS CONCERNING TRE PERXODIC AR

Andel (1983) proved the following assertions:

Theorem. Assume model ( x )

,

where X I ,

...,

Xn are given constants and {ytl are i.i.d. N(0,o 2 ) varfables.

If

x l ,

...,

^xN

is a known realization, put

[N-n-k

"

_k ^{= ]}

^{+ 1}

^{(where [}

³

is the integer part), P

(kJ = "k

qij _h=

1 ₁

^{Xn+k+ (h-1 ) p-i}

^x

^{n+k+ (h-1 ) p-j I}

Let the prior density of a,b be a-I for a > (I. Then (npv) (N-n-np)

f

^{(bk-b:) ' Q ~ (bk-b:)} has a posterior F

k=

1

^np

,

N-n-np

distribution, and [n (p-1 ) v] [N-n (p+l )

3

A t H A has a posterior F n(p-1) ,N-n(p+l) distribution, where

Similar assertions are also derived for the model with periodic variances.

GENERALIZATIONS

Model ( x ) c a n b e g e n e r a l l z e d t o a model w i t h exogenous and endogenous v a r i a b l e s * The computations a r e s l i g h t l y e a s i e r when we write t h e a b s o l u t e term ^(pk) s e p a r a t e l y . The model r e a d s

Q u i t e a n a l o g o u s l y w e can g e t a m u l t i p l e model f o r t h e c a s e t h a t Xt a r e random v e c t o r s .

PROBLEMS OF STRUCTURAL CHANGES

For r e a s o n s of s i m p l i c i t y , we o n l y d l s c u s s h e r e t h e main i d e a s i n t h e f i e l d of c l a s s i c a l A'R models. T h e i r g e n e r a l i z a - t i o n t o a p e r i o d i c AR model i s t h e n obvious.

L e t Xt b e c r e a t e d by t h e model

where Yt a r e i n d e p e n d e n t v a r i a b l e s , Yt

-

^N ( 0 , ~ ~ ) 2

.

W e assume t h a t i n some unknownmoment, t = 8 , t h e r e can b e a change of p a r a m e t e r s :

Some methods for the

detection

of point 8 are described by Segen and Sanderson (1980) and

by

Basseville and Benveniste

(1983). Their procedures are Based on the cumulative sum (CUSUM) technique. Simulations show, howwer, that the CUSUM method is in many cases not sensit2ve enough.

Another promising model for detecting structural changes can be the model with exogenous and endogenous variables, e.g.

where

The problem is to estimate parameters

p ,

b l , ..., ^{bn, a,}

8, and a

2 =

var Yt. The change occurs in tine t

= 8 ,

mainly influencing the mean value of the process. This formulation reminds of the piece-wise linear regression with unknown break- points, where the solution is often Based on the maximum

likelihood method.

Both cases mentioned here can be generalized to more complicated models. Further research in th2s field is neces- sary, because the problem is interesting and important. It is clear that the theoretical results should be complemented by available computer programs.

REFERENCES

Andel, J. (1983). Statistical analysis of periodic auto- regression. Appl. Mat. 28:364-385.

Basseville, M., and A. Benveniste (19831. Sequential detection of abrupt changes in spectral characteristics of digital signals. IEEE Trans. of Inf. Th. IT-29:709-724.

Box, G.E.P., and G.M. Jenkins (1970). Time Series Analysis.

Forecasting and Control. San Francisco: Holden-Day.

Cleveland, W.P., and G.C. Tiao (1979). Modeling seasonal

time series. Rev. Economie Appllquee 32:107-129.

DeGroot, M.H. (1970). Optimal Statistical Decisions. New York: Mc Graw-Hill.

Gladyshev,E.G. (1961). Periodically correlated random sequences. Soviet Math. 2:385-388.

Jones, R.H., and W.M. Brelsford (1967). Time series with periodic structure. Biometrika 54:403-408.

Newton, H.J. (1982). Using periodic autoregression for

multiple spectral estimation. Technometrics 24:109-116.

Pagano, M. (1978). On periodic and mulitple autoregression.

Ann. Statist. 6:1310-1317.

Segen, J., and A.C. Sanderson (1980). Detecting change in a time series. IEEE Trans. on Tnf. Th. IT-26:249-255.

Tiao, C.G., and M.R. Grupe (1980). Hidden periodic auto- regressive-moving average models tn time series data.

Biometrika 67:365-373.

Troutman, B.M. (1979). Some results in periodic autoregression.

Biometrika 66:219-228.