Applied Time Series Analysis

(1)

Applied Time Series Analysis

SS 2013 – Week 11

Marcel Dettling

Institute for Data Analysis and Process Design Zurich University of Applied Sciences

marcel.dettling@zhaw.ch http://stat.ethz.ch/~dettling

ETH Zürich, May 6, 2013

(2)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting with Time Series

Goal: Prediction of future observations with a measure of uncertainty (confidence interval)

Note: - will be based on a stochastic model

- builds on the dependency structure and past data - is an extrapolation, thus to take with a grain of salt

- similar to driving a car by using the rear window mirror

(3)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting, More Technical

Past Future

| | | … | | | | … | x₁ x₂ x₃ x_n-1 x_n X_n+1 X_n+2 X_n+k

observed forecast

observations estimates

 x

1

,  , x

_n

  X

1ⁿ

 ^X ^ˆ

ⁿ^^1,ⁿ

^, ^ ^, ^X ^ˆ

^{n k n}^ ^,



(4)

Applied Time Series Analysis

SS 2013 – Week 11

Sources of Uncertainty in Forecasting

There are 4 main sources of uncertainty:

1) Does the data generating model from the past also apply in the future? Or are there any breaks?

2) Is the AR(p)-model we fitted to the data

correctly chosen? What is the “true” order?

3) Are the parameters , and accurately estimated? How much do they differ from the “truth”?

4) The stochastic variability coming from the innovation

 we will here restrict to short-term forecasting!

 x

1

,  , x

_n



1,..., _p

  

_E²

m

E

t

(5)

Applied Time Series Analysis

SS 2013 – Week 11

How to Forecast?

Probabilistic principle for point forecasts:

 we forecast the expected value, given our observations Probabilistic principle for prediction intervals:

 we use the conditional variance

, 1

ˆ

_{n k n} _{n k}

|

ⁿ

X

_

  E X 

_

X  



^{n k}

^|

¹ⁿ



Var X

_

X

(6)

Applied Time Series Analysis

SS 2013 – Week 11

MA(1) Forecasting: Summary

• We have seen that for an MA(1)-process, the k-step forecast for k>1 is equal to .

• In case of k=1, we obtain for the MA(1)-forecast:

The conditional expectation is (too) difficult to compute

• As a trick, we not only condition on observations 1,…,n, but on the infinite past:

m

1, 1 1

ˆ

_n _n

[

_n

|

ⁿ

]

X

_

   m  E E X

: [ |

ⁿ

]

n n

e  E E X

_

(7)

Applied Time Series Analysis

SS 2013 – Week 11

MA(1) Forecasting: Summary

• We then write the MA(1) as an AR(∞) and solve the model equation for :

• In practice, we plug-in the time series observations

where available. For the „early“ times, where we don‘t have observations, we plug-in .

• This is of course only an approximation to the true MA(1)- forecast, but it works well in practice, because of:

E

n

1 0

( ) (

^j

)

n n j

j

E

^

 X

_

m



    

| 

1

| 1 

x

n j_

m ˆ

(8)

Applied Time Series Analysis

SS 2013 – Week 11

ARMA(p,q) Forecasting

As with MA(1)/MA(q) forecasting, we face problems with

which is difficult to compute. We use the same tricks as for MA(1) and obtain

where …

[

_n 1 _j

|

ⁿ

] E E

_{ }

X

_

,

1

ˆ ( [ | ] )

p

n

n k n i n k i

i

X

_

m  E X

_{ }

X

_

m



   

1

[ | ] [ | ]

q

n n

n k j n k j

j

E E

_

X

_

 E E

_{ }

X

_



  

(9)

Applied Time Series Analysis

SS 2013 – Week 11

ARMA(p,q) Forecasting

…where

if t≤n if t>n and

if t≤n 0 if t>n with

[

_t

|

ⁿ

]

E X X

_

 x

^t

ˆ ,

Xt n

[

_t

|

ⁿ

]

E E X

_

 e

^t

1 1

( )

p q

t t i t i j t j

i j

e x m  x

_

m  e

_

 

      

(10)

Applied Time Series Analysis

SS 2013 – Week 11

ARMA(p,q) Forecasting: Douglas Fir

Time

series

1200 1400 1600 1800

-6-4-202460.01.0

Lag k

Auto-Korr.

0 5 10 15 20 25 30

-0.30.0

Lag k

part. Autokorr

0 5 10 15 20 25 30

(11)

Applied Time Series Analysis

SS 2013 – Week 11

ARMA(p,q) Forecasting: Example

Time

0 20 40 60 80 100

-0.2-0.10.00.10.2

Forecasting the Differenced Douglas Fir Series

(12)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting Decomposed Series

The principle for forecasting time series that are decomposed into trend, seasonal effect and remainder is:

1) Stationary Remainder

Is usually modelled with an ARMA(p,q), so we can generate a time series forecast with the methodology from before.

2) Seasonal Effect

Is assumed as remaining “as is”, or “as it was last” (in the case of evolving seasonal effect) and extrapolated.

3) Trend

Is either extrapolated linearly, or sometimes even manually.

(13)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting Decomposed Series: Example

Unemployment in Maine

Time

(%)

1996 1998 2000 2002 2004 2006

3456

(14)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting Decomposed Series: Example

Logged Unemployment in Maine

Time

log(%)

1996 1998 2000 2002 2004 2006

1.01.21.41.61.8

(15)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting Decomposed Series: Example

STL-Decomposition of Logged Maine Unemployment Series

1.01.41.8data -0.20.00.2

seasonal 1.21.5trend -0.050.05

1996 1998 2000 2002 2004 2006

remainder

time

(16)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting Decomposed Series: Example

0.0 0.5 1.0 1.5

-0.40.00.40.8

Lag

ACF

ACF of Remainder Series

0.5 1.0 1.5

-0.20.00.20.4

Lag

Partial ACF

PACF of Remainder Series

(17)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting Decomposed Series: Example

Time

fit$time.series[, 3]

1996 1998 2000 2002 2004 2006 2008

-0.050.000.05

AR(4) Forecast for Remainder Series

(18)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting Decomposed Series: Example

Time

fit$time.series[, 2]

1996 1998 2000 2002 2004 2006 2008

1.21.31.41.51.61.7

Trend Forecast by Linear Extrapolation

(19)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting Decomposed Series: Example

Forecast of Logged Unemployment in Maine

Time

log(%)

1996 1998 2000 2002 2004 2006 2008

1.01.21.41.61.8

(20)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting with SARIMA

We have seen that forecasting decomposed series can be a somewhat laborious process. In R, it is easier and quicker to use a SARIMA model for forecasting season/trend-series.

• The SARIMA model is fitted in R as usual. Then, we can simply employ the predict() command and obtain the forecast plus a prediction interval.

• Technically, the forecast comes from the non-stationary ARMA(p,q)-formulation of the SARIMA model.

• The disadvantage of working with SARIMA forecasts is that it is much more of a black box approach than the one before!

(21)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting with SARIMA: Example

Time

log(AP)

1955 1956 1957 1958 1959 1960 1961

5.65.86.06.26.4

Forecast of log(AP) with SARIMA(0,1,1)(0,1,1)

(22)

Applied Time Series Analysis

SS 2013 – Week 11

Exponential Smoothing

Simple exponential smoothing:

- works for stationary time series without trend & season - is a heuristic, model-free approach

- further in the past -> less weight in the forecast Turns out to yield these forecasts:

where and

 See the blackboard for the derivation...

1 1,

0

ˆ ⁿ

n n i n i

i

X w x



 





^w⁰ ^ ^w¹ ^ ^w² ^{ }^... ⁰ ¹

0

1

n

i i

w









(23)

23

Marcel Dettling, Zurich University of Applied Sciences

Applied Time Series Analysis

SS 2013 – Week 11

Choice of Weights

An usual choice are exponentially decaying weights:

where (1 )ⁱ

wi  a a a (0,1)

0 5 10 15

0.00.10.20.30.40.5

a=0.5

w_i

0 5 10 15

0.00.10.20.30.40.5

w_i

a=0.1

(24)

Applied Time Series Analysis

SS 2013 – Week 11

Forecasting with Exponential Smoothing

The 1-step forecast is:

General Formula “Update”-Formula Remarks:

- in real applications (finite sum), the weights do not add to 1.

- the update-formula is useful if “new” observations appear.

- the k-step forecast is identical to the 1-step forecast.

1

1, , 1

0

ˆ (1 ) (1 ) ˆ

n

i

n n n i n n n

i

X a a x ax a X



  



      

(25)

Applied Time Series Analysis

SS 2013 – Week 11

Exponential Smoothing: Remarks

• the parameter can be determined by evaluating forecasts that were generated from different . We then choose the one resulting in the lowest sum of squared residuals.

• exponential smoothing is fundamentally different from AR(p)- forecasting. All past values are regarded for the 1-step

forecast, but all k-step forecasts are identical to the 1-step.

• It can be shown that exponential smoothing can be optimal for MA(1)-models.

• there are double/triple exponential smoothing approaches that can deal with linear/quadratic trends.

a

(26)

Applied Time Series Analysis

SS 2013 – Week 11

Exponential Smoothing: Example

Complaints to a Motorizing Organization

Time

1996 1997 1998 1999 2000

5101520253035

(27)

Applied Time Series Analysis

SS 2013 – Week 11

Exponential Smoothing: Example

> fit <- HoltWinters(cmpl, beta=F, gamma=F)

Holt-Winters exponential smoothing without trend and without seasonal component.

Smoothing parameters:

alpha: 0.1429622 beta : FALSE

gamma: FALSE Coefficients:

[,1]

a 17.70343

(28)

Applied Time Series Analysis

SS 2013 – Week 11

Exponential Smoothing: Example

Holt-Winters filtering

Time

Observed / Fitted

1996 1997 1998 1999 2000

5101520253035

(29)

Applied Time Series Analysis

SS 2013 – Week 11

Holt-Winters Method

Purpose:

- is for time series with deterministic trend and/or seasonality - is still a heuristic, model-free approach

- again based on weighted averaging Is based on these 3 formulae:

 See the blackboard for the derivation...

1 1

( ) (1 )( )

( ) (1 )

t t t p t t

t t t t

t t t t p

a x s a b

b a a b

s x a s

 

 

 

  

 



    

   

(30)

Applied Time Series Analysis

SS 2013 – Week 11

Holt-Winters: Example

Sales of Australian White Wine

Time

1980 1985 1990 1995

100200300400500600

(31)

Applied Time Series Analysis

SS 2013 – Week 11

Holt-Winters: Example

Logged Sales of Australian White Wine

Time

1980 1985 1990 1995

4.55.05.56.06.5

(32)

Applied Time Series Analysis

SS 2013 – Week 11

Holt-Winters: R-Code and Output

> HoltWinters(x = log(aww))

Holt-Winters exponential smoothing with trend and additive seasonal component.

Smoothing parameters:

alpha: 0.4148028; beta : 0; gamma: 0.4741967 Coefficients:

a 5.62591329; b 0.01148402

s1 -0.01230437; s2 0.01344762; s3 0.06000025 s4 0.20894897; s5 0.45515787; s6 -0.37315236 s7 -0.09709593; s8 -0.25718994; s9 -0.17107682 s10 -0.29304652; s11 -0.26986816; s12 -0.01984965

(33)

Applied Time Series Analysis

SS 2013 – Week 11

Holt-Winters: Fitted Values & Predictions

Holt-Winters filtering

Time

Observed / Fitted

1980 1985 1990 1995

4.55.05.56.06.5

(34)

Applied Time Series Analysis

SS 2013 – Week 11

Holt-Winters: In-Sample Analysis

4.55.5xhat 4.85.46.0level 0.0080.014trend -0.20.2

1985 1990 1995

season

Time

Holt-Winters-Fit

(35)

Applied Time Series Analysis

SS 2013 – Week 11

Holt-Winters: Predictions on Original Scale

Time

aww

1980 1985 1990 1995

100200300400500600

Holt-Winters-Forecast for the Original Series

(36)

Applied Time Series Analysis

SS 2013 – Week 11

Exercise

Data:

 use the Australian white wine sales data...

 ... or any other dataset you like Goal:

- Find a good model describing these data

- Evaluate which model yields the best predictions - Generate a 29-month forecast from this model Method:

 Remove the last 29 observations and mimic oos-forecasting

Applied Time Series Analysis

Applied Time Series Analysis

Marcel Dettling

ETH Zürich, May 6, 2013

Applied Time Series Analysis

Forecasting with Time Series

Applied Time Series Analysis

Forecasting, More Technical

 x

,  , x

  X

 X ˆ

,  , X ˆ



Applied Time Series Analysis

Sources of Uncertainty in Forecasting

 x

,  , x



  

m

E

Applied Time Series Analysis

How to Forecast?

ˆ

|

X

  E X 

X  



|



Var X

X

Applied Time Series Analysis

MA(1) Forecasting: Summary

m

ˆ

[

|

]

X

   m  E E X

: [ |

]

e  E E X

Applied Time Series Analysis

MA(1) Forecasting: Summary

E

( ) (

)

E

 X

m

    

| 

| 1 

x

m ˆ

Applied Time Series Analysis

ARMA(p,q) Forecasting

[

|

] E E

X

ˆ ( [ | ] )

X

m  E X

X

m

   

[ | ] [ | ]

E E

X

 E E

X

  

Applied Time Series Analysis

ARMA(p,q) Forecasting

[

 ^X ^ˆ

^, ^ ^, ^X ^ˆ

^|