1
Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series Analysis
SS 2014 – Week 08
Marcel Dettling
Institute for Data Analysis and Process Design Zurich University of Applied Sciences
marcel.dettling@zhaw.ch http://stat.ethz.ch/~dettling
ETH Zürich, April 7, 2014
Applied Time Series Analysis
SS 2014 – Week 08
Non-Stationary Models: ARIMA and SARIMA
Why?
We have seen that many time series we encounter in prac- tice show trends and/or seasonality. While we could de-
compose them and model the stationary part, it might also be attractive to directly model a non-stationary series.
How does it work?
There is a mechanism, "the integration" or "the seasonal integration" which takes care of the deterministic features, while the remainder is modeled using an ARMA(p,q).
There are some peculiarities!
see blackboard!
3
Applied Time Series Analysis
SS 2014 – Week 08
Example: Monthly Oil Prices
Time
oil.price
1990 1995 2000 2005
102030405060
Monthly Price for a Barrel of Crude Oil
Applied Time Series Analysis
SS 2014 – Week 08
Taking the Logarithm is Key
Time
log(Price)
1990 1995 2000 2005
2.53.03.54.0
Logged Monthly Price for a Crude Oil Barrel
5
Applied Time Series Analysis
SS 2014 – Week 08
Differencing Yields a Stationary Series
Time
Differences
1990 1995 2000 2005
-0.4-0.20.00.20.4
Differences of Logged Monthly Crude Oil Prices
Applied Time Series Analysis
SS 2014 – Week 08
ARIMA(p,d,q)-Models
Idea: Fit an ARMA(p,q) to a time series where the d
thorder difference with lag 1 was taken before.
Example: If , then
Notation: With backshift-operator B()
Stationarity: ARIMA-models are usually non-stationary!
Advantage: it‘s easier to forecast in R!
1
(1 ) ~ ( , )
t t t t
Y X X
B X ARMA p q
( )(1 B B )
dX
t( ) B E
t
~ ( ,1, )
X
tARIMA p q
7
Applied Time Series Analysis
SS 2014 – Week 08
ACF/PACF of the Differenced Series
0.0 0.5 1.0 1.5 2.0
-1.0-0.50.00.51.0
Lag
ACF
ACF
0.5 1.0 1.5 2.0
-1.0-0.50.00.51.0
Lag
Partial ACF
PACF
Applied Time Series Analysis
SS 2014 – Week 08
Fitting an ARIMA in R
We start by fitting an ARIMA(1,1,2) to the oil series:
> arima(lop, order=c(1,1,2)) Call:
arima(x = lop, order = c(1, 1, 2)) Coefficients:
ar1 ma1 ma2 0.8429 -0.5730 -0.3104 s.e. 0.1548 0.1594 0.0675
sigma^2 = 0.0066: ll = 261.88, aic = -515.75
9
Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series Analysis
SS 2014 – Week 08
Alternative Fitting
Instead of fitting an ARIMA(1,1,2) to the logged oil series, we can also take the differenced log-oil series and fit an ARMA(1,2) to it.
IMPORTANT:
In this case, we have to do fitting without including an intercept (why?), thus:
> arima(diff(log(oil.price)), order=c(1,0,2),
include.mean=FALSE)
Applied Time Series Analysis
SS 2014 – Week 08
Meaning of the Model / Recipe
We can rewrite the ARIMA(1,1,2) model as an ARMA(2,2), see blackboard...
Some guidelines on how to fit ARIMA models to observed
time series can also be found on the blackboard...
11
Applied Time Series Analysis
SS 2014 – Week 08
Residual Analysis of the ARIMA(1,1,2)
0.0 0.5 1.0 1.5
0.00.20.40.60.81.0
Lag
ACF
ACF
0.5 1.0 1.5
-0.100.000.10
Lag
Partial ACF
PACF
Applied Time Series Analysis
SS 2014 – Week 08
SARIMA(p,d,q)(P,D,Q) s
= a.k.a. Airline Model. We are looking at the log-trsf. airline data
Log-Transformed Airline Data
Time
AirPassengers
1950 1952 1954 1956 1958 1960
100200300400600
13
Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series Analysis
SS 2014 – Week 08
Seasonal Differencing Helps…
or at the log-transformed Australian Beer Production
Time
diff(lap, lag = 12)
1950 1952 1954 1956 1958 1960
0.00.10.20.3
Seasonally Differenced Airline Passenger Series
Applied Time Series Analysis
SS 2014 – Week 08
… But More Is Needed!
or at the log-transformed Australian Beer Production
Time
diff(diff(lap, lag = 12))
1950 1952 1954 1956 1958 1960
-0.15-0.050.050.15
Differenced Seasonally Differenced Airline Passenger Series
15
Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series Analysis
SS 2014 – Week 08
SARIMA(p,d,q)(P,D,Q) s
We perform some differencing… ( see blackboard)
Applied Time Series Analysis
SS 2014 – Week 08
ACF/PACF of SARIMA(p,d,q)(P,D,Q) s
Time
series
1950 1952 1954 1956 1958 1960
-0.15-0.10-0.050.000.050.100.150.00.51.0
Lag k
Auto-Korr.
0 5 10 15 20 -0.4-0.10.2
Lag k
part. Autokorr
1 5 10 15 20
17
Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series Analysis
SS 2014 – Week 08
Modeling the Airline Data
Since there are “big gaps” in ACF/PACF:
This is an MA(13)-model with many coefficients equal to 0, or equivalently, a SARIMA(0,1,1)(0,1,1)
12.
Note: Every SARIMA(p,d,q)(P,D,Q)
scan be written as an ARMA(p+sP,q+sQ), where many coefficients will be equal to 0.
12
1 1
(1 )(1 )
t t
Z B B E
1 1 1 12 1 1 13
t t t t
E E
E
E
Applied Time Series Analysis
SS 2014 – Week 08
SARIMA(p,d,q)(P,D,Q) s
The general notation is:
Interpretation:
- one typically chooses d=D=1
- s = periodicity in the data (season)
- P,Q describe the dependency on multiples of the period
see blackboard...
(1 ) (1 )
( ) ( ) ( ) ( )
d s D
t t
s s
s t s t
Z B B X
B B Z B B E
19
Applied Time Series Analysis
SS 2014 – Week 08
Forecasting Airline Data
Time
log(AP)
1955 1956 1957 1958 1959 1960 1961
5.65.86.06.26.4
Forecast of log(AP) with SARIMA(0,1,1)(0,1,1)
Applied Time Series Analysis
SS 2014 – Week 08
Residual Analysis of SARIMA(0,1,1)(0,1,1)
0.0 0.5 1.0 1.5
-0.20.00.20.40.60.81.0
Lag
ACF
ACF
0.5 1.0 1.5
-0.15-0.050.050.15
Lag
Partial ACF
PACF
21
Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series Analysis
SS 2014 – Week 08
Outlook to Non-Linear Models
What are linear models?
Models which can be written as a linear combination of i.e. all AR-, MA- and ARMA-models
What are non-linear models?
Everything else, e.g. non-linear combinations of ,
terms like in the linear combination, and much more!
Motivation for non-linear models?
- modeling cyclic behavior with quicker increase then decrease - non-constant variance, even after transforming the series
X
tX
t 2X
tApplied Time Series Analysis
SS 2014 – Week 08
SMI Log-Returns
SMI Log-Returns
Time
lret.smi
2500 3000 3500 4000
-0.08-0.040.000.04
23
Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series Analysis
SS 2014 – Week 08
Normal Plot of SMI Log-Returns
+ +
+ + +
+ +
+
+ + + + ++
+ +
+ + ++++ ++ ++ + ++
+ + + +
+
+
+ + + ++
+
+ + + + +
+ +
+ ++ + ++ +
+ +
+ + ++
+ +
+ ++ +
+ + + ++ + ++ +
+ + +
+ ++ + +
+ + + +++
++ +
+ +
+ + + +
+ +
+ + ++
+
+ +
+ +
+ +
+ + +
+ +
+ + + + + +
+ +
+ +
+
+ + +
+ +
+ +
++
+ + +
+
+ ++ + + ++ +
+ +
+ ++ + + ++ + + + + +
+
+ + +
+ + + +
+ ++ + ++
+ + + +
++
+ +
+
+ +
+ +
++ + +
+ + +
+ +
+ + +
+
+ +
+ + + ++
+
+ +
+ +
+
+ + +
+ + +
+ ++
+ +
+ +
+
+ +
+ +
++
+ + + ++ + + + + + ++
+ +
+ ++
+ +
+ + +
+
+ + +
+ +
+ ++
++
+ + +
+ +
+
+
+
+ ++ +
+
++ + + +
++++ + + ++
++ +
+ +
+ +
+
+ ++ + +
+ +
+ +
+ +
+
+
+ + +
+
+ + ++
+
+ + + +
+
+ + + +
+ +
+ +
+ ++
+ + ++
+ +
+ + +++ ++++
+
+ ++ +
+
+ + + +
+ +
+ +
+ + + + +
+ +
++ +
+
+ + + + + + +
+
+
+ +
+
+ ++
+ +
+ +
+ +
+ + +
+
+
+ +
+ +
+
+ + +
++ + ++ + +
+ ++
+ +
+ + +
+
+ +
+ ++ +
+ ++
++ + +
+ + +
+ +
+
+ + +
+ + + +
+ + + + + + ++ +
+ + + +++ + + ++
+ +
+ +
++
++ + +
+ ++ + +
++ ++++ + + +
+
+ + + + +
+ + ++
+
+ +
+ + +
+
+
+ +
+ ++
++
+ +
+ + + + +
+ + +
+
+
+ +
+ + ++ ++
+ + ++ +
+
+ + + +
+
+
+++ + +
+
+ ++
+ +
+ +
+ +
++ +
+ +
++ + + +
+ + +
+ +
+ ++ +
+
+ + +
+ ++ +
+
+ + + +
+
++ ++++ + + +
+
+ +
+ + ++ + + + +
+ + ++ + +
+
+ +
+ + + +++ + +
+ ++ + ++ + +
+
+ +
+
+ + ++ +
+
+ + +
+ + + + +
+ + ++
+
+ + +
+ +
+ +
+ ++
+ +
+ +
+ + + +
+
+
+ + + +
+
+ + +
+
+ +
++
+ +
+ +
+ +
+ +
+
+ + + ++ + + +
++ + +
+
+ +
+ +
+
+ +
++
+ +
+ +
+ + + +
+ + +
+ + + +
+ + +
+ +
+ + + +
+ + +
+ +
+ + + + +
+++
+
+
+ +
+
+
+ + ++
+
+ ++
+
+
+ +
+ + +
+ +
++ ++
+
+ + +
+ + + +
+ +
+ + ++
+ + ++ +
+ ++ +
+
+ +
+
+ +
+
+ +
+ + +
+ +
+
+
+ + ++
+ + +
++ +
+ +
+ +
+ +
+
+ ++
+ + +
+
+ + ++
+ + ++
+
+ +
+
+ + +
+ + +
+ + + + +
+ ++ + +
+ +
+
+ + +
+
+ + +++ + ++++ ++ + +
++ +
+ +
+ +
+ ++ ++ ++ + +
+
+ +
+ + ++
+ +
+ + + +
+ + + + + +
+ + + + +
+ ++ + + ++
+ + +
+ +
+ ++
+ +
+ + +
+
+ + +
+ +
+
+
+ +
++ + + +++ + +
+ +
+ + +
+
+ +
+ +
+ + + ++
+ + + +
+ + +
+ +
+ +
+
+ + + + ++ + +
++ + ++ + +
+
+ + +
++ +
+ +
+ + +
+ + +
+ + +
+ +
+ ++
+
++ + ++ + + +
+ +
+ ++ +
+ + +
+ +
+ +
+ +
+ + +
+ + +
+ + +
+ +
+ + + + +
+ +
+
+
+ +
+
+ +
+ + + + +
+ + + +
+
+ ++
+ + +
+ + +
+ + +
+
+ +
+ + +
+ + + +
+ + +
+
+ + +
+ ++ + + +
+ + ++ ++
+
+ + +
+ + +
++
+
+ +
+ +
+ + +++
+ +
+ ++
+ +
+ +
+ +
+ + ++
+ +
+ +
+ + + +
+
+ + + + +++++ + +
++ +
+ +
+ +
+ + +
+
+
+ +
++ + +
++ + ++ +
+ +
+
+ ++
+
+ +
+ + +
+
++ + ++ + + +
+ + +
+ + ++ ++ + +
+
+
+ +
+ + +
+ +
+
+ +
+ + ++
+ +
+ + + +
+ ++
+ ++ ++ ++
+ + +
+
+
+ +
+ + +
+
+ + + ++
+
+ + +
+
+
+
+ + +
+ + +
+
+ + +
++
+ + +
+ +
+
+ +
+ +
+ +
+ +
+
+ +
+ ++
+
+ ++ +
+
+ ++ +
+
+ + + +
+ + + +
+
+ + +
+ +++
+ ++ + +
+
+ + +
+ + + ++ + +
+ ++ +
+ + +
++ + +
+ + + +
+ + +
+
+ +
+
+ +
+ +
+
+ +
++ + + + ++
+ + + +++
+ + ++
+
+ +
+ + +
+ +
+ +
+ +
+ +
+ +
+++
+
+ + + +
+ + +
+
+ + +
+
++ ++ +
+ ++ + + + +
+
+ +
+
+ +
+ +
+ + +
+ +
+
+ + +
+ +
+ ++ +
+ +
+ ++
+ + +
+
+ + +
+
+ + +
+ + +
+ + ++ + +
+
+ + + +
+ +
+ +
+ + +
+
+ + +
+
+ + ++
+ +
+ + +
+
+ +
+
+ +
+ +
+ + +
++
+ +
+ +
+ +
+
+
+ + +
+
+
+ +
+
+
+
+ + + +
+ +++
+ +
+ +
+ +
+
+
+
+
+ + +
+ + +
++
+ +
+ +
+ +
+ + +
+
+ ++
+ + + + ++
+
+
+ +
+ + +
+
+ +
+
+
+ +
+
+
+ + +
+ +
+
++
+ + +
+ + +
++ +
+ + + +
+
+ +++ +
+ + + +
+
+ + +
+
+ + +++
+ +
+++
+
+ +
+ + +
+ + +
+ + +
+ +
+ + +
+
+ + +
+ +
+ +
+ + +
+
+ +
+
+ + +
+ +
+ + +
+ +
+
+
+ +
+ + +
+
+ +
+
+
+ + ++
+
+ + ++ + + + +
+
++ ++ +
+
+
+ +
+ + + +
+ +
+ +
+
+ +
+
+ +
++ ++ +
+
+ + + +
+
+ +
++ + + +
+ + + +
+
++
+
+
+
+ +
+ +
+
+ +
+
+
+ + +
+ +
+ + + + + +
+ + +
+ +
+ +
+ +
+ +
+
+ +
+
+
+ +
+ +
+
-3 -2 -1 0 1 2 3
-0.08-0.040.000.04
Normal Plot
Theoretical Quantiles
Sample Quantiles
Applied Time Series Analysis
SS 2014 – Week 08
ACF of SMI Log-Returns
0 5 10 15 20 25 30
0.00.20.40.60.81.0
Lag
ACF
ACF of SMI Log-Returns
25
Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series Analysis
SS 2014 – Week 08
ACF of of Squared SMI Log-Returns
0 5 10 15 20 25 30
0.00.20.40.60.81.0
Lag
ACF
ACF of Squared Log-Returns
Applied Time Series Analysis
SS 2014 – Week 08
The ARCH / GARCH Model
See blackboard...
27
Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series Analysis
SS 2014 – Week 08
Model Choice
0 5 10 15 20 25 30
0.00.20.40.60.81.0
Lag
ACF
ACF of Squared Log-Returns
0 5 10 15 20 25 30
0.00.20.40.60.81.0
Lag
Partial ACF
