• Keine Ergebnisse gefunden

Accounting for the different sample periods of the available data, recursively estimated monthly pseudo out-of-sample forecasts are constructed for the period between 1991.01 and 2017.11. The main objective is to forecast the latest release of the real Brent crude oil price.10 All the models described in section 2 will be evaluated by means of a quadratic loss function.

Thus, the MSPE ratios of the individual model will be compared to the no-change benchmark.

It is not possible to provide accurate p-values for equal predictive accuracy as no such tests are available at the moment. This is due to the fact that the forecasts are generated in real-time and are subject to revisions which violates the assumption of the data being stationary which in turn is required for standard tests. Furthermore, these tests are based on the population MSPE and not on the out-of-sample MSPE used in this work. However, as it is hard to draw conclusions about the forecasting performance without statistical tests, p-values for the test of equal predictive accuracy of Clark and West (2007) are reported. Nevertheless, these p-values should only be interpreted with caution as they are biased towards rejecting the null hypothesis of equal MSPE. A more detailed reasoning of why the available tests do not apply in this context can be found in Baumeister et al. (2015), Baumeister and Kilian (2012), Inoue and Kilian (2005), and Y. Wang et al. (2017) and references therein.

Moreover, the success ratio of the different models will be tested using the directional accu-racy test by Pesaran and Timmermann (1995). Subsequently, the bias of the forecast errors will be examined to address a possible systematic distortion of the forecasts.

2.4.1 Forecasting Results

Table 2.2 summarizes the average forecasting performance of the individual models for fore-casting horizons up to 24 months. A value lower than one indicates an improvement of the indi-vidual forecast as compared to the no-change benchmark on average over the complete sample period. Throughout all models under consideration, the ‘Futures’ model consistently outper-forms the no-change forecast over all horizons and reduces the MSPE between 2% and 25%.

The ‘ARMA’, ‘VAR Steel’ and the ‘CRB Index’ model offer gains in the MSPE ratio for short-term projections. However, for medium- and long-short-term projections the ‘Inventories OECD’, the ‘Oil Stock’ and the ‘VAR CLI’ model provide considerable reductions in the MSPE ratio.

Additionally, the average forecasting performance across all 24 horizons confirms a good fore-casting performance for the ‘Futures’, ‘Inventories OECD’, ‘VAR CLI’ and ‘Oil Stock’ model on average. The results for the directional accuracy test can be found in Table 2.3. At each hori-zon, there is at least one model that predicts the direction of change significantly better than the no-change benchmark. Again, the ‘Futures’ model seems to have the best performance and is significantly better at almost all horizons. While the ‘Oil Stock’ model shows a poor per-formance in this context, the ‘CRB Index’ shows the highest success ratio of all the models considered with up to 64% and the second highest performance on average with 58% across all horizons. In addition, the results for the ‘Inventories OECD’ and ‘VAR CLI’ model seem to be in alignment with the results obtained for the MSPE ratios.

10 The results are robust with regard to the WTI price as well as to the chosen specification of the benchmark. The latter has been tested using an AR process and using the average growth rate of the real price of oil. A robustness check can be found in Appendix B.

TABLE2.2: Recursive MSPE ratios relative to the no-change benchmark

Horizon ARMA Futures Inventories VAR VAR CRB Oil

OECD CLI Steel Index Stock

1 0.9153(0.1114) 0.7549(0.0004) 1.0751 (0.9956) 0.8867(0.1449) 0.9535(0.3046) 0.9819(0.4116) 1.0782 (0.9783) 2 0.9453(0.2309) 0.9312(0.0680) 1.0599 (0.9293) 0.8612(0.1537) 0.9721(0.2299) 0.9302(0.1980) 1.0005 (0.7316) 3 0.9639(0.3265) 0.9641(0.2095) 1.0239 (0.6973) 0.8600(0.1826) 0.9675(0.1728) 0.9648(0.3177) 0.9988(0.4433) 4 0.9799(0.4089) 0.9763(0.2908) 1.0219 (0.6911) 0.8949(0.2748) 0.9982(0.4723) 1.0334 (0.6824) 0.9936(0.3699) 5 0.9880(0.4493) 0.9806(0.3275) 1.0453 (0.8714) 0.9539(0.4149) 1.0188 (0.7898) 1.1246 (0.9429) 1.0051 (0.5280) 6 0.9944(0.4781) 0.9746(0.2888) 1.0619 (0.9254) 0.9128(0.3294) 1.0295 (0.9148) 1.1728 (0.9636) 0.9876(0.3749) 7 0.9981(0.4931) 0.9616(0.2044) 1.0455 (0.8855) 0.8936(0.2806) 1.0242 (0.8714) 1.1912 (0.9574) 0.9843(0.3699) 8 1.0007 (0.5023) 0.9448(0.1356) 1.0077 (0.6081) 0.8853(0.2497) 1.0211 (0.8359) 1.1814 (0.9395) 0.9745(0.3156) 9 1.0067 (0.5202) 0.9287(0.0998) 0.9643(0.1599) 0.8993(0.2573) 1.0230 (0.8584) 1.1653 (0.9075) 0.9684(0.2944) 10 1.0097 (0.5268) 0.9285(0.1245) 0.9283(0.0680) 0.9181(0.2869) 1.0150 (0.8131) 1.1230 (0.8240) 0.9588(0.2626) 11 1.0092 (0.5238) 0.9003(0.0753) 0.9104(0.0506) 0.9399(0.3357) 1.0084 (0.7847) 1.0942 (0.7448) 0.9458(0.2402) 12 1.0062 (0.5153) 0.8486(0.0143) 0.9067(0.0443) 0.9550(0.3762) 1.0058 (0.7745) 1.0807 (0.7019) 0.9421(0.2400) 13 1.0039 (0.5093) 0.8314(0.0130) 0.9075(0.0397) 0.9677(0.4109) 1.0097 (0.9288) 1.0802 (0.6962) 0.9387(0.2412) 14 1.0042 (0.5098) 0.8154(0.0143) 0.9103(0.0358) 0.9759(0.4343) 1.0071 (0.8714) 1.0856 (0.6962) 0.9351(0.2483) 15 1.0051 (0.5112) 0.8071(0.0218) 0.9115(0.0269) 0.9873(0.4658) 1.0058 (0.8723) 1.1114 (0.7393) 0.9351(0.2609) 16 1.0065 (0.5136) 0.7977(0.0288) 0.9146(0.0212) 0.9992(0.4979) 1.0058 (0.9221) 1.1537 (0.7952) 0.9309(0.2571) 17 1.0093 (0.5187) 0.8001(0.0428) 0.9252(0.0269) 1.0127 (0.5329) 1.0075 (0.9430) 1.1899 (0.8282) 0.9341(0.2706) 18 1.0138 (0.5374) 0.7970(0.0534) 0.9427(0.0981) 1.0246 (0.5630) 1.0080 (0.9221) 1.2369 (0.8654) 0.9326(0.2756) 19 1.0183 (0.5335) 0.7978(0.0701) 0.9537(0.0803) 1.0354 (0.5888) 1.0090 (0.9388) 1.2893 (0.8895) 0.9285(0.2732) 20 1.0214 (0.5374) 0.8003(0.0896) 0.9585(0.1409) 1.0451 (0.6099) 1.0093 (0.9462) 1.3316 (0.8932) 0.9251(0.2695) 21 1.0235 (0.5393) 0.8040(0.1099) 0.9591(0.1535) 1.0525 (0.6227) 1.0078 (0.9428) 1.3747 (0.8912) 0.9307(0.2814) 22 1.0222 (0.5358) 0.7905(0.1064) 0.9595(0.1600) 1.0614 (0.6367) 1.0065 (0.9065) 1.3983 (0.8816) 0.9316(0.2890) 23 1.018 (0.5284) 0.7823(0.1122) 0.9660(0.2039) 1.0669 (0.6443) 1.0065 (0.9060) 1.4237 (0.8751) 0.9208(0.2707) 24 1.0127 (0.5198) 0.7542(0.0816) 0.9763(0.2812) 1.0734 (0.6543) 1.0041 (0.8314) 1.4572 (0.8758) 0.9181(0.2701) Average 0.9990(0.4737) 0.8613(0.1076) 0.9723(0.3403) 0.9651(0.4145) 1.0052 (0.7814) 1.1740 (0.7715) 0.9583(0.3482) Note: Bold values denote an improvement relative to the no-change benchmark. P-values in parenthesis are based on the test of Clark and West (2007). The average across all 24 horizons is presented to assist the reader in summa-rizing the overall performance of the models and the average p-values should not be interpreted as anything other than a summary measure.

Subsequently, a systematic bias of the forecasts will be examined based on the 1-, 12-, 18- and 24-month forecast horizons. For this purpose, the following test equation was used:

Yt = β0+β1Ytf+h+ut+h, (6) whereYt is the real price of oil andYtf+h are the respective forecasts of the individual models.

A good forecast under the given loss function should have no bias on average (H0 : β0 = 0) and should have a similar volatility (H0 : β1 = 1). However, a rejection of the null hypothesis for β0 would lead to the conclusion of too high (β0 > 0) or too low (β0 < 0) forecasts on average whereas values less than one for β1 would indicate a higher volatility of the fore-casts on average. Table 2.4 summarize the forecast efficiency and asymptotic p-values for the t- and Wald-tests based on HAC standard errors which are used to account for the possible autocorrelation ofut+hfor horizons greater than one month.

TABLE2.3: Recursive success ratios relative to the no-change benchmark

Horizon ARMA Futures Inventories VAR VAR CRB Oil

OECD CLI Steel Index Stock

1 0.52 0.64 0.53 0.53 0.51 0.55∗∗ 0.56

2 0.54 0.58 0.47 0.53 0.49 0.52 0.36

3 0.50 0.53 0.52 0.57 0.52 0.57 0.50

4 0.51 0.58 0.47 0.58 0.50 0.59 0.47

5 0.50 0.53 0.46 0.55∗∗ 0.47 0.56 0.46

6 0.51 0.54∗∗ 0.49 0.54∗∗ 0.49 0.57 0.48

7 0.50 0.57 0.53 0.57 0.48 0.58 0.47

8 0.49 0.54∗∗ 0.56∗∗ 0.56 0.53 0.58 0.49

9 0.50 0.59 0.58 0.55∗∗ 0.54 0.57 0.50

10 0.50 0.59 0.58 0.55∗∗ 0.52 0.63 0.50

11 0.53 0.62 0.57 0.56 0.52 0.63 0.53

12 0.52 0.63 0.62 0.56∗∗ 0.50 0.63 0.51

13 0.52 0.61 0.59 0.54∗∗ 0.49 0.59 0.49

14 0.53 0.61 0.59 0.54∗∗ 0.50 0.62 0.50

15 0.53 0.60 0.61 0.55∗∗ 0.52 0.64 0.49

16 0.55∗∗ 0.62 0.57 0.57 0.51 0.63 0.51

17 0.56∗∗ 0.61 0.59 0.55∗∗ 0.47 0.61 0.51

18 0.55 0.62 0.59 0.54 0.49 0.60 0.48

19 0.55∗∗ 0.61 0.56∗∗ 0.55 0.48 0.58 0.50

20 0.54 0.61 0.56∗∗ 0.52 0.46 0.58 0.47

21 0.55∗∗ 0.61 0.56∗∗ 0.54 0.47 0.55∗∗ 0.46

22 0.55∗∗ 0.60 0.57 0.51 0.46 0.54 0.47

23 0.54 0.63 0.58 0.52 0.50 0.52 0.46

24 0.54 0.56 0.52 0.52 0.50 0.52 0.49

Average 0.52 0.59∗∗ 0.55 0.55 0.50 0.58∗∗ 0.49

Note: Bold values denote a significance level of P<0.01, ** and * 0.05 and 0.1, respectively based on the directional accuracy test by Pesaran and Timmermann (1995). The average across all 24 horizons is presented to assist the reader in summarizing the overall perfor-mance of the models and the average p-values should not be interpreted as anything other than a summary measure.

All models have no systematic distortion for 1-month ahead forecasts. This is different for greater forecast horizons. For 12-, 18- and 24-month horizons, all the examined models pro-duce significantly too high forecasts on average except for the ‘ARMA’ model at 24 months.

The results for the volatility are mixed for longer horizons between 18 and 24 months. While the ’Inventories OECD’, ’Var Steel’, ’CRB Index’ and ’Oil Stock’ model seem to underestimate the volatility on average, all other models are either closer to the desired level or overstate it on average. However, one has to reject the joint null hypothesis for all the models for horizons greater than one month. This leads to the conclusion, that it is possible to find a model that is better than a no-change benchmark for each particular horizon but that all the considered models have the disadvantage of being systematically biased at least at horizons greater than one month.

2.4.2 Individual Model Performance Over Time

This section addresses the idea of time-variation in the forecast accuracy for the considered models. Therefore, the MSPE ratios of the described models will be estimated using a 60-months window and plotted over time. This allows for a better illustration of the time-variation in the forecast performance than using a recursive approach where the MSPE ratio would still depend on forecasts of the beginning of the evaluation period (Manescu and van Robays 2016).

Hence, a value lower than one states that the particular model outperforms the benchmark on average over the last 60 months and vice versa for values greater than one. The results are

TABLE2.4: Forecast efficiency

1 Month Horizon RW ARMA Futures Inventories VAR VAR CRB Oil

OECD CLI Steel Index Stock

β0 0.2962 0.0934 −0.0140 0.3120 0.2439 0.4332 0.3710 0.3314

H0:β0=0 0.0796 0.3194 0.4701 0.0788 0.1073 0.0154 0.0384 0.0698

β1 0.9879 1.0013 0.9910 0.9868 0.9942 0.9811 0.9853 0.9885

H0:β1=1 0.1404 0.4502 0.1824 0.1353 0.2804 0.0331 0.0961 0.1734

H0:β0=0,β1=1 0.3339 0.4392 0.0628 0.3330 0.2573 0.0960 0.1622 0.2000 12 Month Horizon

β0 4.8127 2.1842 3.8807 4.2386 3.5315 4.8609 6.3823 3.7167

H0:β0=0 0.0000 0.0112 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000

β1 0.7966 0.9767 0.8634 0.8182 0.9012 0.7928 0.7300 0.8756

H0:β1=1 0.0000 0.3344 0.0014 0.0000 0.0083 0.0000 0.0000 0.0035

H0:β0=0,β1=1 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 18 Month Horizon

β0 6.6320 3.1715 5.4134 6.3033 5.2930 6.7253 8.9748 5.0379

H0:β0=0 0.0000 0.0017 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000

β1 0.7289 0.9700 0.8446 0.7397 0.8619 0.7242 0.6218 0.8314

H0:β1=1 0.0000 0.3113 0.0003 0.0000 0.0027 0.0000 0.0000 0.0002

H0:β0=0,β1=1 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 24 Month Horizon

β0 6.7910 1.4691 4.8310 6.7817 4.8703 6.8523 11.2475 4.6521

H0:β0=0 0.0000 0.1343 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000

β1 0.7600 1.1206 0.9335 0.7536 0.9428 0.7569 0.5451 0.8818

H0:β1=1 0.0000 0.0550 0.0919 0.0000 0.1891 0.0000 0.0000 0.0112

H0:β0=0,β1=1 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 Note: Asymptotic p-values based on HAC standard errors.

summarized in Figure 2.1 for the individual models again for the 1-, 12-, 18- and 24-month forecasting horizon. First, when comparing the forecasting performance of the different mod-els relative to the no-change benchmark, one can conclude that none of the modmod-els consistently outperforms the no-change forecast over time as well as across the different horizons.11 Partic-ularly apparent is that the two estimated VAR models, the ‘ARMA’ and the ‘CRB Index’ model have great problems over longer horizons when the market for oil is not in a calm situation.

Nevertheless, at each point in time, there is at least one different model that works well.

There is an interesting pattern that emerges when comparing the ‘Futures’ model for hori-zons greater than one month. Even though, the ‘Futures’ model was better on average over the complete sample horizon, there exists a time period between 2004 and 2009 where the ‘Invento-ries OECD’ and the ‘CRB Index’ models outperform the ‘Futures’ model on average. However, we have to keep in mind that this graph shows the average performance over the preceding 60 months. Hence, this is mainly the ‘relatively calm market’ period described by Fan and Xu (2011) between 2000 and 2004. This reverses during the period of rapidly increasing prices be-tween 2004 and 2008 and beyond as the ‘Futures’ model outperforms all the other models.

These structural breaks can be best seen in Figure 2.1 (a) and explain the clearly visible breaks in this graph as the oil price is shifting from the ‘relatively calm market’ to the rapid increase in prices over to the post financial crisis with a delay of 60 months each. In addition, the ‘VAR’

model seems to capture different dynamics when compared to the three alternative VAR mod-els. While the latter three face increasing MSPE ratios, the classic model points in the other

11 The large MSPE ratios in the beginning of the sample can be explained due to the Gulf War shortly before the begin of the evaluation period.

FIGURE2.1:Time-VariationofrecursiveMSPEratiowitha1-,12-,18-and24-monthforecastinghorizon 969798990001020304050607080910111213141516170.5

1

1.5

2 Futures Inventories OECD CRB Index Oil Stock ARMA VAR CLI VAR Steel Best4 (a)

969798990001020304050607080910111213141516170.5

1

1.5

2 Futures Inventories OECD CRB Index Oil Stock ARMA VAR CLI VAR Steel Best4 (b) 969798990001020304050607080910111213141516170.5

1

1.5

2 Futures Inventories OECD CRB Index Oil Stock ARMA VAR CLI VAR Steel Best4 (c)

969798990001020304050607080910111213141516170.5

1

1.5

2 Futures Inventories OECD CRB Index Oil Stock ARMA VAR CLI VAR Steel Best4 (d) Note:ThisgraphshowstherecursiveMSPEratiosestimatedoverarollingwindowwithalengthof60monthsforallindividualmodelsandthebest4modelcombination. Panel(a)showsthe1-,(b)the12-,(c)the18-and(d)the24-monthforecastinghorizon.

direction. This effect reverses during the financial crisis. Nevertheless, none of the four varia-tions seem to capture the price decline during the middle of 2014 well. All things considered, one can conclude that there might exist a combination of models that could be beneficial for the forecasting performance.