Survey Expectations Data
Dissertation
zur Erlangung des akademischen Grades eines Doktors der Wirtschaftswissenschaften (Dr. rer. pol.)
vorgelegt von
Frieder Mokinski
an der
Sektion Politik – Recht – Wirtschaft
Fachbereich Wirtschaftswissenschaften
Tag der mündlichen Prüfung: 10. Juli 2014 1. Referent: Prof. Dr. Winfried Pohlmeier
2. Referent: Prof. Dr. Ralf Brüggemann
Konstanzer Online-Publikations-System (KOPS) URL: http://nbn-resolving.de/urn:nbn:de:bsz:352-285972
Hier möchte ich mich bei einigen Personen bedanken, ohne deren Unterstützung diese Doktorarbeit nicht entstanden wäre.
Besonderer Dank gebührt meinem Doktorvater, Prof. Dr. Winfried Pohlmeier, der mein Interesse am empirischen Arbeiten geweckt und mich während meiner Promotion immer unterstützt hat.
Prof. Dr. Ralf Brüggemann danke ich für die Bereitschaft, diese Arbeit zu begutachten und für viele hilfreiche Anregungen.
Meinen Koautoren Christoph Frey, Xuguang Sheng und Jingyun Yang danke ich für die bereichernde Zusammenarbeit.
Fabian Krüger danke ich für zahlreiche Diskussionen und seine Bereitschaft, mir immer wieder Anregungen zu meiner Forschungsarbeit zu geben.
Dem ZEW und meinen Kollegen in der Abteilung „Internationale Finanzmärkte und Fi- nanzmanagement“ danke ich für die gute Arbeitsatmosphäre und für die Unterstützung meiner Forschungsarbeit.
Jesper Riedler, Frauke Schleer, Xuguang Sheng, Anna-Lena Huthmacher und Friederike Franz danke ich für das Korrekturlesen von Teilen der vorliegenden Arbeit.
Zudem möchte ich meiner Freundin Nathalie danken, die mich - wenn es mal nicht so gut lief - zum Durchhalten ermutigt, Teile meiner Dissertation Korrektur gelesen und meine Forschungsideen mit mir diskutiert hat.
Summary 8
Zusammenfassung 10
1 Using Time-Stamped Survey Responses to Measure Expectations at
a Daily Frequency 14
1.1 Introduction . . . 15
1.2 Measurement Models . . . 16
1.2.1 Quantitative Forecasts . . . 16
1.2.2 Qualitative Forecasts . . . 18
1.3 Empirical Results . . . 20
1.3.1 Data . . . 20
1.3.2 Parameter Estimates . . . 23
1.3.3 Forecasting the Next Survey Release . . . 25
1.3.4 Illustrating the Impact of Major Events on Expectations . . . 26
1.3.5 Measuring the Impact of News Releases on Expectations . . . 30
1.4 Concluding Remarks . . . 34
Appendix 1.A Estimation of the Model for Quantitative Expectations . . . 38
Appendix 1.B Estimation of the Model for Qualitative Expectations . . . 40
2 Measuring Disagreement in Qualitative Expectations 44 2.1 Introduction . . . 45
2.2 Measuring Disagreement in Qualitative Survey Data . . . 46
2.2.1 Measures of Nominal or Ordinal Variation . . . 46
2.2.2 Probability Approach . . . 48
2.3 Empirical Applications . . . 54
2.3.1 Data Description . . . 54
2.3.2 Evaluation of Alternative Disagreement Measures . . . 60
2.3.3 Disagreement and the Business Cycle . . . 64
2.4 Concluding Remarks . . . 67
Appendix 2.A Additional Tables . . . 70
3 Forecasting with Bayesian Vector Autoregressions estimated using Professional Forecasts 74 3.1 Introduction . . . 75
3.2 VAR Estimation Using Professional Nowcasts . . . 76
3.2.1 Augmenting a VAR with Survey Nowcasts . . . 76
3.2.3 Adding Wright’s Democratic Steady-State Prior . . . 80
3.3 Empirical Application . . . 81
3.3.1 Trained Hyperparameters . . . 86
3.3.2 Survey Forecasts and Forecast Combination . . . 88
3.4 Concluding Remarks . . . 91
Appendix 3.A A set of assumptions that ensures equal coefficients . . . 92
Appendix 3.B Posterior distribution for the augmented VAR . . . 94
Appendix 3.C Posterior distribution for augmented VAR with a democratic steady-state prior . . . 95
Appendix 3.D Estimation under limited availability of the survey nowcasts . . 96 3.D.1 Model without Wright’s democratic steady-state prior (section 3.2.2) 97 3.D.2 Model with Wright’s democratic steady-state prior (section 3.2.3) . 98
Complete Bibliography 101
Abgrenzung 108
1.3.1 Distribution of responses across the survey period. . . 22 1.3.2 Illustrations of daily estimates around two crises . . . 31 1.3.3 Illustrations of daily estimates around a positive news event . . . 32 2.2.1 Simulated disagreement from the probability approach: Gaussian vs.
piecewise uniform distribution . . . 51 2.3.1 Short-run inflation expectations from the Michigan survey: March 1982
- April 2013 . . . 57 2.3.2 Long-run inflation expectations from the Michigan survey: April 1990
- April 2013 . . . 58 2.3.3 Short-run inflation expectations from the Swedish survey: November
2001 - April 2013 . . . 61 2.3.4 Disagreement and the business cycle . . . 65 3.2.1 Variance Ratio VR(r, ρ) . . . 78
1.3.1 Availability of directional forecasts. . . 21
1.3.2 Estimation results. . . 24
1.3.3 Forecasting Experiment 1 . . . 27
1.3.4 Forecasting Experiment 2 . . . 28
1.3.5 Event Study: Regression Results . . . 34
1.A.1 The models for point forecasts as special cases of the general model. . . 39
1.B.1 The models for qualitative forecasts as special cases of the general model. 41 2.3.1 Summary Statistics for Inflation Expectations . . . 56
2.3.2 Accuracy of alternative measures for disagreement in qualitative expec- tations . . . 62
2.3.3 Disagreement and macroeconomic variables: simple correlations . . . . 66
2.A.1 Accuracy of alternative measures based on changes in forecast disagree- ment . . . 71
2.A.2 The impact of variable adjustment on the estimated disagreement mea- sures . . . 72
3.3.1 Data description and variable transformation . . . 82
3.3.2 Prior Specifications . . . 83
3.3.3 Forecasts with Different Prior Specifications: Relative Mean Squared Forecast Errors (Evaluation Sample: 1984:Q2 - 2011:Q2) . . . 85
3.3.4 Forecasting with Trained Hyperparameters: Relative Mean Squared Forecast Errors (Evaluation Sample: 1990:Q4 - 2011:Q2) . . . 87
3.3.5 Comparing the Forecasts with both Survey Expectations and Combined Forecasts: Relative Mean Squared Forecast Errors (Evaluation Sample: 1990:Q4 - 2011:Q2) . . . 89
This dissertation consists of three essays that have a common focus on the econometrics of survey expectations data. Such data play a crucial role in economics. First, as most de- cisions depend on expectations, these data facilitate a better understanding of economic dynamics. Second, survey data provide information that is otherwise unavailable, e.g.
the economic expectations of professional forecasters or firms’ hiring, investment and pro- duction activities. This thesis contributes to the relevant literature by considering novel aspects in the measurement of expectations, and by proposing a new approach to exploit survey expectations data in forecasting. Below, I give a detailed summary of the three essays.
Chapter 1 addresses a drawback of survey-based measures of expectations. Due to the monthly or quarterly frequency of survey waves, updates are relatively infrequent. To ex- tract a daily measure, I propose a state-space method that considers time-stamped survey responses as its daily measurements and the population distribution of expectations as the latent state. To facilitate measurement at times, when no responses are observed, I aug- ment the measurement model with financial variables such as bond yields that indirectly measure expectations. This article is the first in literature to study how the variation of responses throughout survey periods can be used to extract high-frequency measures of expectations.
In an application to respondent-level data from the ZEW financial market survey, I eval- uate this new method. A forecasting experiment suggests that both time-stamped in- dividual responses and indirect measures of expectations contribute to its accuracy. To further validate the estimates, I analyze how they react to major events (e.g. the Lehman Brothers crash) and regular indicator releases (e.g. purchasing manager indexes). All analyses suggest economically plausible response patterns.
High-frequency measurement of expectations is useful both for practitioners and re- searchers. First, it allows survey collectors to present survey results in a different way, showing how expectations have changed throughout a survey period and in response to particular events. Second, it facilitates real-time economic monitoring. Central banks,
may, for example, shed new light on the frequency at which respondents revise their fore- casts and the causes of these revisions.
Chapter 2, co-authored with Xuguang Sheng (American University, Washington D.C.) and Jingyun Yang (Rush University Medical Center, Chicago), considers the measurement of disagreement in qualitative survey data. Studies in empirical macroeconomics have re- cently applied a range of different approaches for this purpose. Our analysis suggests that these measurement approaches can deliver vastly different results. We therefore seek to give guidance on choosing among them. Using the data from two household surveys that ask both for qualitative and quantitative inflation expectations, we are able to directly evaluate the different measures of disagreement in qualitative expectations against a stan- dard benchmark measure of disagreement in point forecasts, the cross-sectional standard deviation.
We find that (i) statistical measures of nominal or ordinal variation are only weakly re- lated with the disagreement among point forecasts, (ii) measures based on the classical probability approach (Carlson and Parkin, 1975) generally display moderate correlations with the benchmark, (iii) variants of the probability approach that use non-normal dis- tributions - such as the piecewise uniform proposed in this paper or the t distribution - typically perform better, and (iv) allowing for time-varying categorization thresholds in the probability approach leads to further improvement in most cases.
Chapter 3, co-authored with Christoph Frey (Universität Konstanz), proposes a Bayesian shrinkage approach for vector autoregressions (VARs) that relies on survey forecasts as non-sample information. Shrinkage is popular in forecasting applications of VARs. The reason is that even medium-sized models have several hundred parameters to estimate, which makes potential over-fitting an immediate threat to forecast accuracy. Our idea is that including the additional information provided by the survey nowcasts will help us pin down the model coefficients. Therefore, we augment the vector of dependent variables by these nowcasts, and claim through a Bayesian prior that each variable of the VAR and its nowcast are likely to depend in a similar way on the lagged dependent variables.
In an empirical application to U.S. macroeconomic and financial data, we show that our method successfully improves forecasts. Across a range of horizons and variables, we find that a VAR fitted by our approach yields smaller mean squared forecast errors than a range of benchmark methods.
Die vorliegende Dissertation umfasst drei Essays, die sich mit der ökonometrischen Analy- se umfragebasierter Erwartungsdaten befassen. Diese Daten spielen eine wesentliche Rolle in der Wirtschaftsforschung. Ein Grund hierfür ist, dass wirtschaftliches Handeln fast im- mer durch Erwartungen getrieben ist. Erwartungsdaten ermöglichen es daher, wirtschaftli- che Zusammenhänge besser zu verstehen. Hinzu kommt, dass einige wirtschaftliche Größen nicht anders gemessen werden können, beispielsweise die Erwartungen professioneller Pro- gnostiker oder die Einstellungs-, Investitions- und Produktionspläne von Unternehmen.
Die Beiträge dieser Dissertation liegen in zwei Bereichen. Zum einen werden Aspekte der Messung von Erwartungen thematisiert. Zum anderen wird eine Methode entwickelt, die sowohl Erwartungs- als auch Realisationsdaten verwendet, um bessere Vorhersagen zu erstellen. Nachfolgend gebe ich eine detaillierte Zusammenfassung der drei Essays.
Kapitel 1 befasst sich mit einem Nachteil von umfragebasierten Erwartungsdaten. Weil Erhebungen in aller Regel monatlich oder vierteljährlich stattfinden, werden Messergebnis- se verhältnismäßig selten bekanntgegeben. In dem Kapitel wird eine Methode entwickelt, um die Messfrequenz anhand der bestehenden Datengrundlage zu erhöhen. Diese Methode gründet auf einer Zustandsraumdarstellung des Messproblems: Einzelne Rückmeldungen, die anhand von Zeitstempeln einem Umfragetag zugeordnet werden können, werden hier- bei als Messvariablen betrachtet, die zur Schätzung des Zustands - der Verteilung der Erwartungen in der Gesamtpopulation am entsprechenden Tag - verwendet werden. Um die Präzision des Erwartungsmaßes gerade an solchen Tagen zu verbessern, an denen keine Erwartungsdaten erhoben werden, wird das Modell um verschiedene Variablen erweitert, die (Finanzmarkt-)Erwartungen indirekt messen (beispielsweise Zinsraten oder die Ren- dite von Staatsanleihen). Die Innovation des Ansatzes liegt in der Verwendung datierter Einzelantworten. Es handelt sich um die erste Studie in der einschlägigen Literatur, die auf dieser Basis ein hochfrequentes Erwartungsmaß schätzt.
Ich wende das Verfahren auf den ZEW-Finanzmarkttest an. Ein Prognoseexperiment zeigt, dass beide Informationsquellen - datierte Einzelrückmeldungen und indirekte Erwar- tungsmaße - für sich genommen die Präzision der Methode erhöhen. Um das Maß zusätz-
(wie Einkaufsmanagerindizes) reagiert. Dabei zeigt sich, dass die Reaktionsmuster durch- gängig erwartungsgemäß sind.
Hochfrequente Erwartungsmaße sind sowohl für die praktische Anwendung als auch für die einschlägige Wissenschaft interessant: Erstens bieten sie die Möglichkeit, Umfrageergeb- nisse in anderer Form darzustellen, z.B. indem aufgezeigt wird, wie sich Einschätzungen über einen Umfragezeitraum und als Reaktion auf ein bestimmtes Ereignis verändert ha- ben. Zweitens erleichtern sie die Überwachung des wirtschaftlichen Geschehens beispiels- weise für Notenbanken, die stark daran interessiert sind, Inflationserwartungen zeitnah zu messen. Drittens können sie eingesetzt werden, um Aspekte der Erwartungsbildung zu untersuchen, zum Beispiel, wie häufig Prognosen revidiert und wodurch Revisionen ausgelöst werden.
Das zweite Kapital, bei dem es sich um ein gemeinsames Forschungsprojekt mit Xu- guang Sheng (American University, Washington D.C.) und Jingyun Yang (Rush Univer- sity Medical Center, Chicago) handelt, beschäftigt sich mit der Messung von Uneinigkeit in qualitativen Umfragedaten. Verschiedene empirisch-makroökonomische Studien haben sich in der jüngeren Vergangenheit mit dem Phänomen Uneinigkeit befasst und zu dessen Messung auf der Basis qualitativer Daten eine Vielzahl verschiedener Ansätze gewählt.
Unsere Ergebnisse legen nahe, dass diese verschiedenen Ansätze zu sehr unterschiedli- chen Analyseergebnissen führen können. Wir versuchen daher in dieser Forschungsarbeit, eine Orientierungshilfe zur Wahl eines angemessenen Ansatzes zu geben. Indem wir die Daten zweier Haushaltsumfragen auswerten, die sowohl qualitative als auch quantitative Inflationserwartungen abfragen, können wir die Eignung der verschiedenen Ansätze direkt evaluieren. Hierzu vergleichen wir die einzelnen Meßkonzepte für qualitative Erwartungen mit der Standardabweichung der Punktprognosen, die sich als Messkonzept für diesen Pro- gnosetyp etabliert hat. In unserer Analyse zeigt sich, dass (i) statistische Dispersionsmaße für nominal- und ordinalskalierte Variablen nur schwach mit dem Vergleichsmaß korrelie- ren, (ii) der klassische Wahrscheinlichkeitsansatz von Carlson and Parkin (1975) ein etwas besseres Maß liefert, das normalerweise moderate Korrelationen aufweist, (iii) Varianten des Wahrscheinlichkeitsansatzes, die andere Verteilungsannahmen machen (z.B. eine ab- schnittsweise Gleichverteilung oder eine t-Verteilung), zu einer Verbesserung führen und (iv) im Zeitverlauf schwankende Kategorisierungsschwellen im Wahrscheinlichkeitsansatz in den meisten Fällen - gemessen an der Korrelation mit dem Vergleichspunkt - das beste Uneinigkeitsmaß liefern.
kroökonomischer und finanzwirtschaftlicher Variablen zu verbessern. Konkret befasst es sich mit der Prognosestellung anhand eines Bayesianischen Vektorautoregressionsmodels (VAR), das umfragebasierte Prognosen für das aktuelle Quartal oder den aktuellen Mo- nat (sogenannte Nowcasts) als Shrinkageinstrument verwendet. Die Idee der Shrinkage- schätzung besteht - vereinfacht dargestellt - darin, Modellschätzer zu verbessern, indem zusätzliche Informationen in den Schätzvorgang einbezogen werden. Shrinkagemethoden sind besonders in der Schätzung von VARs beliebt, da bereits mittelgroße Modelle mit 10-20 Variablen über mehrere hundert zu schätzende Parameter verfügen. Aufgrund der großen Parameteranzahl und der einhergehenden Schätzunsicherheit besteht bei VARs ein erhöhtes Risiko unpräziser Prognosen. Die Idee unseres Ansatzes ist, die Schätzunsi- cherheit durch die zusätzliche Information in den Umfrageprognosen zu verringern. Hier- zu erweitern wir den Vektor der abhängigen Variablen des VARs um umfragebasierte Kurzfristprognosen und unterstellen anhand eines Bayesianischen Priors, dass Beobach- tungsdaten und Kurzfristprognosen in ähnlicher Weise von den verzögerten abhängigen Variablen abhängen.
In einer empirischen Anwendung auf US-Amerikanische Wirtschafts- und Finanzdaten zeigen wir, dass der Ansatz tatsächlich zu besseren Vorhersagen führt. Über verschiedene Vorhersagehorizonte und Variablen hinweg, stellen wir fest, dass ein VAR, das mit unserem Ansatz geschätzt wurde, meist geringere mittlere quadrierte Prognosefehler generiert als eine Reihe von Vergleichsmethoden.
Bibliography
Carlson, J. A. and M. J. Parkin (1975): “Inflation Expectations,” Economica, 42, 123–138.
Using Time-Stamped Survey Responses to
Measure Expectations at a Daily Frequency
1.1 Introduction
Survey-based measures of expectations are a valuable source of economic data. As most decisions depend on expectations, these data facilitate a better understanding of the func- tioning of economies and provide early signals about the economic future. A major draw- back is the low frequency of survey releases. To circumvent their limited frequency, central banks (e.g. Haubrich, Pennacchi, and Ritchken, 2008) and economic research (Adrian and Wu 2009, Jochmann, Koop, and Potter 2010, Gürkaynak, Sack, and Wright 2010) have begun to rely on indirect measures of expectations. Such measures are often derived from financial variables like interest rates, which are available at the daily frequency.
In this paper, I develop a method to extract a daily measure of expectations directly from time-stamped individual survey responses. This method involves a state-space model, in which the responses of a day are the measurement variables, and the population distri- bution of expectations is the latent state. Appropriate filtering and smoothing recursions allow estimating how the population distribution of expectations has changed over time, what it is today, and what it will be in the future. To facilitate measurement at times, when no responses are observed, I augment the measurement model with financial vari- ables such as bond yields that indirectly measure expectations. This article is the first in literature to study how the variation of responses throughout survey periods can be used to extract high-frequency measures of expectations.
Using my new method, I analyze the data from a German financial sector survey. My results suggest that (1) both interview time stamps and variables that indirectly measure expectations improve the real-time estimate of the next survey release, (2) the time series of daily estimates provide plausible measures of the impact of major events such as the September 11 attacks (Figure 1.3.2a or the crash of Lehman Brothers (Figure 1.3.2b), and (3) regularly scheduled news announcements such as purchasing manager indexes (PMIs) exert a measurable impact on expectations at the daily frequency.
This paper is closely related to Ghysels and Wright (2009), who came up with the idea of measuring expectations at the daily level. Unlike my method, their two alternative approaches rely on aggregated survey data, i.e. summary statistics of the responses given in a survey period. Their first approach uses a MIDAS regression model to learn how the average response relates to preceding changes in interest rates. Using the estimated model, they predict average expectations on days with no survey responses. Alternatively, they use a state-space model in the spirit of this paper, which assumes that (1) the average forecast of a survey period is a blend of the latent expectations that were held on the days before the survey deadline date, and (2) interest rates display some co-movement with the population distribution of expectations. My paper thus extends the approach of Ghysels and Wright (2009) by showing how time-stamped survey responses can be used
to improve the inference about the daily variation in the distribution of expectations. The empirical application demonstrates that this information greatly improves the precision of the measure.
The state-space method I use in this paper is closely related to approaches that extract measures of business conditions, see e.g. Aruoba, Diebold, and Scotti (2009) for a daily measure using indicators observed at different frequencies, or Frale, Marcellino, Mazzi, and Proietti (2011) for a measure based on a large number of indicators. Similar methods can also be found in the nowcasting literature, where the forecast of a variable referring to the ongoing month or quarter is continuously updated with new information that materializes in different indicators (e.g. Camacho and Perez-Quiros, 2010).
1.2 Measurement Models
This section introduces measurement models for quantitative as well as qualitative fore- casts. In these state-space models, the responses of a day are seen as draws from the population distribution of expectations, which changes from day to day. The goal is to estimate its shape on a given day.
The state-space modeling approach is a natural choice for this problem: First, within its framework, I can easily formalize that the responses on a specific day are noisy signals about the population distribution of forecasts. Second, it allows us to account for variation in the signal precision arising from a changing number of responses per day. Third, the approach naturally overcomes the hurdle of the missing values I observe when no survey responses are collected. Fourth, I can easily augment the model with additional measurement variables such as financial asset prices and relate them to the dynamics in the population distribution of forecasts.
1.2.1 Quantitative Forecasts
Denoting the point expectation of respondent i stated on day t by yit, and the average across the responses collected on day t by y¯t = 1/ntP
iyit, I set up the following state- space model
¯
yt = µt+εt, εtiid∼ N(0, σε2/nt), (1.1) µt+1 = µt+ηt+1, ηt+1 iid∼ N(0, ση2). (1.2) Given that σ2ε and ση2 are constant, interest lies in estimating how µt, the mean of the population distribution of forecasts, changes over time. Estimation is based on Maximum Likelihood and the Kalman Filter. Its details are outlined in Appendix 1.A.
I next discuss the model: The measurement equation (eq. 1.1) implicitly assumes that the individual responses on daytare draws from a distribution with meanµtand variance σε2, such that the average response has variance σε2/nt, reflecting that the quality of the signal y¯t for µt increases with the number of responses.
The implicit assumption about the individual responses is not particularly restrictive as it is compatible with individual responses that have the following structure
yit =µt+ui+ξit,
where µt is a common trend, ui is an individual random effect with ui
iid∼ (E[ui] = 0, V[ui] = σu2), and ξit is an individual disturbance with ξit iid∼ (E[ξit] = 0, V[ξit] = σ2ξ
it).
The random effect (ui) reflects that some respondents are generally more or less optimistic about the future, thus permanently over- or under-predicting economic variables. The individual disturbances (ξit) captures that, despite the overlap between the information respondents have and the interpretations they derive from it, differences typically exist.1 From these assumptions it immediately follows that the average response y¯t has mean µt and variance σε2/nt = (σu2 +σ2ξ)/nt, as I state in equation (1.1). Moreover, approximate normality of the measurement equation (eq. 1.1) arises if either (a) ntis reasonably large according to the Lindeberg-Levy central limit theorem, or if (b) the distributions of the two disturbancesuiandξitare normal themselves. Contrary to my assumption of a serially uncorrelated measurement error εt, a random effectui may induce correlation among any pair of measurement errorsy¯t−µt = 1/ntP
(ui+ξit)andy¯t+h−µt+h = 1/ntP
(ui+ξit+h), provided that there is significant overlap in the samples collected at t and at t+h, such that the same random effects are featured in both samples. As respondents reply (at most) once in a survey period, there can be no overlap for small values of h. If the value of h implies that I compare different survey waves, provided that respondents systematically respond on specific days of the survey period, the random effect may seriously invalidate the assumption of no serial correlation in the measurement errors. As this does not hold in the data I have at hand, I conjecture that the assumption is a reasonable simplification.
The state equation (eq. 1.2) has the structure of a random walk. It reflects that with no other information at hand, my best estimate of tomorrow’s state of expectations is today’s. Using the Kalman Filter, I will revise this estimate after observing tomorrow’s responses.2 Later in this article, I will use regularly observed data such as the change in the price of certain financial assets to improve my estimate of the state of expectations.
1see the relatively recent strand of literature about forecast disagreement, e.g. Lahiri and Sheng (2008) or Mankiw, Reis, and Wolfers (2004).
2The random walk assumption does not imply that expectations are not influenced by other observable data and that I cannot learn from these data about the state of expectations.
My baseline model can be extended in several directions: One interesting avenue is to relax the assumption that the measurement error (εt) is homoscedastic and thus capture that the disagreement of expectations may change over time. Another is to relax the homoscedasticity of the state disturbance (ηt), e.g. by allowing that shocks may either come from a business-as-usual distribution with moderate variance or from an major-event distribution with a large variance.
1.2.1.1 Adding another measurement variable
Recognizing that surveys do not collect survey responses all the time, we may want to use additional data that are observed on a regular basis to estimate changes in the latent distribution of survey forecasts. The additional data I have in mind are variables that indirectly measure (market) expectations. For example, long-term bond yields impound expectations about future short-term nominal and real interest rates and about inflation.
To benefit from such indirect measures of expectations (denoted by xt), I augment the measurement equation (eq. 1.1) in the following way:
"
¯ yt xt
#
=
"
µt b×[µt−µt−1]
# +
"
εt εx,t
#
=
"
1 0 b −b
# "
µt µt−1
# +
"
εt εx,t
#
, (1.3)
and assume that the measurement error εx,t has variance σ2x and that it is serially uncor- related and uncorrelated at all leads and lags with both the measurement errorεtand the state disturbance ηt.
In equation (1.3), I assume that xt is related to the day-over-day change in µt, which implies that xt must be stationary. Non-stationary financial asset prices will thus have to be transformed to stationarity, either by taking the day-over-day price change or the day-over-day return. If I were to assume that xt loads on the level of µt, the unit root in µt (see eq. 1.2) would require that xt and y¯t are cointegrated - an overly restrictive assumption.
The way I link the additional measurement to the state variable is identical to Ghysels and Wright (2009). The idea is that the same economic news that impact on economic expectations are likely to be “impounded in financial asset prices” (Ghysels and Wright, 2009, p.504), such that the additional variable provides a noisy signal about the change in the state variable.
1.2.2 Qualitative Forecasts
Suppose instead of point forecasts, as in the case of the empirical application, I observe qualitative forecasts. Individual responses are either “will rise”, “will stay the same”, or
“will fall” and can thus be seen as draws from a multinomial distribution. Denoting by y¯t a two-element column vector holding the shares of “will rise” and “will fall” responses on day t3, I set up the following non-linear state space model:
¯ yt =
"
¯ yR,t
¯ yF,t
#
=
"
πR,t
πF,t
#
+εt, (1.4)
πt =
"
πR,t πF,t
#
=
" exp(αR,t)
1+exp(αR,t)+exp(αF,t) exp(αF,t) 1+exp(αR,t)+exp(αF,t)
#
, (1.5)
αt =
"
αR,t αF,t
#
=
"
αR,t−1
αF,t−1
#
+ηt, (1.6)
whereηti.i.d.∼ N(0,Ση),andεt is a serially independent disturbance vector with mean zero and variance
V ar[εt] = n−1t
"
πR,t(1−πR,t) −πR,tπF,t
−πR,tπF,t πF,t(1−πF,t)
#
= n−1t
"exp(αR,t)(1+exp(αF,t))
1+exp(αR,t)+exp(αF,t)
−exp(αR,t+αF,t) 1+exp(αR,t)+exp(αF,t)
−exp(αR,t+αF,t) 1+exp(αR,t)+exp(αF,t)
exp(αF,t)(1+exp(αR,t)) 1+exp(αR,t)+exp(αF,t)
#
. (1.7)
Interest lies in estimating the path of πt, the vector of the population probabilities for a “will rise” and a “will fall” response. This is a dual estimation problem as it involves the estimation of both the model parameters and the path of the state variables. Pro- vided with candidate model parameters, the Unscented Kalman Filter (e.g. Wan and Van Der Merwe, 2000) estimates the path of the state variables, which is in turn used to compute the model likelihood. Parameter estimates are obtained by maximizing the like- lihood with respect to the model parameters (maximum likelihood estimation). Details are given in Appendix 1.B.
To understand the model, note that the mean and variance of the measurement equation (1.4) follow from the assumption that the responses are independent and identically dis- tributed draws from a multinomial distribution with πR,t being the probability of a “will rise” and πF,t being the probability of a a “will fall” response.
Finally, it is interesting to note that the state-space model specified by equations (1.4- 1.7) implies that the vector of log-odds ratios with elementsln(πR,t/πS,t)andln(πF,t/πS,t), evolves according to the bivariate random walk specified in equation (1.6).
3Note, that in equation (1.4), I have not included the share of “stay the same” responses, since it is linearly determined from the two other shares as 1−1/ntP
i
1(yit=’rise’)+1(yit=’fall’)
. A similar argument holds for the probability of a “stay the same” response.
1.2.2.1 Adding another measurement variable
As for the model for point forecasts of section 1.2.1, we may want to use additional daily variables to estimate changes in the latent distribution of survey forecasts. I augment the measurement equation (1.4) in the following way:
"
¯ yt xt
#
=
"
πt
b×[(αr,t−αf,t)−(αr,t−1−αf,t−1)]
# +
"
εt εx,t
#
, (1.8)
and assume that the second measurement’s error εx,t has variance σx2, and that it is serially independent and independent at all leads and lags from the measurement error vector εt and the state disturbance vector ηt. Similar to the model for point forecasts with an additional measurement variable, xt is related to the day-over-day change in the difference of the two state variables. Thus, xt must be stationary.
1.3 Empirical Results
1.3.1 Data
I use respondent-level data from the monthly ZEW Financial Market Survey (FMS), available from December 1991. Each monthly survey interviews around 300 German financial experts, of which, according to October 2012 figures, 75% work in banks, 9%
are employed in industrial companies, and 6% are in the insurance sector.4 Contrary to e.g. the polls of Consensus Economics, individual responses remain anonymous. The response rate is high: An average of87.9%of those who respond in one survey period also respond in the next period, 71.4% respond one year later, and 33.3% still respond five years later. Questions of the FMS are qualitative5 and refer to a range of macroeconomic and financial variables. Table 1.3.1 shows which variables have been forecasted (six-month forecast horizon) for how long.6
As the timing of responses is crucial to my method, I discuss this aspect of the survey in some detail below. Each survey wave is two weeks long and starts on a Monday, either early in the same month or late in the previous month. Key results of the survey are published one day after the survey period ends, on Tuesday 11:00 a.m. CET. To report
4The remainder works in a range of other sectors.
5Forecasts are either ”will rise/improve”,”will stay the same”, or ”will fall/deteriorate”, Assessments are
”good”, ”normal”, ”bad” or ”has improved”, ”has stayed the same”, ”has deteriorated”
6Beyond the variables in table 1.3.1, the questionnaire asks for qualitative forecasts of the Brent crude oil price (since1999m02), the profit situation of several German sectors (some of them only since 1999m04), assessments of the current economic situation in all regions (since1999m01also for the Euro area), and an assessment of the change in the economic situation in Germany during the preceding six months (since 2011m01).
Table 1.3.1: Availability of directional forecasts.
Var.(↓) Country(→) GER EUR USA JAP GB FRA ITA
Current Econ. Sit. H#b
Forecast Econ. Sit. H#b
Inflation Rate H#c
3m Interbank Rate G#2 H#a G#2 G#2
10y Gov. Bond. Yie. G#3 G#3
Stock Index H#b
FX-rate vs. Eur/DM # # G#1 G#1
indicates that directional forecasts for the row-variable and column-country combi- nation were collected since 1991m12; G# indicates that the forecasts are not collected anymore; H# indicates that the forecasts were included in the questionnaire later than 1991m12; # indicates that the forecasts have never been part of the questionnaire. 1/2/3 indicate that forecasts were only raised before 1998m06/1998m12/2003m12. a/b/c indi- cate that forecasts have been raised since 1998m12/1999m01/1999m02.Forecasts for the EUR/DM-Swiss Franc exchange rate have also been collected since 1991m12.
their expectations, respondents can use an online-questionnaire or its paper version, which is returned either by e-mail or by telefax. Date and time of submission are recorded for each response.7 I use this information to compute the daily average response, which I need to estimate the models of section 1.2. In Figure 1.3.1, I have used the time stamps to depict the typical distribution of survey responses across a survey period: Though ten or more responses are typically observed on each survey day, responses tend to cluster at the start and the end of each survey period.
The monthly releases of the FMS attract a great deal of attention. The ZEW Indicator of Economic Sentiment, which is obtained from six-month forecasts of the economic situation in Germany, is the survey’s headline indicator: According to Entorf, Gross, and Steiner (2012), its release has a high-frequency impact on both the daily returns and the volatility of the German stock market index DAX. Hess and Niessen (2010) show that the indicator has a high-frequency impact on the price of government bonds. The FMS data have increasingly been used in economic research. Some academic publications that use the FMS data are Deaves, Lüders, and Schröder (2010), Dick and Menkhoff (2013), Lux (2009), Menkhoff, Rebitzky, and Schröder (2009), Nolte and Pohlmeier (2007), Nolte, Nolte, and Pohlmeier (2014) and Schmeling and Schrimpf (2011).
7I have modified the time stamp of observations that have been submitted on Saturdays or Sundays.
Specifically, I allocate these observations to the following Monday, thus arriving at a weekdays-only data set. The consideration is that formation of expectation comes to halt at weekends, because usually only little new information that could drive expectations is generated on weekend days. Note that the impact of the modification is small because responses are rarely submitted on weekend days: Only in a few cases, more than 3 weekend responses are observed in a single survey period.
Figure 1.3.1: Distribution of responses across the survey period.
Mon.[1]
Wed.[1]
Fri.[1]
Mon.[2] Wed.[2]
Fri.[2]
Mon.[3]
Tue.[3]
Tue.[1]
Thu.[1]
Tue.[2]
Thu.[2]
020406080
0 5 Day 10
10th percentile mean
90th percentile
The mean, 5th percentile and 95th percentile of the number of responses on each day of the survey period are depicted. In constructing the graph, it was required that survey periods extend over 14 or 15 days, which left 108 out of the 208 surveys waves.
1.3.2 Parameter Estimates
I estimate the measurement model for a subset of the variables listed in table 1.3.1, using three alternative variables that indirectly measure expectations. Specifically, I consider the assessment of the current economic situation, its six-month forecast, and forecasts at the same horizon for consumer price inflation and the three-month interbank rate.
For each variable I consider the survey expectations for Germany, the United States of America, and the Euro area.8 As indirect measures of expectations, I use day-over-day changes in the yield of German government bonds that mature one year ahead (denoted
∆r1;only available from September 1997) and nine to ten years ahead (denoted∆r10), and as an alternative, the day-over-day change in the log Brent crude oil price (denoted∆oil).
The U.S. equivalents of the former two series have been used by Ghysels and Wright (2009). When matching survey responses and an additional measurement, I apply the following convention: Since the additional variables are end-of-day values and responses are typically submitted before the end of the day, I take the end-of-day value of dayt−1 as the additional measurement variable xt in equation (1.8).
Table 1.3.2 summarizes parameter estimates for the state-space models for qualitative responses presented in section 1.2.2. Some aspects are worth noting. First, it turns out that σ2η1 and ση22 , the variances of the two state disturbances that govern how strongly the latent mean of the distribution of the responses varies from one day to the next, are typically lower for macroeconomic target variables (economic situation, inflation) than for financial variables (3-month interbank rates). The finding is intuitive because the former are slowly varying variables related to the business cycle, whereas financial variables tend to have higher volatilities. Second, turning to the model with an additional measurement variable, I find that parameter estimates for the state equation do not change much against the model without an additional measurement variable.9 Eventually, concerning the loadings of the additional measurement variables, it turns out that only the long-term (9-10 year) yield on German government bonds significantly loads on the state variables in all cases, whereas the two alternative additional measurements have insignificant loadings in several cases.
8The only exception is the German three-month interbank rate, which is excluded because it has not been predicted by the respondents after December 1998.
9It may seem that differences are large for the models including the yield of German government bonds with a remaining maturity of one year. It should be noted, however, that this only happens in those cases where the estimation samples differ due to the fact that the short-term Government yield is only available from September 1997.
Table1.3.2:Estimationresults. Add.Meas.Qual.Qual.+∆r10Qual.+∆oilQual.+∆r1Qual.Qual.+∆r10Qual.+∆oilQual.+∆r1Qual.Qual.+∆r10Qual.+∆oilQual.+∆r1 (1)AssessmentEconomicSituation(Germany)(2)AssessmentEconomicSituation(Unit.St.)(3)AssessmentEconomicSituation(EuroAr.) σ2 η10.00770.00770.00760.00740.00910.00890.00880.00860.00630.00610.00610.0056 σ2 η20.00860.00870.00860.00650.00800.00790.00780.00800.00620.00600.00600.0054 ρη1,η2-0.7173-0.7405-0.7383-0.7512-0.6036-0.6091-0.6026-0.6495-0.7046-0.7131-0.7091-0.6924 b0.06970.01650.11270.09140.01420.08380.06450.02710.1450 p(b6=0)∗∗∗∗∗∗∗∗∗∗∗0.0000∗∗∗∗∗∗∗∗∗∗∗ σ2 x0.00180.00050.00130.00170.00050.00140.00180.00050.0013 Nmonths245245245245245245245245160160160160 (4)ForecastEconomicSituation(Germany)(5)ForecastEconomicSituation(Unit.St.)(6)ForecastEconomicSituation(EuroAr.) σ2 η10.00570.00550.00560.00520.00410.00420.00410.00360.00490.00480.00480.0047 σ2 η20.00580.00590.00590.00610.00530.00540.00540.00540.00560.00570.00560.0057 ρη1,η2-0.6546-0.6530-0.6599-0.7035-0.4134-0.4508-0.4299-0.3820-0.6806-0.6751-0.6723-0.6764 b0.07770.00220.02920.0927-0.00230.04050.12230.01160.0414 p(b6=0)∗∗∗∗∗∗∗∗∗ σ2 x0.00180.00050.00160.00180.00050.00160.00160.00050.0016 Nmonths245245245245245245245245160160160160 (7)ForecastInflation(Germany)(8)ForecastInflation(Unit.St.)(9)ForecastInflation(EuroAr.) σ2 η10.00530.00520.00530.00580.00380.00360.00360.00410.00610.00600.00600.0062 σ2 η20.00630.00620.00610.00670.00530.00500.00500.00530.00700.00650.00640.0067 ρη1,η2-0.3113-0.3153-0.3301-0.2919-0.4121-0.4123-0.4011-0.4457-0.3047-0.3191-0.3223-0.3443 b0.11950.03700.05790.16750.05350.08110.10610.05930.0628 p(b6=0)∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗ σ2 x0.00170.00040.00160.00160.00040.00150.00170.00040.0016 Nmonths245245245245245245245245159159159159 (10)Forecast3m-InterbankRate(EuroAr.)(11)Forecast3m-InterbankRate(Unit.St.) σ2 η10.01780.01580.01620.01570.01060.00980.00980.0112 σ2 η20.02700.02580.02430.02460.02030.01980.01930.0237 ρη1,η2-0.3323-0.2918-0.3404-0.3477-0.4753-0.4792-0.4822-0.4195 b0.10200.00490.07790.10110.00820.0759 p(b6=0)∗∗∗∗∗∗∗∗∗∗∗∗ σ2 x0.00130.00050.00130.00150.00050.0013 Nmonths161161161161245245245245 Eachblockpresentsestimationresultsforaseriesofsurveyresponses.Additionalmeasurements∆r10/∆oil/∆r1aretheday-over-daychangeintheyield onGermangovernmentbondswitharemainingmaturityof9-10years/thechangeinthelogBrentcrudeoilprice/thechangeintheyieldonGerman governmentbondswitharemainingmaturityof1year.Rowp(b6=0)showstheresultsofatestforazeroloadingoftheadditionalmeasurementonthe statevariables.∗/∗∗/∗∗∗indicaterejectionatthe10/5/1percentlevel.RowNmonthsshowsthenumberofdatamonthsusedasanestimationsample.The remainingrowsrefertomodelparametersandarelabeledinthesamewayasinsection(1.2).
1.3.3 Forecasting the Next Survey Release
In this section, I evaluate if the two sources of information used to obtain a daily measure of expectations, i.e. interview time stamps and indirect measures of expectations based on financial variables, improve the precision of the estimate. As the population distribution of expectations on a given day is unobserved, I resort to an indirect strategy: I use the measurement models of section 1.2 to forecast the next survey release. As a benchmark I consider the random walk model, which predicts that the next survey release will be identical to the current.10 By comparing the forecasts of the measurement model that uses time stamps but no additional indirect measure of expectations (see section 1.2.2) with the random walk model, I figure out how the survey micro data contributes to the quality of the estimate. Additionally, I compare the forecasts of the model with and without an indirect expectations measure to assess the value of this source of information. To sum up, my results suggest that both sources of information play their distinct roles in providing a better estimate.
The pseudo real-time out-of-sample forecasting experiment has the following design: Each model is estimated using a fixed estimation sample11 comprising the data until the De- cember 2004 survey. Starting with the January 2005 survey, I estimate the probabilities of a “will rise” (πR,t)and a “will fall” response(πF,t)at each survey deadline date using an appropriate filtering algorithm (see Appendix 1.B). I use these estimates as forecasts of the shares of “will rise” and “will fall” responses reported in the following survey release, and their difference as a forecast of the reported balance statistic. The balance statistic is defined as the difference of the share of “will rise” and the share of “will fall” responses.12 Table 1.3.3 collects the results: First, I use a Diebold and Mariano (1995) test for each series and model to check whether it forecasts equally well as the random walk model, using the mean squared forecast error (MSFE) as a loss function. Restricting my focus to forecasts of the balance statistic, I find that in each case my method improves compared to the random walk in nominal MSFE terms, and that this improvement is statistically significant at the ten percent level in 29 out of the 35 cases.
10As an alternative benchmark, I have considered forecasts from an AR(1) model, which I have re- estimated each period on an expanding estimation window. The AR(1) turned out to produce inferior forecasts. Results are available upon request.
11I estimate each model only once because estimation is rather time-consuming. Using self-programmed Gauss procedures, ML estimation of the parameters of the model of section 1.2.2.1 on the full data sample typically takes between five and twenty minutes on a mid-range year-2011 notebook.
12The balance statistic is thus a concept to condense the result of a tendency survey in a single number.
Its value ranges between minus one (maximal pessimism) and plus one (maximal optimism). Coding ”will rise/improve”,”will stay the same”, and ”will fall/deteriorate” responses by+1,0, and−1respectively, the balance statistic can also be seen as the average coded response. Evidently, the balance statistic can be computed on different samples, e.g. for a single survey wave or for a specific day.
Second, I test if the forecasts obtained from models with an additional measurement variable improve over the forecasts of the model with no such variable. To address that I am comparing nested models13, I use the test proposed by Clark and West (2007). I find that including an additional measurement variable improves predictive ability only in a few cases. Specifically, forecasts of inflation expectations both for the U.S. and the Euro area benefit from adding the oil price, whereas including the yield on short- term government bonds (one year to maturity) improves forecasts for 3m-interbank rate expectations and for U.S. inflation expectations. Third, I use a Diebold and Mariano (1995) test to check whether the predictive ability differs between the appropriate model for qualitative responses (section 1.2.2) and the model for quantitative responses (section 1.2.1) applied to the balance statistic of the daily responses: In four out of seven cases, the model for qualitative responses improves significantly in comparison to the model for quantitative responses, only once does it produce significantly inferior forecasts. I conclude that it is preferable to employ the somewhat more complex model for qualitative responses.
Moreover, I conduct a second experiment, by which I judge whether indirect measures of expectations based on financial variables improve the estimate at times when no survey responses are observed. Specifically, instead of estimating the probabilities on the last day of each survey period as in the first experiment, I generate my estimates on the last day before the following survey period begins. Thus, I inject in the augmented measurement model the additional information implicit in the indirect expectations measure that has accrued after the preceding survey period has ended. Thus, if this information is useful, I expect an improvement over the model that relies only on the interview time stamps.
Table 1.3.4 shows that an additional, permanently observed measurement variable can indeed help in many more cases than suggested by the first experiment. Specifically, at the ten percent test level, the yield on long-term government bonds improves forecasts for the balance statistic for six out of seven variables. The same holds true for the oil prices in three out of seven cases, and for the yield on government with a single year to maturity in four out of seven cases.
1.3.4 Illustrating the Impact of Major Events on Expectations
An extensive literature analyzes how structural shocks affect the dynamics of an econ- omy. Theoretical models are used to make predictions about the prospective reactions of economic variables to e.g. monetary policy (e.g. Mankiw and Reis, 2002) or uncertainty shocks (e.g. Bloom, 2009). The empirical literature studies whether predicted patterns appear in real-world data (e.g. Kilian 2009, Bernanke, Boivin, and Eliasz 2005). Without
13The model with no additional measurement arises if the loadingb is zero.
