Appendix 1.B Estimation of the Model for Qualitative Expectations
2.3 Empirical Applications
2.3.3 Disagreement and the Business Cycle
Figure 2.3.4 plots the quantified disagreement measure t4,RW and benchmark disagree-ment. The shaded regions represent periods of economic recession as dated by the Na-tional Bureau of Economic Research (NBER) for the U.S. and by the Organization for Economic Co-operation and Development (OECD) for Sweden. Evidently, both disagree-ment measures rise during the recessions and t4,RW tracks the benchmark very closely.
To further illustrate the association between disagreement and macroeconomic variables, we report their correlations in Table 2.3.3. First, all variants of the probability approach that use time-varying thresholds suggest a very strong relationship between disagreement and the level of inflation. This relation is weaker for measures based on constant thresh-olds and even weaker for the measures of nominal or ordinal variation. This weakness again casts doubt on their reliability. Second, while large changes in the inflation rate (in either direction) tend to be associated with increased disagreement in the short-run inflation expectations from the Michigan survey, there is no such pronounced relationship for the two remaining series. Finally, disagreement appears to rise during periods in which industrial production declines, suggesting that disagreement might be useful in signaling upcoming structural and temporal changes in the economy. In sum, our analysis shows that consumer disagreement about the future path of inflation tends to (i) rise with the
13In Table 2.A.2, we report the results with and without making this adjustment.
Figure 2.3.4: Disagreement and the business cycle
(a) U.S. Short-Run
0 1 2 3 4 5 6 7 8 9 10
1982M03 1986M03 1990M03 1994M03 1998M03 2002M03 2006M03 2010M03 recession (NBER) t4, RW STDEV
(b) U.S. Long-Run
0 1 2 3 4 5 6 7
1990M04 1994M04 1998M04 2002M04 2006M04 2010M04
recession (NBER) t4, RW STDEV
(c) Swedish Short-Run
0 1 2 3 4 5 6 7
2001M11 2005M11 2009M11
recession (OECD) t4, RW STDEV
Table 2.3.3: Disagreement and macroeconomic variables: simple correlations
(1) level of inflation (2) squared change in inflation (3) industrial production growth
US SR US LR Sw.SR US SR US LR Sw.SR US SR US LR Sw.SR
(A) Measures of nominal or ordinal variation
IQV -0.33∗∗∗ -0.01 -0.74∗∗∗ 0.41∗∗∗ -0.31∗∗∗ 0.07 -0.43∗∗∗ -0.60∗∗∗ -0.30∗∗∗
BL -0.30∗∗∗ 0.02 -0.66∗∗∗ 0.44∗∗∗ -0.31∗∗∗ 0.05 -0.51∗∗∗ -0.61∗∗∗ -0.36∗∗∗
COV -0.30∗∗∗ 0.02 -0.64∗∗∗ 0.44∗∗∗ -0.30∗∗∗ 0.05 -0.54∗∗∗ -0.60∗∗∗ -0.37∗∗∗
Reardon -0.29∗∗∗ 0.03 -0.64∗∗∗ 0.43∗∗∗ -0.32∗∗∗ 0.04 -0.52∗∗∗ -0.62∗∗∗ -0.37∗∗∗
BES -0.25∗∗∗ 0.05 -0.59∗∗∗ 0.44∗∗∗ -0.32∗∗∗ 0.03 -0.54∗∗∗ -0.62∗∗∗ -0.38∗∗∗
(B) Probability approaches with constant thresholds
N 0.30∗∗∗ 0.15∗∗ 0.32∗∗∗ 0.05 0.09 -0.07 -0.29∗∗∗ 0.06 -0.28∗∗∗
t2 -0.04 0.14∗∗ -0.31∗∗∗ 0.39∗∗∗ -0.11∗ -0.01 -0.64∗∗∗ -0.31∗∗∗ -0.42∗∗∗
t4 0.11∗∗ 0.16∗∗∗ -0.13 0.28∗∗∗ -0.02 -0.03 -0.55∗∗∗ -0.15∗∗ -0.41∗∗∗
t6 0.18∗∗∗ 0.16∗∗∗ -0.01 0.21∗∗∗ 0.01 -0.04 -0.48∗∗∗ -0.08 -0.39∗∗∗
t8 0.21∗∗∗ 0.16∗∗∗ 0.06 0.17∗∗∗ 0.03 -0.05 -0.44∗∗∗ -0.05 -0.37∗∗∗
PU1 -0.25∗∗∗ 0.05 -0.51∗∗∗ 0.44∗∗∗ -0.32∗∗∗ 0.01 -0.55∗∗∗ -0.62∗∗∗ -0.38∗∗∗
PU2 -0.20∗∗∗ 0.05 -0.22∗∗ 0.42∗∗∗ -0.31∗∗∗ -0.04 -0.57∗∗∗ -0.62∗∗∗ -0.34∗∗∗
PU3 -0.05 0.06 0.37∗∗∗ 0.34∗∗∗ -0.31∗∗∗ -0.12 -0.55∗∗∗ -0.61∗∗∗ -0.13 PU4 0.28∗∗∗ 0.07 0.69∗∗∗ -0.04 -0.30∗∗∗ -0.13 -0.17∗∗∗ -0.61∗∗∗ 0.08
(C) Probability approaches with time-varying thresholds
N, Realiz. 0.52∗∗∗ 0.39∗∗∗ 0.44∗∗∗ -0.08 0.18∗∗∗ -0.10 0.09∗ 0.37∗∗∗ 0.22∗∗
t4, Realiz. 0.45∗∗∗ 0.43∗∗∗ 0.34∗∗∗ 0.08 0.10∗∗ -0.10 -0.09∗ 0.24∗∗∗ 0.16∗ PU3, Real. 0.35∗∗∗ 0.62∗∗∗ 0.31∗∗∗ 0.01 0.08 -0.02 0.18∗∗∗ 0.46∗∗∗ 0.16∗ N, AO 0.74∗∗∗ 0.48∗∗∗ 0.83∗∗∗ 0.01 0.06 -0.01 -0.13∗∗ 0.18∗∗∗ -0.08 t4, AO 0.69∗∗∗ 0.50∗∗∗ 0.76∗∗∗ 0.12∗∗ 0.00 0.00 -0.26∗∗∗ 0.08 -0.20∗∗
PU3, AO 0.59∗∗∗ 0.69∗∗∗ 0.64∗∗∗ 0.28∗∗∗ -0.09 0.11 -0.27∗∗∗ 0.14∗∗ -0.28∗∗∗
N, RW 0.74∗∗∗ 0.61∗∗∗ 0.87∗∗∗ 0.00 0.21∗∗∗ -0.07 -0.12∗∗ 0.20∗∗∗ 0.01 t4, RW 0.70∗∗∗ 0.63∗∗∗ 0.83∗∗∗ 0.12∗∗ 0.13∗∗ -0.08 -0.25∗∗∗ 0.06 -0.07 PU3, RW 0.61∗∗∗ 0.91∗∗∗ 0.76∗∗∗ 0.25∗∗∗ 0.28∗∗∗ 0.00 -0.23∗∗∗ 0.25∗∗∗ -0.14
(D) Combination of alternative approaches
Panel A -0.30∗∗∗ 0.02 -0.66∗∗∗ 0.43∗∗∗ -0.32∗∗∗ 0.05 -0.51∗∗∗ -0.61∗∗∗ -0.36∗∗∗
Panel B 0.07 0.14∗∗ 0.03 0.30∗∗∗ -0.17∗∗∗ -0.06 -0.56∗∗∗ -0.41∗∗∗ -0.34∗∗∗
Panel C 0.75∗∗∗ 0.70∗∗∗ 0.77∗∗∗ 0.11∗∗ 0.13∗∗ -0.04 -0.15∗∗∗ 0.26∗∗∗ -0.03 all 0.18∗∗∗ 0.38∗∗∗ 0.01 0.39∗∗∗ -0.18∗∗∗ -0.02 -0.55∗∗∗ -0.38∗∗∗ -0.39∗∗∗
1st PC 0.21∗∗∗ 0.36∗∗∗ 0.71∗∗∗ 0.39∗∗∗ -0.12∗∗ -0.03 -0.57∗∗∗ -0.31∗∗∗ 0.32∗∗∗
For definitions of the measurement approaches, see Table 2.3.2. Among the macroeco-nomic variables, (i) “level of inflation” is defined as the annual inflation rate in the case of short-run expectations (SR) and as the five-year inflation rate in the case of long-run expectations (LR), (ii) “squared changed in inflation” is defined as the squared change of the “level of inflation” relative to its value one-year ago (short-run) or five years ago (long-run), (iii) “industrial production growth” is defined as the annual (SR) or the five-year (LR) growth in the respective industrial production index. *,** and *** indicate rejections of the null hypothesis of no correlation at the 10%, 5% and 1% significance level, respectively (see Mudholkar, 2004).
level of inflation, (ii) rise with the variability of inflation and (iii) rise when real economic activity declines. Note that none of these findings necessarily reflect causality. However, we believe that they provide additional evidence on the accuracy of various measures of disagreement in qualitative expectations.
2.4 Concluding Remarks
During the past decade, many central banks and academic researchers have begun to ex-plore ways to better analyze consumer and firm expectations from survey data. Our paper contributes to this ongoing literature by providing a detailed review and assessment of the measurement of disagreement in qualitative survey data. We consider about two dozen individual approaches, which may be categorized as measures of dispersion in nominal and ordinal variables and the probability approach with alternative distributional and threshold assumptions. Using data from two household surveys that collect both qualita-tive and quantitaqualita-tive inflation expectations, we find that the accuracy of these approaches varies greatly and depends on the forecast horizon and the distribution of responses in the survey. In terms of the correlation with established measures of disagreement in point forecasts, the probability approach performs better than the measures of dispersion in nominal and ordinal variables. Within the probability approach, the variants that use time-varying thresholds tend to outperform those with constant thresholds and the ones with the t or piecewise uniform distribution tend to dominate those with the normal dis-tribution. Our results provide guidance to the growing field of applied macroeconomics research that studies the disagreement among non-professional forecasters, who typically report qualitative expectations.
Admittedly, the quantified disagreement measures are not perfect. A major question is to what extent the measures of disagreement in qualitative expectations can be improved.
Since our results suggest that respondents use time-varying categorization thresholds, one possible solution is to explicitly state the thresholds in the questionnaire. As an example, one could ask whether the respondent expects prices to go down by more than one percent/not change by more than one percent in either direction/rise by more than one percent over the next 12 months. Another alternative is to ask respondents to report their thresholds directly.
Bibliography
Andrade, P. and H. Le-Bihan (2013): “Inattentive Professional Forecasters,” Journal of Monetary Economics, 60, 967–982.
Ang, A., G. Bekaert, and M. Wei (2007): “The Term Structure of Real Rates and Expected Inflation,” NBER Working Paper.
Atkeson, A. and L. Ohanian (2001): “Are Phillips Curves Useful for Forecasting Inflation?” Federal Reserve Bank of Minneapolis Quarterly Review, 25, 2–11.
Bachmann, R., S. Elstner, and E. Sims(2013): “Uncertainty and Economic Activity:
Evidence from Business Survey Data,” American Economic Journal: Macroeconomics, 5, 217–249.
Batchelor, R. A. (1986): “Quantitative v. Qualitative Measures of Inflation Expecta-tions,” Oxford Bulletin of Economics and Statistics, 48, 99–120.
Batchelor, R. A. and A. B. Orr (1988): “Inflation Expectations Revisited,” Eco-nomica, 55, 317–331.
Blair, J. and M. Lacy (2000): “Statistics of Ordinal Variation,” Sociological Methods and Research, 28, 251–280.
Bloom, N. (2009): “The Impact of Uncertainty Shocks,” Econometrica, 77, 623–685.
Breitung, J. and M. Schmeling (2013): “Quantifying Survey Expectations: What’s Wrong with the Probability Approach?” International Journal of Forecasting, 29, 142–
154.
Carlson, J. A. and M. J. Parkin (1975): “Inflation Expectations,” Economica, 42, 123–138.
Carroll, C. D. (2003): “Macroeconomic Expectations of Households and Professional Forecasters,” The Quarterly Journal of Economics, 118, 269–298.
Coibion, O. and Y. Gorodnichenko (2012): “What Can Survey Forecasts Tell Us about Information Rigidities?” Journal of Political Economy, 120, 116–159.
Curtin, R. (1996): “Procedure to Estimate Price Expectations,” Manuscript, University of Michigan Survey Research Center.
——— (2007): “Consumer Sentiment Surveys: Worldwide Review and Assessment,” Jour-nal of Business Cycle Measurement and AJour-nalysis, 3, 7–42.
Dasgupta, S. and K. Lahiri (1992): “A Comparative Study of Alternative Methods of Quantifying Qualitative Survey Responses Using NAPM Data,” Journal of Business
& Economic Statistics, 10, 391–400.
——— (1993): “On the Use of Dispersion Measures from NAPM Surveys in Business Cycle Forecasting,” Journal of Forecasting, 12, 239–253.
Dovern, J., U. Fritsche, and J. Slacalek(2012): “Disagreement among Forecasters in G7 Countries,” Review of Economics and Statistics, 94, 1081–1096.
Dräger, L. and M. Lamla (2012): “Updating Inflation Expectations: Evidence from Micro-Data,” Economics Letters, 117, 807–810.
Ehrmann, M., S. Eijffinger, and M. Fratzscher (2012): “The Role of Central Bank Transparency for Guiding Private Sector Forecasts,” The Scandinavian Journal of Economics, 114, 1018–1052.
Faust, J. and J. H. Wright (2013): “Forecasting Inflation,” inHandbook of Economic Forecasting II, ed. by G. Elliott and A. Timmermann, Elsevier, 2–56.
Gibbs, J. P. and D. L. Poston (1975): “The Division of Labor: Conceptualization and Related Measures,” Social Forces, 53, 468–476.
Hubert, P.(2013): “FOMC Forecasts as a Focal Point for Private Expectations,” Journal of Money, Credit and Banking, Forthcoming.
Koopman, S. J.(1997): “Exact Initial Kalman Filtering and Smoothing for Nonstation-ary Time Series Models,” Journal of the American Statistical Association, 92, 1630–
1638.
Kvalseth, T. (1995): “Coefficients of Variation for Nominal and Ordinal Categorical Data,” Perceptual and Motor Skills, 80, 843–847.
Lahiri, K. and Y. Zhao (2014): “Quantifying Survey Expectations: A Critical Review and Generalization of the Carlson-Parkin Method,” International Journal of Forecast-ing, forthcoming.
Lamla, M. J. and T. Maag (2012): “The Role of Media for Inflation Forecast Dis-agreement of Households and Professional Forecasters,” Journal of Money, Credit and Banking, 44, 1325–1350.
Löffler, G. (1999): “Refining the Carlson-Parkin Method,” Economics Letters, 64, 167–171.
Maag, T. (2009): “On the Accuracy of the Probability Method for Quantifying Beliefs about Inflation,” KOF Working Paper No. 230.
Mankiw, N. G. and R. Reis (2002): “Sticky Information versus Sticky Prices: A Proposal to Replace the New Keynesian Phillips Curve,” The Quarterly Journal of Economics, 117, 1295–1328.
Mankiw, N. G., R. Reis, and J. Wolfers(2004): “Disagreement about Inflation Ex-pectations,” in NBER Macroeconomics Annual 2003, ed. by M. Gertler and K. Rogoff, MA: MIT Press, 209–248.
Mudholkar, G. S. (2004): “Fisher’s Z-Transformation,” in Encyclopedia of Statistical Sciences, John Wiley & Sons, Inc.
Nardo, M. (2003): “The Quantification of Qualitative Survey Data: A Critical Assess-ment,” Journal of Economic Surveys, 17, 645–668.
Pesaran, M. H. and M. Weale (2006): “Survey Expectations,” in Handbook of Eco-nomic Forecasting I, ed. by G. Elliott, C. W. Granger, and A. Timmermann, North-Holland, 715–776.
Reardon, S. F. (2009): “Measures of Ordinal Segregation,” Research on Economic In-equality, 17, 129–155.
Smith, J. and M. McAleer (1995): “Alternative Procedures for Converting Qual-itative Response Data to QuantQual-itative Expectations: An Application to Australian Manufacturing,” Journal of Applied Econometrics, 10, 165–185.
Theil, H. (1955): “Recent Experiences with the Munich Business Test: An Expository Article,” Econometrica, 23, 184–192.