Meta-Regression Analysis

Paying the Piper and Calling the Tune?

76

research can obtain “desired” results on seemingly scientific grounds because economic theory is ambiguous on the sign and magnitude of the employment dividend. His analysis suggests that the economic paradigm of self-interested, rational behavior should be applied to the process of economic advising itself and contracting bodies might matter for the modeling outcome.

Cross-comparisons of the simulation results on the economic impacts of environmental tax reforms have been performed previously, indicating that specific model characteristics are significant determinants of simulation results. Barker et al. (2002) use a meta-regression analysis to evaluate the literature on the economic impacts of climate policies. Bosquet (2000) provides a qualitative survey article on the empirical evidence for a second dividend. Patuelli et al. (2005) undertake a meta-analytical synthesis of simulation studies on environmental tax reforms.

Our meta-regression analysis which builds on a large pool of model-based studies on the employment effects of environmental tax reforms complements the existing literature in three respects. First, we assess more closely the role of different labor market specifications for the model-based simulation outcomes. Second, we exploit the explanatory information of the publication outcome which is not covered by observable study characteristics. Third, we follow the suggestion of Kirchgässner (2005) and test for political economy aspects of economic policy advice by classifying simulations studies according to contracting bodies:

Does he who pays the piper also call the tune?

The remainder of this paper is organized as follows. Section 4.1 presents the methodology and dataset of our meta-regression analysis. Section 4.2 provides the estimation results across model-based simulation studies on environmental tax reforms. Section 4.3 concludes.

Paying the Piper and Calling the Tune? 77 4.1.1 Methodology

We employ meta-regression analysis (MRA) to assess the influence of central study characteristics on the simulated employment effect of an environmental tax reform in which revenues are recycled for labor tax cuts or the reduction of social security contributions.

Stanley and Jarrell (1989) proposed MRA as a quantitative methodology of systematically reviewing the economic literature: Applying statistical methods, MRA may overcome biases of qualitative literature surveys.³⁸

The basic meta-regression model can be written as:

1 K

j k jk j

k

Y β Z ε

=

= Ψ +

∑

where

Ψ denotes the ‘true’ value of the parameter of interest, Yj captures the reported estimate of Ψ by the j-th study,

Zjk refers to the meta-independent variable measuring relevant characteristics of an empirical study,

βk is the meta-regression coefficient incorporating the effect of particular study characteristics k on the reported estimate, and

εj reflects the disturbance term.

The explicit goal of MRA is to explain the variation among empirical study results by central study features captured inZ_jk.

4.1.2 Dataset and estimation approach

Our dataset comprises 41 published studies on the employment double-dividend hypothesis for four selected regions: Germany, Austria, Switzerland and the aggregate European Union (EU). As a number of studies provide multiple results for alternative model assumptions, our full dataset includes 73 different study specifications (i.e. observations). Appendix 4.4.2 lists the complete set of studies for our meta-regression analysis.

38 The development of meta-analysis goes back to Glass (1976), who introduced it in the context of educational research.

Paying the Piper and Calling the Tune?

78

We start the analysis with a simple linear multiple regression model estimated by ordinary least squares (OLS). Adopting the notations of equation (1), the linear regression model reads as:

1 1 2 2 ...

j j j K Kj j

Y = +α βZ +β Z + +β Z +ε (2)

where α denotes a constant.

The variation in study outcomes Y_j on the simulated employment effect can be explained by central study characteristics captured in Z_kj. Previous analyses such as Bosquet (2000) and Patuelli et al. (2005) have discussed a number of determinants which we incorporate into our regression analysis: the stringency of the environmental policy, the time period of policy simulations, the choice of regions (countries), and the model type underlying the simulation.

In addition, we pay special attention to the role of labor market imperfections where we distinguish three alternative ways of characterizing involuntary unemployment. In this paper, the regional focus of the study is covered by two dummy variables, one for Germany and one for Austria and Switzerland (reference category: EU). Another dummy variable controls for the employed model type, i.e. the use of a macroeconometric model (reference category: CGE model). Finally, alternative specifications of imperfect labor markets are controlled for by three dummy variables; one for fixed real wage regimes, one for unionized labor markets, and one for the wage curve mechanism (reference category: perfect labor market with flexible wages).³⁹

As a central objective of this paper is to assess the role of contracting bodies for the findings of commissioned studies, the potential influence of the contracting body is captured by two additional dummy variables for the contractor who commissioned the study: One for an environmental contracting body and one for an industrial contracting body of the study (reference: no contracting body). A contracting body is classified “environmental” when being a related governmental entity (such as a ministry of environment) or an environmental non-governmental organization. It is considered “industrial” when being a related governmental body, an industrial enterprise or an industrial association. Published studies without explicit third-party funding (e.g. university studies) or studies commissioned by research-related governmental bodies were classified as “no contracting body”. A description of all variables employed in the linear regression model can be found in Table 19 of Appendix

39 The wage curve reflects empirical evidence on the inverse relationship between the level of wages and the rate of unemployment (Blanchflower and Oswald, 1994).

Paying the Piper and Calling the Tune? 79 4.4.2. Table 20 and 5 of Appendix 4.4.2 report the summary statistics for metric and dummy variables.

Besides assessing these observable determinants of publication outcomes, we additionally control for unobservable study characteristics. Following Nelson and Kennedy (2008), we include dummy variables for those studies that feature multiple results via alternative model assumptions and thus have a relatively high weight in our sample of publications (yielding in total 15 study dummies).⁴⁰ These dummy variables take over the value 1 for each simulation result of a study and 0 otherwise. They capture all explanatory information of the publication outcome that is not covered by our core regressors, thus accounting for omitted variable bias and representing unobservable study characteristics not reported consistently across the respective articles. We test for the joint significance of these additional dummy variables using the Wald test for the parameters of the correspondingly fitted regression model.

Finally, Stanley and Jarrell (1989) emphasize that meta-regression errors are likely to be heteroscedastic⁴¹ because studies may differ in the employed datasets and other characteristics. Thus, we will test for heteroscedasticity and employ robust estimation techniques.