Sensitivity analysis and caveats of the approach

In this section, we assess the sensitivity of the calculated vulnerability threshold to some of the choices we made in deriving it, and discuss the main limitations of the approach.

Sensitivity to the chosen time-frame and macroeconomic conditions

While vulnerability to poverty is a forward looking concept, the measurement of vulnerability is necessarily backward looking (Cafiero & Vakis, 2006). Consequently, our analysis implicitly assumes that the economic conditions which determined vulnerability in the past remain unchanged in the present and the future. This may not be true if there were important changes in the macroeconomic and/or institutional environment affecting poverty risks. Thus, while the vulnerability threshold is absolute in the sense that the risk of being poor is fixed at 10 per cent, the monetary value associated with it is likely to shift as conditions change.¹⁹

19 Similarly, to apply the vulnerability threshold to measuring the middle class in countries other than the country in which it was calculated is to assume that the macroeconomic conditions in both countries are the same, which may be even more problematic. For a discussion of approaches to defining and measuring the global middle class see Jayadev, Lahoti, and Reddy (2015).

44 Chapter 2

Nevertheless, as shown in Table 2.7, we find that the middle class as of 2008 remained relatively resilient to poverty over the three subsequent survey waves. Specifically, while 8.9 per cent of the initial middle class were observed to be poor in 2012, a somewhat larger share of 9.4 per cent had fallen into poverty in 2010/11. The fact that this difference is relatively small – in spite of the effects of the 2008/2009 financial crisis that were probably felt strongest during this period – increases our confidence in the economic stability of the identified middle class. Furthermore, in line with the drop in poverty between NIDS waves 3 and 4, only 7 per cent of the 2008 middle class were observed to be poor in 2014/15.

Accordingly, if we had calculated the vulnerability threshold using 2008 to 2014/15 as the reference period, a substantially lower threshold of R2,379 would have been estimated.

However, this difference may at least partly be attributable to sampling issues.

Finally, it is worth noting that even those in the “stable” middle class can face a higher cumulative risk of poverty. In fact, 19.4 per cent of those who were middle class in 2008 had fallen into poverty in at least one out of three subsequent survey waves (see Table 2.7).

However, they were very unlikely to remain poor persistently. That is, among the initial middle class, only 3.6 per cent were observed to be poor in all three following survey waves.

Table 2.7 Population share falling into poverty by initial class status in 2008

Class in 2008 Poor Vulnerable Middle Class Elite

Poor in 2010/11 (%) 86.84 48.77 9.44 4.20

Poor in 2012 (%) 83.88 45.89 8.88 4.05

Poor in 2014/15 (%) 73.24 35.97 7.02 3.28

Poor in 2010/11, 2012, OR 2014/15 (%) 96.99 67.07 19.43 7.87

Poor in 2010/11, 2012, AND 2014/15 (%) 61.07 22.40 3.53 1.52

Sensitivity to the choice of the probability cut-off

The choice of the probability cut-off is clearly critical to our analysis. We follow López-Calva and Ortiz-Juarez (2014) in fixing the threshold at 10 per cent. However, it would have been in principle possible to make an argument for the use of any other probability threshold in a potential range of, for example, 5 to 20 per cent.

Table 2.8 compares the calculated vulnerability line and the associated size of the middle class using a range of alternative cut-off points. We find a sharp drop in the derived vulnerability threshold particularly over the probability interval between 10 and 15 per cent – from R2,794 to R1,991. Using this latter threshold, the population share held by the middle class in 2008 would have been about six percentage points larger.

Table 2.8 Sensitivity of the vulnerability thresholds to the choice of the probability cut-off

Sensitivity to the choice of the poverty line

Naturally, the estimated probability of being poor and the associated vulnerability cut-off will depend on where we set the poverty line. In order to investigate the sensitivity of our results to the choice of Stats SA’s UBPL (2015), we first replicate the same approach using the UBPL that was recently suggested by Budlender et al. (2015). The use of this slightly higher poverty line results in a somewhat higher vulnerability threshold of R3,053 per month ($12.7 a day). While the population share held by the middle class is about one percentage point lower under this definition, its growth from 1993 to 2008 is very similar across specifications.

Second, we replicate our analysis using Stats SA’s LBPL (2015). Using this lower poverty line, a substantially lower vulnerability threshold of R1,830 and upper threshold of R4,794 are estimated, equivalent to an approximate range of $7.6 to $20.0 a day (in 2005 PPPs). Using these thresholds, the relative size of the middle class is surprisingly similar to the original specification using the UBPL. However, if we held the upper threshold fixed, then the share of the middle class would increase by close to 50 per cent under this specification (to 18.7 per cent in 1993 and 22.7 per cent in 2008).

Table 2.9 Sensitivity of the vulnerability thresholds to the choice of the poverty line

Poverty Line in January

Certainly, it is important to keep in mind that the interpretation of the group that is identified using the LBPL is qualitatively different. As explained in Section 2.2, individuals with a per capita expenditure level equal to the LBPL will need to sacrifice some food consumption in order to fulfil their non-food needs. In consequence, despite being at low risk of falling under the minimum subsistence level, the middle class that we identify in reference

46 Chapter 2

to the LBPL will still face an elevated risk of slipping into a precarious situation in the sense of not being able to purchase both adequate food and non-food items.

Sample attrition and changes in the composition of households

As in any panel data study, sample attrition is a concern. Despite using panel weights throughout the analysis, we cannot fully rule out the possibility that the derived vulnerability threshold could be affected by non-random attrition.

Out of 26,776 original sample members, 5,385 individuals (20.1 per cent) had left the sample in 2012. The attrition rate was lower among initially poor (16.8 per cent) than initially non-poor individuals (28.7 per cent) and was highest in the top decile of the expenditure distribution (43.4 per cent). Accordingly, if one suspects that those at higher risk of poverty were generally more likely to remain in the panel, our estimate of the vulnerability threshold could be upwardly biased.

In an attempt to quantify this bias, we rerun the log-linearised per capita expenditure model with data from 2012 and use the estimated regression coefficients to impute values of per capita household expenditure level and poverty status for those in the baseline sample (2008), who could not be followed up at the end of the period (2012). Since we have no information on the change in the household size and number of employed household members, in a next step we rerun the probability estimation including these imputed observations using the model specification presented in Table 2.4 column (2).²⁰ The predicted vulnerability threshold associated with a risk of poverty of 10 per cent including imputations is R3,198, as compared to R3,274 without imputations using the same model specification (see bottom of Table 2.4 column (2)). This suggests that our chosen lower middle class threshold may be upwardly biased by about R76 due to sample attrition, which would imply a minor underestimation of the size of the middle class by 0.5 percentage points.

Moreover, NIDS is a panel study which follows individuals and not households. This means that individuals that belong to a particular household in wave 1 will not necessarily belong to the same household in subsequent waves. Our analysis nevertheless works on the assumption that household resources in wave 1 are relevant in determining poverty status in wave 3, adding a limited set of controls for changes in the household composition.

Investigating to what extent dynamics of household formation as well as changes in the geographic location of households are driving mobility will be a worthwhile line of research for future investigation, which is beyond the scope of this dissertation.

20 This increases the number of observation used in the probit estimation from 17,567 to 22,152.

Measurement error in income or expenditure data

The presented vulnerability analysis essentially builds on investigating the correlates of movements into and out of poverty observed in the available panel data. However, if the expenditure variable that determines the poverty status of the household is measured with error, which will generally be the case in survey data, this noise in the data may lead us to overstate the actual degree of mobility and thus the actual risk of falling into poverty.

While a number of studies investigate the degree of income or expenditure mobility and poverty transitions using panel data, relatively few have suggested ways to correct for error bias in these analyses (see Lee, Ridder, and Strauss, 2010). The most promising attempt using NIDS data has been presented by Burger, Klasen, and Zoch (2016). Their instrumental variables approach, however, is more concerned with understanding the extent of “true”

overall income mobility in NIDS, but does not allow quantifying measurement error at the individual level. Their results suggest that up to 20 per cent of the variation in reported household income in the first three waves of NIDS is attributable to measurement error.

Replicating the above analysis for a simulated “measurement-error free” data set would be a highly valuable but not straight-forward exercise, which unfortunately is beyond the scope of the analysis presented in this dissertation.