Attrition

Attrition describes the loss of participants within a study. It can either emerge when participants fully drop out or when certain outcome or control variables are not collected for some reason. Attrition might threaten internal validity in a case where the characteristics of the participants that are lost systematically differ between the treatment and control groups and those characteristics are correlated with the outcome (so called differential attrition).

Whereas a loss of participants that is unrelated to the treatment status, i.e. exogenous, is less of a concern, the systematic loss of observations leads to a selection bias and hence possibly biased impact estimates (Duflo, Glennerster and Kremer, 2007). For instance, in our setting it might have been possible that the iron treatment causes a systematic loss of children that were already at a high cognitive and education level at the baseline because the treatment increased the cognitive ability and education outcomes of such children in a way that many of them qualified for scholarships and moved to higher quality schools. In such a case, the baseline cognitive and education outcomes for the treatment group would have been systematically lower in the samples included in our regressions, compared to the full initial sample, i.e. our estimates would be biased downward. If too many observations are lost it might further challenge the power of the estimates and the representativeness of the results (Fewtrell et al., 2008).

Generally, we reduced the number of lost participants by making all effort in the field to locate the children and their parents. Enumerators were assigned a smaller number of interviews per day such that they had sufficient time to extensively search for parents and children.³⁶ We returned to the village or schools a second time in case many children or parents were not present on the day of the survey. During the cognitive tests, which were performed at the schools, we searched for children in the village in case they were not present at the school on the day of the survey. Nevertheless, since the data was collected with three different survey teams that visited children and their parents on different days, we have several cases for which we do not have the full dataset (i.e. household, cognitive and education data for the cognitive sample and household and hemoglobin values for the anemia sample, both for the baseline and the endline surveys).

36 Parental interviews took about 20 minutes and enumerators completed seven interviews per day, on average.

We discuss attrition first for the SD estimates and then for the DD estimates. In our initial sample, 2005 children were listed for which we received parental consent and collected household data.³⁷ There were only isolated cases of households that did not agree to participate in the survey.³⁸ Out of these 2005 children, we collected hemoglobin values for 1724 children during the endline survey. These 1724 children are those that were included in the SD analysis for the hemoglobin and anemia outcomes. Hence the attrition rate from the initial sample to the endline was 14.0% (281/2005).³⁹ The equivalent attrition rate for the SD analysis for the cognitive and education outcomes is 13.2% (265/2005). These attrition rates are low in comparison to other RCTs (e.g. Banerjee et al. (2015) report an attrition rate of 10% between two follow-up surveys Glewwe et al. (2009) report 25% attrition after one year of an intervention and a bit more than 30%, two years after an intervention and Ashraf et al.

(2014) report 26% to 28% attrition in their follow-up survey). Of the 14.0% of children that were lost in the anemia sample, 47.4% belonged to the control group and 52.7% to the treatment group; and of the 13.2% lost in the cognitive and education sample, 52% belonged to the control group and 48% belonged to the treatment group. To test if the treatment caused attrition at the extensive margin, we created an attrition dummy that takes on the value of one if the child’s data on the respective outcome category was not collected at the endline. In a linear probability model, we regress the attrition dummy on treatment status. In table 3.4, we present the regression results for the different samples that we use in our specifications. None of the coefficients are found to be statistically significant and the point estimates are very small, indicating that the treatment did not cause attrition at the extensive margin.

37 This maximum number of 2005 observations is only found for household covariates. For each school, the enumerator team received a list of all children that are registered in the second grade. They built a random ranking of the children on this list and visited the households of the first 20 children on the list for the parental interview and parental consent. As some children are registered at the school but no longer live in the village, missing children were replaced by the next child on the list until the parents of 20 children were interviewed.

The 2005 children that were identified by this procedure build the basis for our sample and hence the basis for the medical, cognition and education team to collect data. In some schools, less than 20 children go to the second grade. In this case, we interviewed all children that go to the second grade but sampled more than 20 children in the next school to have an average of 20 children per school.

38 Only in a village that belonged to one school, were households very suspicious and did not allow our team to enter the village, so we replaced this village by another randomly selected school.

39 1686 observations out of the 1724 endline observations are the same children as in the baseline (i.e. for which we can show the balancing test in table 3.2, Panel A). The remaining 38 children were found at endline but not at the baseline for hemoglobin testing.

Table 3.4: Attrition (extensive margin) for SD samples

N: Number of observations. All estimates are based on OLS regression where treatment status is regressed on attrition. Standard errors are clustered at the school level. *, **, *** denote significance at the 10%, 5% and 1%

level, respectively.

Though attrition was equal in terms of the numbers of attrited children, it might still be the case that the lost participants differ in their characteristics causing an imbalance between the treatment and control groups. The balance of the baseline characteristics between the treatment and control groups of the samples used in the SD regressions has already been discussed in section 3.4.2. In the balancing test, attrited children are already excluded and as stated above, the characteristics are balanced between the treatment and control groups in these samples. Still, we compare the balancing tests without attrited children (Panel A, table 2) to the balancing test of the full initial sample where attrited children are included, i.e. all data that was collected at the baseline (Panel B, table 3.2). The balancing tests without attrited children and the balancing test with attrited children look nearly identical, indicating that differential attrition did not happen in the experiment. In the appendix table 3.A.3, we perform the same exercise for the cognitive and education SD sample and the picture looks very much the same.

Due to the higher data requirements, attrition is higher for the DD estimates. For the balanced panel of the anemia outcomes, we could collect the required data for 1791 children at the baseline and for 1406 children at the endline, resulting in 599 attrited observations from the initial and final samples or 29.9% (599/2005) and 385 attrited children between the baseline and endline surveys, which is equivalent to an attrition rate of 21.5% (385/1791), respectively. Though these attrition rates are much higher than for the SD estimates, they are still in the range of other RCTs (examples reported above). Attrition rates were very similar across the treatment and control groups. Out of the 29.9% of the children that were lost between the initial sample and the final sample, 49.4% were from the control group and 50.6% were from the treatment group. Of the 21.5% of the children lost between the baseline

the balanced panel of cognitive and education outcomes we had collected the required data for 1772 children at the baseline and 1395 children at the endline, meaning 610 attrited children between the initial and final samples (610/2005 = 30.4%) and 377 attrited children between the baseline and endline (377/1772 = 21.8%), respectively. For the 30.4% of the attrited children between the initial sample and the sample included in the DD analysis, 48.4% were lost in the control and 51.6% in the treatment group. For the 21.8% of attrited children between the baseline and the endline, 46% were in the treatment group and 54% in the control group. To test if the treatment caused attrition, we regress the attrition dummy on the treatment status in a linear probability model. In table 3.5 we present the regression results for attrition from the initial sample to the endline sample and in table 3.6 for attrition from the baseline sample to the endline sample. None of the coefficients are found to be statistically significant at conventional levels and the point estimates are very small, showing that treatment did not cause attrition at the extensive margin in the DD estimates.

Table 3.5: Attrition (extensive margin) for DD samples (from the initial sample to endline)

N: Number of observations. All estimates are based on OLS regression where treatment status is regressed on attrition. Standard errors are clustered at the school level. *, **, *** denote significance at the 10%, 5% and 1%

level, respectively.

Table 3.6: Attrition (extensive margin) for DD samples (from baseline to endline)

(1)

N: Number of observations. All estimates are based on OLS regression where treatment status is regressed on attrition. Standard errors are clustered at the school level. *, **, *** denote significance at the 10%, 5% and 1%

level, respectively.

How far attrition in the DD estimates imbalanced the samples used for our analysis is reflected in table 3.3. The balance of the characteristics between the treatment and control groups has already been discussed in section 3.4.3. In the balancing test, attrited children are already excluded and as stated above, the characteristics are balanced between the treatment and control groups in these samples. Still, we compare the balancing tests without attrited children (Panel A) to the balancing test of the full baseline sample (Panel B), as well as the full initial sample. The full initial sample has already been show in the SD attrition tables and can hence be found in Panel B of table 3.2. We therefore compare panel A to panel B in table 3.3 to analyze the attrition between the baseline and endline and Panel A in table 3.3 to Panel B in table 3.2 to analyze the attrition between the initial sample and the sample included in the DD estimates. The balancing tests without attrited children and the balancing test with attrited children look nearly identical, indicating that differential attrition did not happen in the experiment. In the appendix table 3.A.4, we perform the same exercise for the cognitive and education sample and the picture looks very much the same. We therefore conclude that differential attrition is not an issue in our study and that the sample used in the analysis is identical to the one that was initially sampled. This means that results of our analysis are still representative for students going to government-funded schools in the two blocks named above that were in the second grade at the baseline.

3.5 Results