• Keine Ergebnisse gefunden

3. School Feeding, Iron-Fortified Salt and Child Cognitive Ability – Evidence from a

3.6 Challenges to Internal Validity

No or partial compliance, a situation where individuals assigned to the treatment group do not or only partly take up the treatment, or where individuals in the control group (partly) take up the treatment, can threaten the internal validity, if this behavior results in the self-selection of individuals into the treatment or control group. Partial compliance in our setting might occur at the school or the individual level. Even if partial compliance is not systematic, it reduces the power of the estimates (Duflo, Glennerster and Kremer, 2007).

At the school level, it potentially might have been the case that DFS was not or insufficiently used. This risk was strongly reduced since the intervention was approved and supported by the local government, which obliged headmasters to purchase the DFS for the

usage of the MDM. Local authorities even intervened whenever an issue of non-compliance emerged (which was only once the case). The monthly or bi-monthly delivery of DFS to the treatment schools further functioned as a monitoring system. Our team confirmed that DFS was being used as instructed. Also during the unannounced endline survey, DFS was found in all treatment schools, except for two schools where the schools had run out of stock just the day before the survey and the headmasters were not able to contact our team to deliver DFS before the next delivery date. Another potential source of imperfect compliance at the school level might have occurred, if the delivery system had not worked perfectly. At endline, we asked headmasters and cooks in treatment schools if they ever experienced a situation where not sufficient DFS was available at the school for the preparation of the MDM. Seven out of 54 headmasters reported that this situation had ever occurred; however, in two schools this was only the case for one day, in three schools for two to five days and in the other two schools this was the case for ten days.43 Since only a few schools, and only for a limited amount of time compared to a whole year of treatment, were affected by the shortage of DFS, treatment intensity can still be considered high. We also argue that this situation is close to a real-world policy. The lack of DFS emerged either because the stock of DFS was exhausted before the next delivery date and headmasters were not able to contact our team for earlier delivery, or because schools could not be reached due to monsoon flooding, which is typical of the study region and many other parts of South Asia.

One could further imagine that DFS from the treatment schools was resold to other schools, including control schools and households. However, this is very unlikely. First, because the financial incentive of reselling DFS is very small since salt is a relatively cheap product. Furthermore, we know from anecdotal evidence that awareness of the benefits of DFS, and hence the demand for it, is very low – at least among the rural households in our study region. In the parental questionnaire from the endline survey, we asked for the salt package used at the household, and if it was not available, we asked for the last salt brand that was bought. A negligible number of four households in the DD sample (0.28%) had Tata Salt Plus or reported that they had bought it, indicating that the reselling of salt to households did not happen. However, it would have been more likely that the headmasters in the treatment schools sold DFS to headmasters in the control schools, since headmasters in both control and treatment schools were informed of the study for ethical reasons. We told headmasters in the

treatment and control schools that a lottery would take place in which they would have a 50%

chance of being supplied with DFS to be used in the preparation of the MDM. They were later informed of the results of the lottery. Headmasters were further informed that the government supported this project and that they had to obey with the results of the lottery. Furthermore, headmasters in control schools were told that if the given study would yield positive results, they would also have the possibility to purchase DFS. In the endline survey, headmasters in the control schools were asked about their awareness of DFS. 29 out of 53 headmasters had heard about iron-fortified iodized salt, but only two stated that they knew that the salt was available in other schools. None of the control headmasters reported to have ever used DFS in the preparation of the MDM.44 Also, during the unheralded endline survey, no DFS was found in any of the control schools. Apparently, the prospect of the offer of DFS after the completion of the study and the order from the government to obey the study design, provided enough reason for headmasters in control schools to postpone their demand for DFS. In combination with the non-existence of an incentive for headmasters in treatment schools to sell DFS, we are very certain that partial compliance at the school level is not an issue.

Non-compliance at the child level due to low school attendance has been discussed in section 3.5.2. Non-compliance at the individual level could have potentially also occurred, if children that were enrolled in control schools at the baseline went to treatment schools to receive the treatment and the other way around. This potential bias was encountered in several ways. First, only headmasters, but not parents and children, knew about the interventions.

Anecdotal evidence further confirms that awareness of the benefits of DFS was very low, such that the incentive of parents to send their child to a treatment school to receive the DFS was non-existent. In our study region, every village generally has one government-funded school and going to a school in another village is very uncommon and the larger travel distance would have been an additional disincentive.

3.6.2 Attenuation Bias

When conducting research with human subjects, and especially with vulnerable populations such as children, it has to be ensured that the benefits of the research outweigh the risks for ethical reasons (Medical Research Council, 2004). In order to maximize the benefit for the

44 Initially two headmasters reported that they had used DFS. This was verified later on and it turned out that the headmasters had confused DFS with conventional TATA iodized salt.

children involved in the survey, medical personnel was instructed to tell parents in case their child was moderately or severely anemic and to advise them to feed their children more diversely and particularly more food items with high iron content (green leafy vegetables and meat in case they were non-vegetarians), in cases where a child was diagnosed as moderately anemic. In cases of severe anemia, parents were instructed to consult a doctor (which was only the case for 14 children). In general, this additional intervention does not bias our results, since this information intervention affected the treatment and control groups equally and its effect is therefore balanced between these two groups. However, in case the information intervention did indeed provoke a change in feeding practices or medical treatment, a saturation effect might have occurred (i.e. decreasing returns to scale of iron-interventions). It might have been the case that the usage of DFS in the school’s MDM could only add little to the already higher hemoglobin level, the reduced anemia prevalence and the improved cognitive and education outcomes. Compared to an exclusive DFS intervention, the estimated coefficients in this study could therefore be downward biased and might constitute a lower bound. To encounter this potential thread we included the dietary diversity score of the child as a control variable for feeding practices in the DD estimates. We further analyzed the effect of the nutrition information intervention in essay 3 of this dissertation (chapter 4). We did not find any robust effect of the nutrition information intervention on any of the tested outcomes, such that we are very certain that the nutrition information did not provoke attenuation bias.

3.6.3 Hawthorne Effect

The Hawthorne effect describes a change in the behavior of the individuals in the control group as a reaction of being part of an experiment. In our experiment, individuals in the treatment as well as in the control group, were surveyed once at the baseline and once at the endline; hence, any behavioral change that results from the survey itself (being monitored or the evaluation of the education level of the students) is balanced in the treatment and the control group. It might still have been the case that individuals in the treatment group changed their behavior due to the treatment itself (regular delivery of DFS to the school) and that this change affected the outcomes. Since only headmasters, but not parents, knew of the intervention, this behavioral change is limited to headmasters (and maybe some teachers who knew about the intervention).

A change in hemoglobin values due to a behavioral change by the headmasters (e.g. spending more money on the MDM to improve dietary diversity) has very unlikely occurred since

headmasters generally did not know about the hemoglobin testing as it took place in the villages or at the households but not in the schools. Nevertheless, we also control for the average calories and the average iron content of the MDM as measured on the survey day. We also believe that the expected benefits of a behavioral change were too small in comparison to the effort needed to manipulate the outcomes. A change in the components of the MDM would involve additional costs and a change in cognitive or education outcomes would need a large quality improvement in teaching (e.g. more teachers, more training material etc.). This is opposed to the benefits of being offered DFS at a subsidized price for a longer time period and maybe to please the researchers over and above the level that control schools already do because of the baseline and endline survey itself.