Sampling and data

Our study sites are located in the 18 Woredas in which the ISFM+ project was implemented, i.e. six Woredas in Amhara, Oromia and Tigray, respectively. All study sites are located in highland areas above 1,500 meters above sea level (m a.s.l.), with average elevations between 2,000 and 2,500 m a.s.l. for all three regions. In terms of precipitation, the Woredas in Amhara and Oromia can be classified as moist or wet areas (Hurni, 1998), with 1,229 mm respectively 1,426 mm average annual rainfall. By contrast, the Woredas in Tigray are much drier with 661 mm average annual rainfall. To account for these differences in agroecological potential, which might affect both technology choices and welfare outcomes, we distinguish between wet and moist areas (Amhara and Oromia) anddry areas (Tigray) in our analysis, following previous studies in similar settings (Kassie et al., 2008, 2010; Hörner & Wollni, 2020).

Within the 18 Woredas, our primary sampling units are microwatersheds, which are the implementation units of the ISFM+ project. Those are agglomerations of households (typically 200 to 300), organized in one or several villages that share a common rainwater outlet. Out of a sampling frame of 161 microwatersheds, 72 were randomly selected to benefit from the ISFM+ project, while the remaining 89 in the same Woredas are non-beneficiary (control) mi-crowatersheds. In each of the 161 microwatersheds, we randomly draw 15 households from administrative lists to be included in the sample. We restrict our analysis to the 2,059 house-holds that cultivated at least one of the main cereal crops teff, maize and wheat on at least one plot in the 2017 main cropping season, for which ISFM practices are primarily promoted and applied.⁴⁹

The main data collection took place in early 2018 by means of tablet-based structured ques-tionnaires. We collected detailed data on agricultural technology use, production, labor input, crop yields, and different income sources retrospective for the 2017 main agricultural season, as well as other socioeconomic information, inter alia. Additionally, we collected data at the Woreda and microwatershed levels, e.g. on infrastructure and climatic information. Moreover, a first, yet less detailed data collection took place in early 2016, allowing us to include some baseline characteristics in the analysis.

4.2.3 Econometric framework

The objective of our study is to assess the effect of ISFM adoption on different measures of income, food security, labor and children’s education. Hence, we are interested in the average

49 Though the ISFM+ project also advocates the use of ISFM for other crops, adoption rates for these are still low in our sample and consequently, we limit analyses to the three cereal crops.

treatment effect on the treated households (ATET), defined as the average difference in out-comes of ISFM adopters with and without the technology. Following Manda et al. (2018), the ATET is written as:

= { − | = 1}, (4.1)

= ( | = 1) − ( | = 1)

in whichE{.} is the expectation operator, the predicted outcome for ISFM-adopting house-hold i under adoption, the predicted outcome of the same household under non-adoption, while represents the treatment status taking one for ISFM adopters and 0 for non-adopters.

Yet, while the outcome for adopters under adoption ( | = 1) can be observed in the data, the counterfactual outcome ( | = 1) cannot. Replacing these outcomes with those of non-adopters ( | = 0) is likely to result in biased estimates due to possible self-selection of ISFM-adopting households. To overcome this problem, we follow Manda et al. (2018) and apply the doubly-robust inverse probability weighted regression adjustment (IPWRA) method.

The IPWRA estimator is obtained by combining inverse probability weighting (IPW) with re-gression adjustment (RA) (Wooldridge, 2010). While IPW focuses on modelling the treatment selection, RA concentrates on outcomes, which allows controlling for selection bias at both stages. This property is referred to as ‘doubly-robust’, since only one of the two models needs to be correctly specified in order to obtain consistent estimates of treatment effects (Wooldridge, 2010).

In a first step, the inverse probability weights need to be calculated based on the estimated probability of receiving the treatment (ISFM adoption). For this purpose, propensity scores as defined by Rosenbaum and Rubin (1983) are estimated:

( )= ( = 1| )= {ℎ( )}= ( | ) (4.2) whereX represents a vector of exogenous variables including household and farm characteris-tics, infrastructure, weather, shocks, and access to information, and {.} is a cumulative distri-bution function.

Based on the estimated propensity score ̂, inverse probability weights are calculated as for treated households, and for non-treated households. In other words, each observation is weighted by the inverse probability of receiving the treatment level it actually received (Hernán

& Robins, 2019).

The RA method fits separate linear regression models for both treated and untreated obser-vations, and then predicts the covariate-specific outcomes for each subject under each treatment status. Average treatment effects are then obtained by averaging the differences between

predicted outcomes under adoption and non-adoption. The ATET for the RA model can be expressed as follows (Manda et al., 2018):

= ∑ [ ( ,δ ) − ( ,δ )] (4.3) where is the number of adopters, and ( ) describes the regression model for adopters(A) and non-adopters(N) with covariatesX and estimated parameters ( ).

The IPWRA estimator is then constructed by combining the RA method with the inverse probability weights and can be written as:

= ∑ [ ^∗( ,δ^∗) − ^∗( ,δ^∗)] (4.4) in which ^∗( ^{∗ ∗}) and ^∗( ^{∗ ∗}) are obtained from the weighted regression procedure.

To assess whether our sample is balanced after the inverse probability weighting procedure, we run an overidentification test, and additionally calculate normalized differences for each covari-ate as Imbens and Wooldridge (2009) propose:

norm_diffj:^{( )} (4.5) where Aj and Njrepresent the means for variablej for adopters and non-adopters respectively, and and the corresponding standard deviations.

The IPWRA method rests on two assumptions. Firstly, it assumes conditional independence or unconfoundedness. This means, conditional on observed covariates, treatment assignment can be considered random. Since selection into treatment regimes might still be based on unobserv-able characteristics, this is a strong assumption. Yet, conditioning on a rich set of observunobserv-able covariates may help to circumvent or at least reduce selection bias due to unobservables (Im-bens & Wooldridge, 2009). The second assumption postulates that, conditional on covariates, each observation has a positive probability of receiving the treatment. This is often called over-lap assumption and ensures that for each adopting household, a non-adopting household with similar characteristics exists. If this assumption is violated, estimators are overly sensitive to model specification, potentially leading to imprecise estimates. Therefore, we will set a toler-ance level for the estimated probability of receiving the treatment between ̂ =0.001 and ̂ = 0.999.

As a robustness check for the IPWRA estimations, we use a simple propensity score matching (PSM) approach by matching the three nearest neighbors, as commonly done in the literature (e.g. Takahashi & Barrett, 2014).