COST BENEFIT ANALYSIS FOR COMMUNITY BASED CLIMATE AND DISASTER RISK MANAGEMENT:
SYNTHESIS REPORT
August 2010
Developed and Commissioned by:
Oenone Chadburn, Tearfund
Jacobo Ocharan and Karey Kenst, Oxfam America
Courtenay Cabot Venton, Author, Freelance Consultant
This paper has been developed with the very helpful participation of a range of key experts. We would like to acknowledge and thank the following people for their contributions: Robert Roots, British Red Cross (BRC); Daniel Kull, International Federation of Red Cross and Red Crescent Societies (IFRC); Andrew Mitchell, Accion Contre el Faim (ACF); Karl Deering, CARE International; Jo Khinmaung and Jessica Faleiro, Tearfund; Marcus Moench, ISET; Nana Kuenkel, GTZ; Susan Romanski, Jose Fernandez and Teron Moore, Mercy Corps; and Paula Holland, SOPAC. We would also like to thank Bina Desai at the International Strategy for Disaster Reduction (ISDR), who was instrumental in the initial discussions that brought about this report.
1. Introduction
1.1 BackgroundCost Benefit Analysis (CBA) is increasingly being used to inform and evaluate a range of interventions that can address climate and disaster risk. The findings from these
analyses are being used for multiple purposes – first and foremost, CBA can be used as a decision support tool, to help decide between a range of possible interventions that reduce risk and to maximise benefit for every dollar of investment spent. Furthermore, CBA can be used to make an economic argument for investment in risk reduction (rather than responding to the impacts of a future disaster event). While CBA has historically been used to assess larger scale infrastructure projects and public investment projects, its use at a local or community level is becoming more widespread.
A variety of case studies looking at the impacts, costs and benefits of Community Based Disaster Risk Management (CBDRM) and Climate Change Adaptation (CCA) have been undertaken over recent years. Further, Non‐Governmental Organisations (NGOs) and others are beginning to look more closely at the applicability of CBA as a tool to sit alongside existing processes, such as Vulnerability and Capacity Assessment (VCA) and Monitoring and Evaluation Frameworks (M&E), to help project partners examine in greater detail the quantifiable, as well as the more qualitative impacts of their programming.
At the same time, in the context of a changing climate, a variety of initiatives are attempting to shed more light on the costs and benefits of various disaster risk and climate adaptation options at a macro level, to help prioritise and inform decision making and public policy going forward. These studies are part of a critical path to move the adaptation agenda from a pilot/localised level, to a more scaled up and global approach to tackling adaptation. The challenge lies in identifying a range of potential interventions that are suited to different situations, identifying basic principles that are universally applicable, and finally developing the analytical tools that enable measures to be tailored to local contexts. However, there will never be a “one size fits all”
approach to risk reduction, and to this end, the development of strategies at a global level need to be interlinked and take account of evidence from household, community and regional levels.
It is recognised that a stock‐taking of research on CBA of CBDRM/CCA to date is needed, with recommendations for how, and in what context, this tool may be useful and
applicable going forward, as well as the implications of findings for wider initiatives to promote adaptation and reduce risk.
Box 1: What is CBA, and why is it useful for DRR/CCA?
CBA is an economic tool used to compare the benefits against the costs of a given project or activity. CBA can be a particularly useful tool in a disaster/climate risk context. Firstly, CBA can help communities and local partners, as well as government, NGOs and donors, decide on program options, by entering into a more robust process of weighing up costs and benefits of different interventions, both qualitative and quantitative. Secondly, risk reduction requires significant resources to be spent before a disaster, and the benefits are not always overtly obvious. CBA can provide a powerful tool for demonstrating the value of pre‐emptive action and investment in risk reduction.
In order to be effective, CBA must be linked with other techniques – such as community engagement – and it needs to be conducted in a transparent and accessible manner.
1.2 Aim of this Report
The aim of this report is to present a brief synthesis that takes stock of the significant efforts on Cost Benefit Analysis of CBDRM/CCA to date, reflecting not only on findings, methodological approaches and lessons learned, but also addressing the implications for where and when CBA may be usefully applied at a community level, and pointing to gaps and methodological constraints that could usefully be addressed going forward.
1.3 Scope of Research and Approach
Scope: It is important to note that this synthesis is very much focused on the application of CBA to community‐based initiatives for disaster and climate risk management.
Initiatives may be structural or non‐structural, hard or soft, but are part of a community driven process for DRM and are very much bottom‐up.
This report is intended as a first step towards a broader and more in‐depth discussion around the cost effectiveness of various resilience and adaptation strategies, and the applicability and usefulness of CBA at a community level to help inform decision‐making by NGOs, government and donors alike. It is intended as a high level review of recent work – it is not within the scope to conduct a detailed review or in‐depth analysis.
However, it is hoped that this report will act as a stepping stone for further development and discussion.
Research Approach: The research was conducted by undertaking a literature review for relevant recent studies in the area of CBDRM/ CCA and CBA. Further to this, a number of key contacts working in this area were approached to identify additional
reports/research and/or other contact points (see Annex A). The findings from the
studies identified were condensed into a brief synopsis (see Annex B) and summarized here. A draft synthesis was submitted for review to the list of key contacts, and a
consultation/brainstorming meeting was held in London on June 29, 2010 to discuss the draft findings, refine the discussion, and identify ways to move the discussion forward.
This report summarizes not only the literature to date, but also incorporates the comments and issues that were raised during the consultation exercise (Annex C contains a very brief list of the key points raised at the consultation).
1.4 Structure of this Report
This report is structured as follows:
• Section 2 looks backs at recent work on CBA applied at a community level. In particular, it highlights key elements of the approaches used, some of the
methodological issues faced in applying CBA at a community level, and some of the key lessons learned in relation to both the process as well as the cost effectiveness of various interventions.
• Section 3 looks ahead, presenting initial thoughts on the usefulness and applicability of CBA at a community level, and is intended to stimulate discussion for moving forward.
• Annex A contains a listing of the key participants contacted for the research, with an indication of those that attended the workshop.
• Annex B provides a brief synopsis of each of the studies included in the synthesis.
• Annex C highlights the key points coming out of the June 29 consultation exercise.
• Annex D contains a bibliography of sources reviewed.
2. Looking Backward: A Synthesis of Studies Identified on CBA for CBDRM/CCA
2.1 Introduction
This section looks back at the work conducted to date – it provides a synthesis of the key findings from the studies identified on CBA for CBDRM/CCA:
• First, the key elements of the methodological approaches used and study findings are summarized.
• This is followed by a discussion of some of the methodological issues faced in conducting CBA at a community level.
• Finally, this section looks at some of the key lessons learned, both in relation to applying CBA at a community level, as well as the types of activities that are cost effective (and not) for addressing disaster and climate risk.
This section is used to support the discussion in Section 3, which is intended to stimulate discussion and debate on the usefulness and applicability of CBA at a community level going forward.
Box 2: De‐mystifying CBA – is it accessible?
CBA is an economic approach that is frequently associated with the assessment of large infrastructure projects. As such, it is often perceived as a tool that is resource intensive, and that requires specialised technical skills.
However, the principles of CBA are applied to every day decisions – people and organizations regularly weigh up the costs and the benefits of most every activity – whether it’s deciding which crop to plant, which materials to use to build a house, or whether to hire more staff. As such, the principles that underpin a CBA process are highly intuitive, though rarely applied in a systematic manner.
Where CBA is used as part of a participatory process with communities, it can be extremely valuable, by helping communities and programme staff to think through the costs and benefits of different programme options, and targeting resources towards achieving “outcomes”, rather than “outputs”. Data gathering for a CBA does not necessarily require a great deal of extra resource or technical capacity (depending on the availability of data, and the level of analysis undertaken), but rather in many cases relies on additional lines of questioning around the quantitative impacts of program interventions, and is often very similar to existing baseline data collection and VCA processes.
Where data is limited, a quantitative CBA may not be appropriate, and could present
misleading results. However, the CBA process can nonetheless generate a great deal of added value to decision making, especially in the context of an uncertain future.
Benefit to Cost Ratio (BCR): The BCR indicates the level of benefit that will be accrued for every $1 of cost. A ratio greater than 1 therefore indicates that the project is worth investing in from a financial perspective, whereas anything less than one indicates a negative return.
Net Present Value (NPV): The NPV takes the net benefit (benefit minus costs) each year and discounts these to their present day value. If the result is greater than zero, this indicates that the benefits outweigh the costs. The higher the value, the greater the financial argument for initiating the project.
The Discount Rate is used to discount costs and benefits occurring in the future, as people place a higher value on assets provided in the present and a lower value on benefits that may accrue further into the future. The discount rate is normally equivalent to the average return one might expect if the same money was invested in an alternative project.
2.2 Key Elements of the Studies Reviewed
In total, 13 studies were reviewed for this report (a further four case studies are due to be published over the next few months and will be incorporated as they become available). Table 2.1 below highlights some of the key elements of each of the studies, such as where they were conducted, the type of hazard addressed, and some of their main findings. This table is supported by a more detailed Annex B, which contains a brief synopsis on each of the studies.
Box 3: The Oxfam America CBA Toolkit
Oxfam America (OA) is in the process of developing a Toolkit that will facilitate regional offices and partners to undertake Cost Benefit Analysis as a routine part of the project cycle. OA wants to progressively introduce CBA in its DRR programs to appraise and present the cost and benefits of their interventions and inherent tradeoffs in their investment in risk reduction. In 2009 OA’s headquarters‐based DRR staff developed a user‐friendly CBA methodology, designed to enable effective decision‐making in OA’s DRR projects in every region.
The Toolkit is designed to sit alongside existing VCA processes, and is composed of three modules, with a range of associated templates and tools:
‐ Module 9a: Introduction to Community Based CBA for DRR
‐ Module 9b: Methodology for Community Based CBA for DRR
‐ Module 9c: Valuation Worksheets
OA is in the process of field testing the methodology in two countries, El Salvador and the Gambia.
Box 4: Glossary of CBA Terminology
Table 2.1: Summary of CBAs of Community Based Disaster and/or Climate Risk Management
Organization Date Country Hazard Key Elements Key Findings
Tearfund
2004 India Flood;
Drought
• Backward looking
• Interventions include construction of an escape road, provision of boats for evacuation, raised hand pumps
• Data collected through transect walks, focus groups
• Qualitative and quantitative
Bihar BCR = 3.76; AP BCR = 13.38
World Bank 2007 Kenya Flood • Forward looking
• Community Driven Development, including woodlots, medicinal plants, indigenous vegetables, beekeeping.
• Data collected primarily from research institutions with pilot projects on related activities.
See case study summary – a wide variety of initiatives and scenarios are estimated, some not viable, some viable.
ISET 2008 Nepal Flood • Backward looking
• Purely qualitative, uses “Shared Learning Dialogue”
• Addressed distributional issues
• Hard and soft resilience measures
• Addresses climate change ‐ qualitative
Structural measures cannot be an effective primary strategy for responding to the increased flood risk anticipated as a consequence of climate change
ISET 2008 India Flood • Backward and forward looking
• Addresses climate change
• Embankments compared with a more people‐
centred basket of interventions (raised house plinth, raised fodder storage, early warning, flood shelters, community seed banks, self help groups, etc)
• Data collected through a household survey
Embankments have not been economically beneficial. The analysis generates a BCR of 1 and it is predicted that this would decrease with climate impacts.
BCRs for people centred approaches range from 2 to 2.5 under current and future climate scenarios.
Organization Date Country Hazard Key Elements Key Findings ISET 2008 India Drought • Insurance mechanisms for addressing drought
risk, groundwater irrigation
• Forward looking
• Risk based modelling framework used to generate probabilistic drought shocks to farmers.
• Incorporates climate change
• Resource and time intensive due to complex modelling needs
All interventions seem economical, with the integrated package of both
interventions delivering similar benefits at lower cost.
ISET 2008 Pakistan Flood • Four measures addressed: warning system, concrete lining of the channel, construction of a dam in the upper reaches of the stream, and relocation of the most exposed population to higher ground.
• Backward‐looking
• A simplified downscaling technique and rainfall runoff model were used to investigate potential climate change impacts
• Used data from 2001 floods
The over‐ designed early warning system in place is the only one with a benefit cost ratio of less than one.
British Red Cross/Nepal Red Cross Society
2008 Nepal Flood • Qualitative and quantitative approach
• Quantifiable measures include mitigation works (flood defence), income generation loans, protection of water sources, and first aid training.
• Backward looking
Full suite of quantifiable measures: BCR = 18.6
Without flood mitigation (only loans, water sources, training): BCR = 2
SOPAC 2008 Samoa Flood • Interventions assessed include: floodwalls, a diversion channel, an improved flood
forecasting system, and development control through the construction of homes with elevated floor heights.
Non‐structural measures were found to be the most economically viable. Improved forecasting system: BCRs range from 1.92 to 1.72. Homes with raised floors: BCRs range from 2 to 44, dependent on the type
Organization Date Country Hazard Key Elements Key Findings
• Flood hazard maps created using impacts of previous floods from public records, household and business surveys.
• Direct and indirect monetary losses estimated.
• Distribution of impacts is accounted for across sectors.
of structure, floor height, and discount rate used in the analysis.
Structural measures were found to be not economically viable, and it is not believed that other non‐quantifiable benefits would be enough to raise ratios above one.
SOPAC 2008 Fiji Flood • Survey used to assess impacts to a range of sectors including household, business, government and donors.
• Intervention assessed is an effective flood warning system.
• Assessed distributional issues.
Overall: BCR of 3.7 to 7.3
Navua community: BCR is infinite (no costs borne)
Govt of Fiji: BCR = 1.1 to 2.2
Oxfam America
2009 El Salvador – ex post
Flood • Field testing of a CBA tool with local partners
• Qualitative and quantitative
• Participatory approaches with communities used to gather primary data.
• Backward looking CBA of a DRR program to improve evacuation and shelters.
The program yields a BCR of 0.97 using conservative assumptions. Sensitivity testing yields BCRs of 1.05 to 1.60.
IFRC 2009 Philippines Flood • Qualitative and quantitative
• Participatory approaches with communities used to gather primary data.
• Conducted as part of a wider evaluation.
• CBA of three specific interventions: a hanging footbridge for evacuation, a sea wall and a dyke
• Backward looking
Two of three interventions are cost effective:
Hanging footbridge: BCR = 24 Sea wall: BCR = 4.9
Dyke: BCR = 0.67
IFRC 2010 Sudan Drought • Qualitative and quantitative
• Participatory approaches with communities used to gather primary data.
• Conducted as part of a wider evaluation.
Earthdams/embankments and water interventions were all found to be economically efficient. However, some of the most important impacts were
Date Country Hazard Key Elements Key Findings
• CBA of individual activities. qualitative, namely educational benefits and women’s groups.
Oxfam America
2010 El Salvador – ex ante
Drought, Pests, Livestock disease
• Field testing of a CBA tool with local partners
• Qualitative and quantitative
• Participatory approaches with communities used to gather primary data.
• Forward looking CBA to assess a range of possible project interventions for investment.
A wide range of interventions were assessed, including silos, alternative food sources for cattle, vaccination, alternative seeds, vegetable gardens, and community organizing. The BCRs range from 0.42 to 86.70. Silos yield a negative BCR – for cultural reasons they need to be provided on a household basis at high cost.
Community organizing for collective bargaining on agricultural inputs yields the highest BCR.
Oxfam America
2010 Gambia – ex ante
In Process
Oxfam America
2010 Gambia – ex post
In Process
Tearfund
2010 Malawi In Process
Mercy Corps 2010 Nepal In Process
Organization
Broadly speaking, the studies reviewed are built on a common methodological approach in as much as they all incorporate the following (though to varying degrees of
complexity and detail):
• A hazard assessment that investigates the hazards affecting the population in question, their magnitude and frequency.
• An impact assessment that investigates the impacts of hazards on the community, specifically in relation to the population’s vulnerabilities, capacities, and exposure to hazards, “without” CBDRM.
• An analysis of risk reduction: what interventions have been/can be introduced to reduce risk and how have the impacts of hazards changed as a result? What are the costs of these interventions? This assessment investigates the impacts of hazards
“with” CBDRM. The difference in impact “without” and “with” CBDRM represents the avoided cost, or benefit, for undertaking CBDRM.
There are also a number of notable differences in the studies/approaches reviewed:
Qualitative versus quantitative. Studies range from purely qualitative, as in the example of ISET Nepal where Shared Learning Dialogues were used to understand the costs and benefits of risk reduction from a purely qualitative perspective through discussion; to a mixture of qualitative and quantitative assessments, where the full range of impacts are assessed, but a subset of those that can be quantified are investigated in further detail.
Of note, CBA studies that investigate more structural measures on a larger scale (for instance, some of the World Bank studies for development projects) typically focus almost entirely on the quantitative aspects, whereas the application of community level CBA has the opportunity to take a more holistic approach by elaborating on both
qualitative and quantitative aspects.
Ex‐post versus ex‐ante: CBA at a community level has been used to assess projects or programmes that have already occurred – referred to as “backward looking” or “ex‐
post”. It can also be used to decide between a suite of interventions, to identify those that are most cost effective going forward – referred to as “forward looking” or “ex‐
ante”. Some assessments could have elements of both – for example, the 2009 Oxfam America study in El Salvador was backward looking, but found that several interventions were too recent to have taken hold, and hence a forward looking assessment, using anticipated impacts and sensitivity testing, was used for those elements.
Data sources: The data used for quantitative assessments comes from a mixture of primary and secondary sources depending on the study and the availability of data in the country. Examples of secondary data collection include: datasets on hazards and
their impacts from government records; data on hazards, their impacts, and the viability of alternative approaches to activities such as agriculture from research institutions;
projected impacts of climate change from meteorological institutions and research bodies; GIS maps from relevant authorities and research organizations; and data on community level impacts from existing NGO baseline studies. Examples of primary data collection include: participatory processes such as transect walks and focus groups to gather data on hazards and their impacts; surveys of affected populations to gather data on hazards and their impacts as well as demographic data and indicators of
vulnerability; and semi‐structured interviews with local officials, CBOs, and other relevant stakeholders.
Data on hazard impacts: Data on hazard impacts can take a number of forms:
• Direct/indirect – in most cases, only direct impacts are included in the analysis (e.g.
loss of assets, damage to houses, etc). In some cases, efforts are made to identify indirect impacts as well – for example, floods may result in business interruption for several months after the fact.
• Monetary/non‐monetary – many impacts are non‐monetary. In other words, they cannot be numerically measured. Or they may be too complex to measure. Or, in the case of placing a value on loss of life, some studies choose not to place a monetary value on this loss from an ethical standpoint.
• Financial/economic – in theory, CBA is used to account for economic impacts – all those impacts that affect the wellbeing of a population. However, in practice most CBAs at the community level are financial in nature, with a focus on those impacts that can be easily monetized. Economic benefits, such as protection of natural resources, can be valued but usually require time intensive studies to do so.
Nonetheless, most of the studies incorporate economic benefits at least from a qualitative perspective.
Types of risk reduction measures assessed: The risk reduction measures included in the CBA can vary, and very much depend on what has been/is being considered under the project or programme. Furthermore, the CBA may either take an approach that
evaluates individual activities under a program (as in the IFRC Philippines study) or the programme as a whole (as in the BRC Nepal study). Measures can be broadly
categorized as prevention versus preparedness (e.g. a dam to “prevent” the flood versus grain stores to ensure food is available during flood times); and structural versus non‐
structural (e.g. water pumps, dams, and embankments, versus training, advocacy and awareness raising measures). By their very nature, community level interventions tend to encompass a range of types of activities, and hence the different studies cover a variety of types of risk reduction measures. Furthermore, the impacts of softer
measures can often be hard to quantify, and hence the range of measures included in a
programme requires an approach that includes both qualitative and quantitative techniques.
2.3 Methodological Issues
The methodological approach for applying CBA at a community level can clearly take a number of forms, as highlighted above. There are aspects of the approach that are intuitive and work well, and other aspects that are harder to apply at a community level.
This section highlights some of the key methodological issues faced in the studies reviewed.
The valuation of non‐monetary benefits is a significant constraint in applying CBA.
Community work brings a whole host of benefits that cannot be quantified – but which are often central to the work being undertaken – for example social and environmental benefits. Decision‐making must take account of the full range of impacts, and the danger with CBA is that a project with a high level of monetary benefits will be selected over a project that may be equally beneficial but not so easily quantified. This issue becomes particularly critical in areas such as slow onset disasters, where it can be very difficult to identify both monetary and non‐monetary benefits of breaking cycles of poverty brought on by successive droughts, or in the case of ecosystem based approaches, where environmental benefits are a key priority.
A clear understanding of risk is inherent to conducting CBA, and yet the probability of hazard occurrence, and associated impacts, can be very difficult to estimate,
particularly when the analysis is taking place at a community level. Ideally, a CBA is built upon probabilistic risk modelling, where the probability of a hazard occurring is estimated for a range of hazard magnitudes. The impacts (and associated reduction in impacts that come about with risk reduction) are then weighted by the probability of an event happening. These points create a loss‐frequency curve. In practice, however, data is often very limited, particularly at a local level, and it is only possible to map two or three hazard/impact probabilities.
Climate change adds another level of complexity to probabilistic risk modelling. The probability of hazards is changing under climate change, and hence loss‐frequency curves will also shift, changing the outcomes of any cost benefit analysis. While
significant efforts are being made to downscale projections on climate change impacts from more global models to country, region, and locale‐specific models, this requires significant amounts of data, and even then, results are highly uncertain. In addition to a certain degree of unpredictability of future human behaviour and natural variations, the down‐scaling of global projections itself is an imprecise science. Hence it becomes very
difficult at a community level to estimate whether a 1‐in‐10 year flood is likely to become a 1‐in‐8 year flood, or a 1‐in‐5 year flood, and indeed, how quickly these
changes will take place. The ISET India study used a risk‐analytic modeling approach, and found that ultimately this was a very resource and time intensive approach, which generated findings that were highly uncertain in any case. The study authors suggest that sensitivity testing for a range of probable climate scenarios could have generated equally reliable findings but more efficiently.
The distribution of benefits from risk reduction is very important from a development perspective, with many projects focusing on the most vulnerable, including women, children, the elderly and disabled. CBA does not traditionally account for distributional impacts. First of all, the work done to date on CBA at a community level has consistently emphasized the need to ensure that the quantitative analysis sits within a wider
qualitative framework. As such, distributional aspects can be discussed and included in a more qualitative fashion. The SOPAC Navua study used a methodology that explicitly demonstrated distributional impacts between households, businesses and government.
The study used a survey to investigate impacts of hazards, and reduction in impacts associated with risk reduction measures, to each of these groups. The study also allocated costs of risk reduction measures according to who would pay for them. The study goes on to estimate CBA figures specific to each of these groups – according to who pays, and who receives the benefit, and as such presents a very interesting case for addressing distributional aspects across a society as a whole.
For backward looking CBAs, the timing of the study with respect to implementation of project interventions can significantly impact on methodology and results. When assessing impacts, if the project interventions took place too far in the past, community members can find it difficult to reconstruct the “without” scenario. For example, in the IFRC Philippines study, the project had been implemented 10 years previously and thus there was a large degree of variation in recounting of impacts. In Samoa, the household survey was conducted 6 years after the event and resulted in values that were so unreliable they had to be replaced with other estimates. On the other hand, if interventions have occurred too recently, it may not yet be possible to observe the impacts of the intervention (as was found in the 2009 Oxfam America study in El Salvador). This is particularly true for activities such as changes in cropping patterns or introduction of new seeds, which require a longer time frame to take hold, and for which impacts are not always easily attributable (a new crop could reduce impacts of drought, but it can be hard to quantify this in the immediate term because so many exogenous factors impact on crop yields).
2.4 Lessons Learned
A number of lessons learned can be highlighted from across the studies reviewed.
There is broad consensus that the CBA process can be useful at a community level…
• CBA at a community level yields findings that are helpful for both evaluation purposes as well as making forward looking planning decisions, and these findings have been used effectively for advocacy and demonstrating the value of CBDRM to donors and government.
• The process of conducting a CBA is fairly intuitive, especially with regards to the field work.
• The process introduces another layer of evaluation, encouraging a more robust analysis of benefits, as well as fostering a greater focus on outcomes as opposed to outputs. Furthermore, CBA encourages an open discussion that fosters consensus building, innovative thinking and transparency, and can help to bridge discussions between government and Community Based Organizations (CBOs). In fact, a key finding is that the process is often more beneficial than the “product” (the final analytical result), because it forces organizations to clarify and test the assumptions they make between an intervention and the desired outcome, as well as opening up a transparent dialogue.
• The studies reviewed have not only confirmed some anticipated outcomes, but have also generated some surprises, and hence added value to the overall decision
making process. For example, the CBA study undertaken by Oxfam in El Salvador in 2010 (as yet unpublished) demonstrated that the use of silos and storage practices to protect crops were actually not cost effective, in large part because of cultural barriers to collective storage that dictated the need (and expense) of household silos, and hence a suite of other options are being investigated and prioritised that can reap greater gains for beneficiaries.
There is also no doubt that it has its limitations…
• CBA, at its core, is about risk assessment, and hence uncertainty is inherent in the process, especially at a community level and in the face of climate change.
• Data limitations can pose a substantial challenge, especially where there is not the capacity/resource to conduct primary data collection. And even where data can be collected, there are often significant levels of uncertainty over the data gathered (e.g. bias in responses, long recollection times, conflicting/inconsistent information among those surveyed).
• Further, while CBA is underpinned by some common principles, due to data
constraints and other limiting factors, it is not applied systematically at a community
level, making it difficult to compare across studies and draw broader lessons around successful interventions.
• A focus on quantitative aspects of programme design sits more comfortably with large infrastructure projects. By contrast, CBDRM, by its very nature, is typically focused on a mix of hard and soft resilience measures, largely implemented by NGOs/CBOs, and hence the focus on quantitative is not as natural, and the benefits are often inherently difficult to measure and quantify.
• There are differing perceptions on how valid the CBA process is at a community level, given these data limitations.
There are some interesting and unexpected lessons learned with respect to those interventions that are most cost effective, with direct relevance to NGOs, governments and donors alike…
… a focus on interventions that bring wider development gains are generally going to be more cost effective. This is particularly emphasized in the face of uncertainty. In areas where the frequency and magnitude of hazard occurrence is less known, activities that focus only on CBDRM are more likely to have a negative return. By contrast, if these activities also bring wider development gains, they are more likely to be cost effective.
For example, in the Tearfund study in India, boats were provided for evacuation
purposes, but were also rented out by villages to neighbouring communities for fishing outside of flood times, generating an important source of income for the community that was then used for community development projects. Indeed many of the interventions assessed for CBA deliver both disaster and development benefits –
evacuation shelters are used at other times for community meetings, provision of raised water wells are not only beneficial in floods, but provide sufficient clean water year‐
round, and training and community organizing for evacuation often results in
community groups that advocate for themselves on a whole range of issues. This finding strongly supports the current discussions around “no‐regrets” development approaches and integrating/mainstreaming DRR/CCA within wider development plans.
… soft resilience measures are often more cost effective and more robust in relation to uncertainties than hard resilience measures. Firstly, soft resilience measures generally cost less (less capital intensive) but can be highly effective. For instance, in El Salvador, Oxfam found that training on evacuation was highly effective and resulted in significant savings as families evacuated livestock in good time. Second, even where the ratio of benefits to costs is similar across soft and hard measures, the absolute cost for softer measures tends to be much smaller. For example, the Maldives study highlighted in Box 6 below found that soft resilience measures yielded similar ratios to hard resilience measures, but the total spend was far less. Further, hard resiliency measures tend to be
“threshold dependent” – designed to withstand a specific magnitude of hazard. As a result, returns from soft measures may be more robust in the context of uncertainty over changing conditions. The Samoa case study also came to a similar conclusion, finding that softer measures such as improved flood forecasting were more cost effective than more structural measures such as floodwalls.
Box 5: Hard Resilience versus Soft Resilience
Most activities undertaken in the name of disaster risk reduction fall into two broad
categories: (1) “Hard Resilience” measures: the strengthening of physical systems to directly withstand or respond to the specific stresses imposed by earthquakes, storms, floods or other extreme events; and (2) “Soft Resilience” measures: a wide variety of “softer” and indirect measures intended to reduce the impact of events on people and assets, improve relief capacities when events occur, and aid recovery.
Source: Moench, M. (2008)
… the design of both soft and hard measures for risk reduction should be fit‐for‐
purpose to ensure returns. The ISET Pakistan study found that, contrary to intuition and previous experience, the Early Warning System (EWS) was not cost effective, because it had been over‐designed for its purpose. This finding also accentuates that there is no one‐size‐fits‐all approach; even an EWS can be cost‐ineffective if it is not tailored to local circumstances.
… CBDRM programming needs to take a holistic view, even if activities are only undertaken in a subset of communities. Benefits accrued from activities are not valid if risk is simply displaced. For instance, in Nepal, BRC/NRC found that mitigation works in the river were having significant benefits for the communities in that section of the river, protecting crops and houses from annual floods. However, there was concern that the displacement of the water from one set of villages could possibly be increasing the flow of water in other villages, and hence simply displacing the impacts of flooding.
While NRC could not operate across the whole river basin, the study findings highlighted the need to take a more holistic approach under consideration. This is a key weakness in many community‐based approaches ‐ they generally miss system level vulnerabilities and/or benefits.
… longer term support can reap significant benefits. In several of the case studies, CBAs were assessed for both the short term and the longer term. Whereas much NGO and donor programming in communities typically runs for one to three years, CBA
demonstrates that returns can often be doubled if a small amount of support, for instance refresher training, or maintenance on physical works, is provided over the longer term. For example, the BRC study in Nepal found that benefits could be doubled
for a minimal amount of support for maintenance of first aid kits, water wells, and check dams over 10 years as opposed to the standard project lifetime of 3 years.
Box 6: Building Resilience in the Maldives
A CBA of three islands in the Maldives was conducted in 2009 to determine the effectiveness of creating “safer islands”, using mostly hard resilience measures to protect selected islands from the risk of sea level rise, flooding and tsunami. Two of the islands were under
consideration for development as safer islands, whereas one of the islands had already been significantly modified following near complete destruction from the 2004 tsunami. A number of scenarios were considered, including a full suite of safe island measures (for instance construction of safe harbors, building of sea walls), a selected suite of measures, and a limited protection scenario.
The findings from the CBA were mixed, with a range of positive and negative findings.
Furthermore, the findings are very specific to island characteristics. In particular, the analysis for Thinadhoo Island is more positive because 1) Thinadhoo has a predicted lower intensity for a tsunami and therefore a standard suite of risk management measures affords more
protection, and 2) much of Thinadhoo’s infrastructure is located away from high intensity zones and therefore easier and less costly to protect.
Furthermore, the study found that soft resilience measures may, in fact, be a more successful and sustainable option for the Maldives. The greatest threat to the Maldives is sea level rise, which is slow onset (unlike other hazards such as flash flooding), and can be monitored (unlike earthquakes). Hence the Maldives can use time to its advantage to look into alternative protection options, allow for development of new technology, and lower cost innovation, while also allowing the natural adaptation processes of the islands to work to their full advantage. Man‐made interventions may only hinder the ability of islands to respond
naturally, and thus while providing some protection in the short term, may contribute to a lack of longer‐term resilience. In addition, many of the more frequent hazard events, such as rainfall flooding, are not reported in the past – they have largely come about as a result of poor development practices on the islands, and hence could be rectified through lower cost measures such as revising and enforcing land use planning.
Source: Cabot Venton et al, 2010.
3. Looking Forward: The Context for Using CBA at a Community Level
3.1 Introduction
As summarized above, a variety of studies have been undertaken using CBA at a community level to evaluate and inform disaster and climate risk reduction
programmes. The synthesis of findings has clearly highlighted that CBA at this level can be very useful, especially where it is part of a wider qualitative evaluation process, but that it also has a number of significant limitations, particularly in data and resource constrained contexts.
It is clear that CBA cannot and should not be used across the board. Hence the question arises: In what context can CBA be useful for application at a community level?
3.2 Approaches to Applying CBA at a Community Level
The research to date suggests that there are three possible approaches to a CBA at a community level, requiring progressively increasing amounts of data and resources.
These three approaches are by no means absolute ‐ they are categories along a continuum and therefore studies may combine elements from different approaches.
Nonetheless, the three approaches help to provide a framework for discussing the application of CBA at a community level.
These approaches are applicable to any organization implementing CBDRM and/or CCA programmes, including NGOs, government, and donors.
Approach 1: Qualitative Assessment
As with the ISET study in Nepal, the principles of CBA can be used to engage in a more quantitative line of questioning with communities through focus groups and other participatory tools, but without actually quantifying benefits in full for a detailed CBA.
This approach can be very beneficial as it encourages a greater discussion around outcomes (as opposed to outputs). This approach can be used in scenarios where capacity and/or resources are significantly constrained, or proposed interventions are too small to justify a more in‐depth analysis. Focus is on the process rather than the product.
Approach 2: Basic CBA
A basic CBA should be used where quantitative analysis/findings are desired, but where there is not enough resource or capacity to do a thorough statistical sampling of the population. It can be run alongside existing VCA/M&E processes, and hence does not need to be time intensive. Fieldwork can be undertaken by project partners/staff/local counterparts. Some specialist assistance may be required for modelling the findings.
Approach 3: Full Scale CBA
A full scale CBA should be used where a significant investment is being made, or where a detailed analysis can help inform wider discussions on scaling up of particular DRR/CCA interventions. For example, where governments are interested in prioritizing certain initiatives in national level policy and planning, CBA at a community level may be appropriate to ensure that implementation of these policies will be effective and takes account of local conditions. Primary data is gathered through surveys, specifically on hazard characteristics and their impacts on lives and livelihoods, as well as the potential reduction in impacts through DRR/CCA. GIS maps can be used to visually represent impacts. This type of study may be particularly relevant in larger/consolidated
populations (e.g. urban), where costs of risk management interventions could be high.
3.3 Which Approach?
Organizations will need to decide where and when to use CBA, if at all. Clearly, CBA is not applicable, nor beneficial, across all programmes within an organization. Rather, it should be used strategically within an organization to make programming decisions in areas that are of key importance or focus, where significant sums of money are designated and/or plans include scaling up of activities.
There is no right answer as to which approach to use – it depends very much on the mission and strategic focus of the organization, levels of resourcing, and levels of capacity (see Box 7). It also depends on the type of organization wishing to undertake the assessment, and their motivation. CBA can be used for a range of purposes – to demonstrate cost effectiveness of a specific intervention, to identify cost effective measures across a broad geographical region, to generate ownership and facilitate decision making as part of participatory processes with communities, to advocate for specific measures, etc – and these will vary depending on whether government is trying to inform policy making, or whether an NGO is deciding how to work with a particular community, or whether a donor is making strategic decisions for investment.
Nonetheless, the flow chart on the following page suggests a number of steps/questions that can help to guide the decision making process as to when and how to implement a CBA.
Box 7: What Skills are Required for CBA?
The implementation of a CBA at a local level, by practitioners who are not experienced in the tool, has had mixed results – some are surprised by how intuitive they find the process, whereas others find it challenging.
Clearly CBA involves technical analysis, and requires a sound understanding of the economic principles on which it is built. It also requires certain mathematical and computing skills. It cannot be implemented as an off‐the‐shelf product, but rather requires some level of training in its use.
Equally, many aspects of the CBA are intuitive, and tie in neatly with existing processes such as VCA participatory approaches and data collection for monitoring and evaluation. This would suggest that, where local partners have good capacity in the skill sets required for VCA/M&E, that the data collection phase of CBA may be fairly easy to implement with a small amount of support. More technical support will likely be required for the data analysis phase.
Flow Chart for Applying CBA to Organizational Activities
Step 1: Identify areas of strategic focus (either historic or planned), where more in‐
depth analysis of costs and benefits of risk reduction programming could be beneficial, both to demonstrate value for money, as well as to decide on a suite of programming options that are most cost effective.
¾ What are the strategic areas of focus for the organization? (e.g. training, advocacy, specific types of interventions such as alternative seeds, irrigation or water
infrastructure, and/or specific countries/regions).
Step 2: Determine a subset of programmes to analyze. Once the organization has decided on two or three strategic areas of focus, evaluate those partner
organizations/local government offices that could carry out a CBA, by asking the following questions:
¾ Is partner capacity on VCA/M&E strong?
¾ Has M&E baseline data already been collected in beneficiary communities?
¾ Will project activities affect a large population/be scaled up in the future?
¾ What level of funding can be committed to undertaking CBA studies (which in turn will dictate how many studies are planned)?
Step 3: Decide on the CBA approach for each programme to be assessed. Where capacity is relatively weak, and/or data is limited, but a CBA analysis is desired for strategic (or other) reasons, a qualitative approach may be most appropriate. At the opposite end of the spectrum, if funds and capacity are available, and a specific set of activities are deemed to be very strategic to the organization or a significant component of government plans, it may be appropriate to invest in a full scale CBA, and use the findings to determine how best to scale up activities, as well as to solicit further support/funding for scaling up.
4. Key Messages and Recommendations
The application of CBA at a community level is clearly adding value to efforts to reduce climate and disaster risk. As the findings above indicate, the case study material conducted to date has added a new dimension to our understanding of the types of interventions that are cost effective, in some instances confirming suppositions, and in others presenting unexpected findings. The process is also proving valuable in helping partners to think through interventions in terms of outcomes, rather than outputs.
However, the application of CBA at a community level also has a number of significant limitations – methodological gaps and capacity constraints need to be addressed, and where and when to apply CBA at a community level needs to be elaborated.
This section highlights some of the key recommendations, from the report and from the consultation exercise, for developing this area of work.
4.1 Key Messages
First, CBA at the community level has demonstrated that the success of risk reduction and adaptation measures is context specific – a common suite of measures applied in similar hazard contexts can in fact have very different outcomes based on cultural characteristics, for example. Certain groups of measures are likely to have a lower cost relative to their benefits. This is true of many softer resilience measures. However, as highlighted by the example of over‐designed EWS in Pakistan, and silos for crop storage in El Salvador, measures cannot be divorced from their context. On the one hand, scaling up of adaptation and risk reduction will require some broader lessons and “rules of thumb” for measures that are most likely to be cost effective. But these discussions must include mechanisms for recognizing the need for risk reduction to be embedded in a participatory process that takes account of local conditions.
Second, while CBA can help to demonstrate the value for money of community development projects, community based risk reduction and adaptation result in a wide range of benefits that cannot be monetised but which are central to good
practice development – gender issues, capacity building, advocacy and governance, and environmental benefits, are all examples of key pillars of development that are often (though not always) difficult to quantify. Any assessment of the value of a programme has to include both qualitative and quantitative aspects, otherwise it could lead to development losses and poor policy choices. Any assessment of the value of a project or programme of work needs to encompass both qualitative and quantitative aspects, particularly when applied to community based development work.
4.2 Specific Recommendations
A number of specific recommendations were elaborated in the CBA studies, and as part of the consultation meeting in London, to build on the work done to date. Many of these are applicable to the full range of stakeholders implementing and financing community based work, including NGOs/CBOs, government and donors.
¾ The development of a consistent CBA methodology and procedure for data collection will help to ensure that findings from a range of studies across agencies and regions are comparable, creating a body of evidence which can help to inform policy choices at national and international levels. A potential starting point is the integration of CBA into M&E and VCA procedures. This will help to institutionalise CBA and ensure that it is implemented in the context of a strong M&E/VCA platform.
Strong M&E/CBA will only improve the transparency and accountability of activities, and the integration of CBA into M&E systems can help to drive more quantitative and efficiency driven monitoring.
¾ Further research is required to address non‐monetary benefits – given that many of the qualitative impacts addressed by DRR/CCA are central to good development, further work is required to 1) identify ways that these non‐monetary benefits can be quantified (drawing from literature in other areas of practice, such as
environmental protection, for example, where some of these issues have been quantified using more complex techniques) and 2) develop procedures for assessing and ranking both qualitative and quantitative impacts for decision‐making (such as risk assessment matrices) to ensure that non‐monetary benefits are explicitly
included in the process. This recommendation is particularly relevant in the context of an increased focus on Ecosystem‐based Approaches (EbA), where soft resilience measures and environmental approaches play a central role.
¾ To date, most of the case study work has occurred in rapid onset disasters. It is recommended that a body of evidence is developed in areas where CBA is more complex, such as conflict, slow onset disasters and cyclical/cumulative impacts, DRR in recovery operations, and multi‐hazard contexts.
¾ Investigate the use of CBA in other areas of development practice, for example the health/HIV communities where demonstrated cost effectiveness has been used to great effect to advocate for further investment. Document lessons and/or
methodological approaches that can be transferred across. As an example, the use of “Knowledge, Attitude, Practice and Behaviour” Surveys (KAPB) in the health sector could provide some useful lessons and methodologies for collecting data.
