1. Introduction
Disaster losses have increased threefold since the 1980s and their impact – both economic and humanitarian – will continue to rise as increasing numbers of people and assets concentrate on risky areas in search of jobs and opportunities. Over the last 30 years, the number of people living in flood-prone river basins has more than doubled, and the number of people exposed to tropical cyclones more than tripled. Direct financial losses have seen a rising trend over the years, reaching an average of USD 165 billion per year during the last decade.Footnote 1
Natural disaster damage can lead to long-term economic impacts that hinder development and economic stability at both a local and international level (Hochrainer, Reference Hochrainer2009; Cavallo & Noy, Reference Cavallo and Noy2010). In many situations the decisions of governments following a disaster are critical to the speed of recovery and long-term impact of the disaster. However, in responding to disasters governments may constrained by liquidity.Footnote 2
In these situations critical expenditures can lead to budget reallocation, that though necessary may be counterproductive to economic stability. Sovereign disaster risk financing and insurance (SDRFI) instruments can protect the national budget and improve the speed at which capital is available and expenditure is undertaken, reducing the damaging impact of natural disasters (Freeman et al., Reference Freeman, Martin, Mechler and Warner2002; Goes & Skees, Reference Goes and Skees2003; Linnerooth-Bayer & Mechler, Reference Linnerooth-Bayer and Mechler2007). Many developing countries and donors are taking steps to prepare against disasters, and are increasingly considering SDRFI as a way to improve their financial position in the event of a natural disaster.
An SDRFI strategy refers to both the combination of financing instruments as well as the planning and expenditures in the event of a disaster. The main financing instruments considered in this framework are:
∙ ex-post budget reallocation;
∙ savings and reserves;
∙ post-disaster borrowing;
∙ contingent loans (e.g. Catastrophe Deferred Drawdown Option, CAT-DDO);
∙ insurance;
∙ catastrophe bonds.
However, this list is by no means exhaustive as new financial instruments are created to fit needs as they arise.
It is expected that an SDRFI will be a combination of these financial instruments and a layered approach to risk will be undertaken. For example, a sovereign government with regular disaster shocks may find insurance for these shocks prohibitively expensive and instead seek to cover the shocks with savings or reserves. Alternatively, a sovereign government with a good budget and cheap borrowing may decide to avoid risk transfer and assume responsibility for disaster risk by borrowing in the aftermath.
A prime example of an SDRFI strategy is Mexico’s natural disaster fund Fondo de Desastres Naturales (FONDEN). FONDEN was first created in 1999 as a budgetary tool through which federal funds were allocated for rapid expenditure on the rehabilitation of federal and state infrastructure affected by disasters (Cardenas et al., Reference Cardenas, Hochrainer, Mechler, Pflug and Linnerooth-Bayer2007). FONDEN has since evolved to leverage private sector financing as part of a sophisticated SDRFI strategy that combines risk retention and risk transfer. In particular, FONDEN has utilised capital markets to transfer parts of its hurricane and earthquake risk through the use of successive cat bonds: $160 million in 2006, $290 million in 2009 and $315 million in 2012.Footnote 3 These cat bonds have been well received by investors with competitive pricing and have evolved to cover increasing levels of risk.
SDRFI strategies are not limited to individual country programmes. For example, regional strategies such as the Caribbean Catastrophic Risk Insurance Facility (CCRIF) can also provide effective disaster risk management. CCRIF is the world’s first regional catastrophe insurance facility and uses parametric insurance to give participating governments quick, short-term liquidity to finance response and early recovery in the event of major earthquakes or hurricanes. CCRIF currently pools risk across 16 countries, and there are proposals to expand the facility to include additional beneficiaries. Other examples of regional strategies are the Pacific Catastrophe Risk Assessment and the Financing Initiative and African Risk Capacity. These recently created programmes are examples of donor-supported regional mechanisms offering quick-disbursing index-based coverage for tropical cyclone and earthquake, and drought, respectively.
The number and type of financial instruments to assist governments in managing their fiscal liability to disaster risk has grown exponentially in recent years. The World Bank and others are supporting countries to develop risk financing strategies that combine complementary instruments in a risk-layering approach. Risk retention instruments such as budget allocations, reserves and contingent lines of credit are used to finance recurrent events, and risk transfer instruments such as reinsurance and catastrophe bonds are used to provide additional financing for more extreme events. Yet, despite growing interest, the evidence on the impact, effectiveness and efficiency of SDRFI programmes remains limited.
There are very few evaluations of SDRFI programmes, and those that do exist typically do not directly address development impact or the impact of programmes on the poor. For example, whereas CCRIF is widely considered to be a success, a recent independent evaluation made it clear that evaluators lack the tools to assess performance beyond simple indicators such as the number of policies sold or the number of claims paid. In general, there is limited evidence on the relative development impact of alternative SDRFI solutions and there is very little evidence from any programme on the potential (expected) development impact of SDRFI investments as compared with other interventions, such as investments in physical risk reduction and mitigation.
For governments and donors, such lack of evidence acts as a clear barrier to scaled up investments in SDRFI programmes. There is currently no widely accepted framework for evaluating the development impact of SDRFI strategies. More common decision-making tools such as cost-effectiveness and pilot studies cannot be directly applied to the evaluation of SDRFI strategies. The low frequency of disaster events and the limited previous experience in responding to these events provides a challenge to establishing appropriate data sets for evaluation and analysis of strategies.
Furthermore, a key challenge in evaluating SDRFI strategies is that they are not an end in themselves. Rather an SDRFI strategy allows governments the necessary liquidity in the aftermath of a disaster. Defining a counterfactual (“what would happen without this strategy?”) is therefore formidable as it requires assumptions about what government action may be in the aftermath of a disaster both with and without the strategy. In particular, when evaluation is based upon humanitarian indicators, financial liquidity does not translate directly to humanitarian outcomes. More often than not, the type, speed and institutional framework of expenditures also play a significant role in the intervention outcomes.
One of the reasons to improve the evaluation of SDRFI is to enhance the evidence for decision makers. Given this goal, the framework developed needs to encompass forward-looking evaluation to determine an optimal SDRFI strategy. Such a forward-looking framework needs to cope with the substantial uncertainty surrounding the impact of an SDRFI strategy. First, there is uncertainty in the physical parameters of the natural disaster itself, such as the size, location, type and frequency. Second, there is uncertainty in the extent of impact (physical damage, economic loss, life loss) that follows a natural disaster. Whilst, third, there is uncertainty in the response, recovery and implementation of an SDRFI strategy. These uncertainties differentiate the evaluation of SDRFI from other development strategies.
Developing a robust and widely agreed evaluation methodology to assess existing SDRFI schemes and new products is critical – both for donors supporting these schemes and for the countries themselves as the principal investors. In an effort to better understand the impact and effectiveness of financial protection strategies, the World Bank and the Global Facility for Disaster Reduction and Recovery (GFDRR) have partnered with the UK Department for International Development for the SDRFI Impact Appraisal Project. The project uses probabilistic disaster risk models to evaluate selected SDRFI programmes based on simulated scenarios. It aims to take the financial resilience agenda to the next level: not only helping countries be financially prepared for disasters, but through improved evidence and forward-looking appraisal ensure that this is done in the most cost-effective way. Section 2 of this paper summarises the main principles for the development of an operational framework to evaluate SDRFI programmes. The implementation of this framework is through five country case studies to be carried out in 2014 and 2015. Section 3 concludes with a discussion of further research gaps that will be filled in the next stages of the project.
2. A Framework for Impact Appraisals
2.1. Key Evidence Gaps
In order to develop the principles for an operational framework for the evaluation of SDRFI programmes, several evidence gaps have been identified. Of the evaluations of SDRFI programmes that do exist, methodologies tend to be multi-method, using document review, empirical interviews, field missions and project sampling (Hinds, Reference Hinds2013). A Number of evaluations highlight the lack of monitoring and evaluation frameworks as an impediment to in-depth evaluation (IEG, 2012), whereas The Grantham Research Institute on Climate Change and Environment (GRICCE) asserts that assessing the effectiveness and sustainability of risk transfer schemes remains a challenge (GRICCE, 2011).
Mahul & Boudreau (Reference Mahul and Boudreau2010) analysed 21 GFDRR-funded projects that have SDRFI as one of their primary components. Hinds (Reference Hinds2013) highlights that this review did not adopt evaluative methodologies, but rather presented some key impacts and lessons learned from DRFI activities. These include:
∙ The development of DRFI strategies is highly technical, requiring extensive risk assessment and loss modelling. Developing counties often lack the technical capacity to develop DRFI schemes, therefore extensive capacity building must accompany technical assistance.
∙ SDRFI is a new, complex and evolving field which many in the development risk management community may not fully understand. Disaster risk financing and insurance requires extensive knowledge management through outreach materials, workshops and other events.
∙ An assessment of the fiscal impact of natural disasters and identifying potential budget gaps in post-disaster needs are crucial steps in engaging in dialogue with Finance Ministries.
An evaluation of the World Bank’s CAT-DDO for Guatemala highlighted that a better definition of outcomes and indicators would have made it easier to measure progress and evaluate accomplishments (World Bank, 2013). Meanwhile, a GFDRR/World Bank evaluation of Mexico’s Natural Disaster Fund suggested additional analysis is required to strengthen the understanding of disaster risk (GFDRR, 2012).
To address the lack of evidence in these areas, the Impact Appraisal Project has commissioned 16 research papers to address six themes.Footnote 4 These themes are the impact of SDRFI, the impact appraisal methodology, public economics and macroeconomics, microeconomics, catastrophe risk modelling and actuarial evaluation of instruments. The key findings in each theme are set out below.
2.2. Impact of SDRFI
Dana & von Dahlen (Reference Dana and von Dahlen2014) introduce seven ways in which an SDRFI impacts the actions of sovereign governments and decision makers. Given the nature of the categories, it is unlikely that the quantification of benefits for the purposes of evaluation will not be possible for all categories. Nonetheless, the nuances of how SDRFI impacts disaster recovery is important to understand in itself even if not necessarily for the purposes of evaluation. Thereby, these categories of impact shown in Figure 1 will be considered in greater depth below:
a. Access to capital: Disasters provide opportunities for high-impact public expenditure that can promote recovery, enhance equality and stimulate growth (Skidmore & Toya, Reference Skidmore and Toya2002). However limited resource availability, especially in low-income economies, can constrain such expenditures. A critical impact from an SDRFI programme is the availability of capital following a disaster. This is particularly crucial to countries with prohibitive borrowing constraints that may face difficulty accessing capital markets following a disaster. Highly developed countries tend not to have this issue and hence their SDRFI programme may rely more heavily on borrowing (Kunreuther & Linnerooth-Bayer, Reference Kunreuther and Linnerooth-Bayer2003). For less-developed countries, an SDRFI programme can have additional benefits by providing improved credit ratings (Fattal, Reference Fattal2014).
Although the accessibility of capital is an outcome in itself, the benefit of access may be best accounted for through the impact on macroeconomic stability and the reduction in disruption to public finances. In particular, the opportunity cost of diverting budgets to disaster spending can have undesirable consequences for existing projects (Bevan & Cook, Reference Bevan and Cook2014). The impact on the macroeconomy and public finances is further explored in section 2.4.
b. Speed of response: The speed of response depends on the speed of access to capital. Dana & von Dahlen (Reference Dana and von Dahlen2014) illustrate that the speed of access to capital differs depending on the type of instruments engaged. For example, the authors report that Colombia requested a contingent loan on 24 December 2010 following major flooding and disbursements were made on 27 December 2010. The authors also posit that risk transfer instruments such as reinsurance and insurance arrangements achieve fast disbursements (von Peter et al., Reference von Peter, von Dahlen and Saxena2012).
Similarly, in theory catastrophe bonds should also provide a quick disbursement of funds. However, thus far the experience has shown a time lag. For example, the Tohoku earthquake that affected Japan in March 2011 triggered the full principal loss of the USD 300 million Muteki Ltd. bond issued in 2008 by MunichRe (2011). However, the triggering of the bond took over a month to determine and was reported on 27 April 2011 (AIR Worldwide, 2011).
With access to funding, early response following a natural disaster can save lives in a number of ways including improved sanitation and access to food. For example, Watson et al. (Reference Watson, Gayer and Connolly2007) find that the displacement of populations, lack of sanitation and contaminated water increase the risk of communicable disease. Further, Clarke & Hill (Reference Clarke and Hill2013) show that a perfectly targeted food security response of USD 400 to an affected household within 2 months of the period of harvest can lead to a benefit worth USD 1,294.
c. Autonomy: SDRFI programmes can provide governments with an important degree of autonomy, the impact of this autonomy can be both positive and negative. With autonomy comes responsibility to use funds in an appropriate and efficient manner. On the one hand, autonomy can provide decision makers with the means to carry out important public investments and local sourcing of goods and services to support response and recovery.
On the other, autonomy can provide incentives for governments to use funds to extend their own political goals. For example, the declaration of a disaster is a heavily political decision that is at the discretion of the prime minister/president in most countries. Reeves (Reference Reeves2011) finds that electoral competitiveness of a state influences the declaration of a disaster in the United States and highly competitive states are twice as likely to receive a disaster declaration.
d. Market signals: SDRFI programmes provide sovereign governments with the opportunity to send clear signals about what risks the government will cover and what the government will do to respond to a disaster event. In particular, this can help inform individuals of their risk and the extent of government support that can be provided. Nonetheless, even with clear signals it is hard to change individual’s established decisions. de Janvry (Reference de Janvry2014) presents examples and a proposal for how such impacts could be quantified.
e. Knowledge: Whilst quantifying the benefits of additional knowledge can be extremely challenging, it is nonetheless an impact that can be identified if knowledge leads to action.Footnote 5
Knowledge in this framework stems from two avenues: risk assessment and risk pricing. These two aspects can aid a government in determining their level of risk and types of risks that they may face. In addition, this knowledge aids decision makers in planning for disasters, as well as determining a cost-effective way to manage the risk.
f. Discipline: SDRFI strategies have the potential to help governments overcome some of the more challenging issues related to discipline by promoting policies that are credible, predictable, transparent and sustainable.
g. Financial management capacity: Implementation of an SDRFI programme can support the building of financial, legal and regulatory capacity within a country. In due course this increased capacity can be applied to other types of high-value public financial transactions.
Sovereign disaster risk financing and insurance (DRFI) impact.
Depending on the context and recent history, it may be possible to accurately identify some of these types of impact through careful ex-post assessments. For example, de Janvry (Reference de Janvry2014), Bevan & Cook (Reference Bevan and Cook2014) and Hallegatte (Reference Hallegatte2014) suggest that questions such as how the establishment of SDRFI programmes can change ex-ante behaviour, what the impact of cuts and spending was and how large macroeconomic multiplier effects are could all be identified through carefully designed studies.
Nevertheless, in general it will be necessary to build appraisals on the foundations of catastrophe risk models. Whilst historical or hypothetical scenarios can be useful for understanding costs and benefits, scenarios alone cannot provide a solid foundation for appraisal. Evaluating SDRFI programmes will clearly require both further investments in the generic evidence base and carefully calibrated structural models build on catastrophe risk models.
2.3. Impact Appraisal Methodology
The development of an impact appraisal approach seeks to clarify the impacts of SDRFI strategies from both the perspective of benefits and costs. Particular difficulty lies in determining what sort of measures to use and how to identify the benefits that flow from the SDRFI programme.
In the assessment of benefits it is necessary to take into account how funding from an SDRFI strategy will be spent and the effects that these expenditures will have. This places a formidable burden on quantifying the benefits of an SDRFI programme. On the side of costs, there are the immediate costs of financing an SDRFI strategy and also the opportunity costs of directing attention and finances away from other programmes.
In general, an SDRFI programme with poor budget targeting and execution or high leakage will have low development impact, regardless of how cost-effective the financing strategy. For example, holding a reinsurance policy that provides a government with speedy access to funds in the aftermath of a disaster will have low development impact if the government’s execution of these funds is slow and poorly targeted. By contrast, acquiring and using additional financing to quickly scale up an efficient social safety net may have very high development impact. In either case, it is not possible to directly assess the development impact of SDRFI strategies in isolation from how the financing is being used.
Underlying these benefits and costs is layers of uncertainty. The identification of impacts from an SDRFI strategy depend heavily on the effect of a disaster. The effects of a disaster in itself are uncertain as is the probability that a disaster will occur. The impact appraisal methodology thereby uses a framework of underlying probabilities of impact caused by natural disasters with the intersection of an SDRFI strategy. The approach provides an impact distribution for each SDRFI strategy. The impact appraisal approach is forward-looking and aims to aid decision makers in choosing an appropriate risk management strategy in the face of natural disaster risk (Teh, Reference Teh2014b).
Whilst there are a number of potential questions that could be asked of SDRFI programmes, one useful question for decision makers is “what is the ‘average annual’ impact of the programme?”. The benefits from an SDRFI programme are probabilistic and typically arise only if a disaster occurs. Even after 10 years of experience from an SDRFI programme, the actual experience is unlikely to accurately reflect the prospective disaster risk faced in a probabilistic sense. For example, if no disasters occur at all during this period then the reinsurance policy may appear to have been poor value, whereas if a 1-in-100-year disaster occurred then the same reinsurance policy may appear to have been excellent value. Of course, experience from an SDRFI programme can still act as a useful input into an evaluation framework, for example, to understand the effectiveness of post-disaster targeting and budget execution. However, it can only ever be one input into an evaluation framework and would need to be adjusted to reflect an objective view on the prospective disaster risk faced.
Teh (Reference Teh2014b) motivates and defines an impact appraisal approach to SDRFI, which acknowledges the annual average impact as a recognised intermediate output on the way to a more sophisticated analysis. Teh (Reference Teh2014b) outlines an impact appraisal approach as a technique for analysing the impacts of SDRFI strategies and provides some procedures for the interpretation of the appraisal results. The process of interpretation acknowledges the possible existence of tipping point thresholds and levels of risk that are particularly important to decision makers and can be highlighted in the analysis of SDRFI. The approach suggested is similar to a probabilistic cost-benefit analysis, as in Clarke & Hill (Reference Clarke and Hill2013) in the analysis of the African Risk Capacity Facility, but where the probabilistic distribution of opportunity costs and household-level impacts are recognised intermediate outputs as steps on the way to a full cost-benefit analysis.
The first step in the framework is to specify a credible joint strategy for contingent public expenditure and for financing this expenditure. For example, this might involve specifying how a social safety net would scale up in a disaster year and how this scaling up would be financed using a combination of budget allocation and risk transfer instruments. Specifying a credible joint strategy will require expert understanding of both the post-disaster fund execution capacity of the country, and the budgetary and risk financing instruments employed. The joint strategy is then linked to a natural disaster risk model to simulate the likely impact of such a joint strategy.
However, the evaluation of the costs and benefits of a proposed SDRFI strategy does not determine the impact of the strategy until compared against a counterfactual. A counterfactual provides the benefits and costs that would be incurred without the SDRFI. Evaluating a clear counterfactual: “What would have been done and how would this have been financed in the absence of the SDRFI programme?”, is an onerous task.
Defining a clear counterfactual can be challenging owing to the fungibility and ad hoc nature of post-disaster public expenditure (Teh, Reference Teh2014a). For example, suppose a government purchases a reinsurance policy to finance the scaling up of a social safety net in the aftermath of a potential large disaster. If the government would have partially scaled up the social safety net anyway by reallocating budget from other sectors, then the SDRFI strategy is partially acting to release post-disaster resources for the government to spend on budget items other than the social safety net. In addition, if the reinsurance premium is being financed by resources being reallocated from other programmes then opportunity costs of funding must be included in the calculation of costs. Alternatively, if the premium is being financed from additional revenue then the public cost of funds must be accounted for.
Teh (Reference Teh2014a) suggests as a starting point three bounding counterfactual scenarios that can be considered and used consistently across evaluations. The grounding of counterfactuals by some bounding scenarios increases the transparency of what could otherwise be interpreted as an arbitrary approach. The bounding scenarios suggested are business as usual, ex-post response and full insurance. The business as usual strategy aims to elucidate the impact of changing the current risk management strategy to a proposed SDRFI strategy. Whilst the ex-post response considers no prior risk management strategy and estimates at the higher end of benefits from some forward planning through an SDRFI strategy and full insurance reflects the impacts of a natural disaster that cannot be minimised by having full funding available in the aftermath.
Another key issue in developing an impact appraisal methodology is assumption setting. Grenham (Reference Grenham2014) suggests that an actuarial approach could be utilised to set assumptions for the evaluation of SDRFI strategies and presents a guide to setting assumptions in this framework. The paper emphasises that assumptions should be consistent and objective whilst advocating a principles-based approach, over a more prescriptive rules-based approach.
Ley-Borras & Fox (Reference Ley-Borras and Fox2014) describe how catastrophe risk models can be used for evaluating SDRFI instruments and strategies. The authors suggest multiple interactions between the population distribution, infrastructure characteristics, severity of disaster, funding availability and response effectiveness, that are affected by the SDRFI strategy from both the financing perspective and expenditure response. Given these interactions, the authors conclude that catastrophic risk models developed for SDRFI evaluation need to be dynamic and map changes over time. In addition, a model should acknowledge the varying levels of uncertainty across interactions and allow for simultaneous appraisals that reflects the interrelated nature of the strategy.
The notion of an impact appraisal methodology for SDRFI is both complicated and daunting. It requires an interdisciplinary approach combining the knowledge of catastrophe modelling specialists, actuarial assumption setters, public finance advisors and economists. Nonetheless, the methodology can provide crucial information for decision makers and improve the knowledge and awareness of risk management strategies. It is hoped that these advances will lead to improvements from a humanitarian, institutional and economic aspect.
2.4. Public Finance and Macroeconomics
Disasters provide opportunities for high-impact public expenditure when funds are mobilised and executed effectively. In the aftermath of a disaster, there are typically a number of opportunities for public expenditure that have high development impact and address social inequality (Masozera et al., Reference Masozera, Bailey and Kerchner2007). Depending on the nature of the disaster, these include activities such as supporting livelihoods, rebuilding critical public infrastructure (e.g. water, electricity and key transportation lines), supporting the rebuilding of low-income housing and rebuilding/repairing all affected public infrastructure. However, if budget mobilisation is not timely, or if expenditure is poorly targeted or executed, the development impact of post-disaster, public expenditure can be much lower. For a poorly targeted or executed expenditure plan, the marginal impact of additional post-disaster public expenditure may not be much higher than the marginal impact in normal years.
Bevan & Cook (Reference Bevan and Cook2014) consider how disasters affect the composition and level of public expenditures and how to value the arising impacts. They argue that disasters may induce changes in the level, timing and composition of spending. However, without further research the authors concede that it is difficult to determine the likely nature of these expenditure reallocations and adjustments. Bevan & Cook (Reference Bevan and Cook2014) focus on the marginal social benefit (MSB) as a way to measure the impact of public expenditure. The authors suggest that priority sectors following the natural disaster should have a higher MSB than non-priority sectors. The priority level of sectors is determined by consideration of the disaster effects. Consistent with the literature on general budget support they suggest that heroic assumptions will be necessary for evaluating the impact of SDRFI strategies.
The identification of previous studies of budget reallocation point to the difficulties in identifying the impacts of shifts and additional expenditure. For example, a major joint evaluation of Partnership General Budget Support in 2006 concluded that “study teams could not confidently track distinct (separately identifiable) Partnership General Budget Support effects to the poverty impact level in most countries”. In a post-disaster situation this task may be made more difficult as the speed and manner of decision making may change. Tracking the use of additional post-disaster financial resources may prove even more challenging.
Hallegatte (Reference Hallegatte2014) shows that the welfare impact of a disaster depends significantly on the macroeconomic resilience of the economy to natural disasters. Here, resilience is decomposed into two components: instantaneous resilience, that is, the ability to limit the magnitude of the immediate loss of income for a given amount of capital losses; and dynamic resilience, that is, the ability to reconstruct and recover quickly. The paper proposes a rule of thumb based on the interest rate, reconstruction duration and the level of idle resources in the economy to estimate macroeconomic resilience.
In addition, theoretical foundations suggest that the standing of sovereign governments in capital markets can also be influenced by the engagement of an SDRFI strategy. Fattal (Reference Fattal2014) presents a macroeconomic model where countries are exposed to disaster risk but able to utilise insurance. Within this framework it is shown that countries with disaster risk, face lower welfare owing to lower levels of consumption and higher savings in equilibrium. However, the introduction of insurance can help smooth consumption paths and bring countries closer to optimality.
The issues facing the evaluation of the macroeconomic and public finance impacts of SDRFI require significant assumptions. The first set of assumptions are to define how public expenditure is financed, the timeliness of execution and the categories of spending. The second set of assumptions are to ascertain the impact of these expenditures in order to evaluate the benefits and costs of government actions. The setting of these assumptions relies heavily on data that as yet may not be available. Modelling, anecdotal evidence and interviews with decision makers may serve as a good indication of the estimation of these assumptions (Bevan & Cook, Reference Bevan and Cook2014).
2.5. Microeconomic Impacts
The most successful policy implementations aim to provide positive impacts at a macroeconomic level as well as at a microeconomic level. In order to quantify benefits it is necessary to determine the metrics by which success is to be measured. Within the field of development there are a range of commonly used household-level indicators. These indicators could provide the required measures for estimates of the annual average development impact of a potential or current SDRFI programme. Although the accuracy of such indicators may vary depending on how much information has been collected and analysed and whether the programme has been implemented, a single set of indicators may be appropriate throughout the policy cycle.
Scott & Shepherd (Reference Scott and Shepherd2014) present such indicators, along with further metrics that could be used to measure annual average development impact. Their paper introduces a range of microeconomic indicators commonly used to measure development outcomes, such as household consumption, household assets, measures of human capital (health, education and nutrition), food security and measures of coping (insurance and social capital), and describes the survey typically used to calculate them. Examples of such impact and outcome-based indicators include:
∙ average annual reduction in number of people in poverty;
∙ average annual reduction in number of people falling into poverty as a result of a disaster;
∙ average annual per capita increase in household consumption of poorest 40% of population;
∙ average annual per capita increase in net present value of income of poorest 40% of population;
∙ average annual reduction in mortality;
∙ average annual reduction in number of days of sickness since previous time period;
∙ average annual increase in child enrolment in primary/secondary school;
∙ average annual increase in value of agricultural assets/livestock/non-agricultural business assets;
∙ average annual per capita increase in household food consumption or average increase in number of meals eaten per day;
∙ nutritional indicators such as annual average reduction in stunting or annual average increase in body mass index.
The authors also present an overview of the relevant data sources available in ten countries to illustrate the possible data shortcomings that will be faced during evaluation.Footnote 6
Anttila-Hughes & Sharma (Reference Anttila-Hughes and Sharma2014) outline the major econometric issues involved in incorporating research from the growing literature on the microeconomic impacts of disasters into a catastrophe model framework. The authors draw attention to issues arising from the generally low recurrence frequencies of disasters, the likely role of difficult-to-document indirect damages in influencing total disaster costs and issues related to generalising disaster response functions across different domains. The authors end by noting the large discrepancy between the current state of the literature on disaster impacts on microeconomic indicators and the level needed for adequate catastrophe risk model performance. In concluding the authors suggest both means of closing that gap as well as potential areas for future research.
de Janvry (Reference de Janvry2014) discusses a range of microeconomic aspects of SDRFI evaluation including: (1) use of experimental games to analyse the private willingness-to-pay for disaster protection through risk transfer or risk retention instruments; (2) use of ex-post analysis of existing SDRFI schemes (such as Mexico’s FONDEN, PACC and Fondos) to analyse the willingness to provide political support to such schemes; (3) use of ex-post analysis of existing schemes to analyse not only the ex-post shock coping but also the ex-ante risk management impact of SDRFI schemes, with the expectation that the latter can have large effects on growth; and (4) use of mainly global data to do ex-post impact analysis of natural disasters and the resilience-enhancing value of SDRFI schemes (examples exist for the disaster–impact relationship that can be extended to the role of SDRFI in risk reduction, shock coping and risk management).
It is likely that the implementation of an SDRFI strategy will have some microeconomic impacts on households. In particular, development indicators such as food consumption or days of sickness may be affected by an SDRFI strategy. The determination of the microeconomic impacts of SDRFI strategies is important to justify the implementation of such strategies, as well as to choose between strategies. Importantly, the focus on quantifying recognisable development indicators will allow the comparison of SDRFI strategies against other development interventions.
2.6. Catastrophe Risk Modelling
Catastrophe risk modelling is an essential and important part of the impact appraisal methodology. Catastrophe risk modelling focuses on the physical characteristics of a disaster, likelihood of disaster and the effects of a disaster. Integrating economic models with catastrophe risk models is necessary to determine the impact of an SDRFI strategy. Once integrated, “annual average development impact” indicators may be supplemented by “1-in-5/10/50-year development impact” to reflect the modelled development impact in a range of disaster scenarios. Indicators that capture the development impact in extreme scenarios may be of particular interest to decision makers as short time periods spent in extreme poverty can have long-run consequences, both for health outcomes (particularly if there are young children in the household) and for livelihoods (particularly if households have to engage in forced selling of assets to meet basic consumption needs).
The paper by Anttila-Hughes & Sharma (Reference Anttila-Hughes and Sharma2014) provides a brief outline of cat risk models as they currently exist, and outlines the major econometric issues involved in incorporating research from the growing literature on the microeconomic impacts of disasters into a model framework. A disaster risk model to be used for the evaluation of SDRFI programmes must be able to jointly model post-disaster public expenditure opportunities and disbursements from risk financing instruments. In the above framework the same probabilistic disaster risk model is used to move from isolated scenarios to “annual average” or “1 in X year” figures both on the expenditure and the financing side. If indexed financial instruments (e.g. parametric reinsurance or catastrophe bonds) are to be considered as financing instruments the risk model must therefore capture the correlation between public expenditure opportunities and the income from these instruments. This allows proper modelling of the extent to which expenditures match liabilities.
As with all information feeding into the evaluation process, the authors note that risk models should be able to be subject to sensitivity analysis. For example, when modelling government’s contingent liability for post-disaster reconstruction of public assets, incomplete exposure databases can lead to large model uncertainty, regardless of how accurate the hazard or vulnerability components of a model are. Similarly, when modelling food security response incomplete information about the geographical location and baseline vulnerability of people can lead to large model uncertainty that needs to be reflected in the calculation process. This model uncertainty should be conveyed through the evaluation through sensitivity analysis.
Muir-Wood (Reference Muir-Wood2014) explores how the methodologies of probabilistic catastrophe loss modelling could be employed to investigate the impact of disasters on poverty. It is argued that impacts on poverty can be captured by employing four key model outputs: (a) direct damages and loss, (b) indirect economic impacts and disruption, (c) deaths and injuries and (d) losses to household “agriculture”. These outputs could become the basis for the determination of benefits and costs from SDRFI.
In a more specific setting, Sharma & Hohl (Reference Sharma and Hohl2014) discuss how catastrophe crop risk models can inform assessments of food security needs at the sovereign level for developing countries. The authors note that current food security vulnerability models are simple and transparent making them easy to use but may limit the detail that such a model can provide.
The development of catastrophe risk modelling that provides household-level outcomes is increasing with improvements in technology and knowledge. However, these models still depend heavily on data at the household level that may not be comprehensively available. Accuracy, transparency and fineness of outputs may also require further tailoring to match the goals of impact appraisal. Nonetheless, the engagement of additional case studies can advance the necessary refinements and outputs from catastrophe risk models.
2.7. Actuarial Evaluation of Instruments
Actuarial techniques have long been utilised in the area of risk transfer and insurance. In the case of evaluating an SDRFI strategy, these techniques can be invaluable in assessing uncertainty and risk. The use of risk transfer instruments such as insurance, catastrophe bonds and contingent loans can benefit from actuarial methods when determining costs and benefits.
Clarke & Poulter (Reference Clarke and Poulter2014) utilise a range of techniques to address the issue of determining the annual average cost of an SDRFI strategy. The determination of the cost of an SDRFI strategy combines evidence on the cost of public (or donor) funds with evidence on the cost of SDRFI instruments. In general, evidence from public finance on the cost of public funds or the opportunity cost of reducing other expenditures allows for the estimation of the cost of state-uncontingent financing, such as budget reallocation and borrowing.
For state-contingent financing, costs are determined by using indicative multiples over the base cost of providing financing. The authors focus on tractable methods that are simple to calculate and easily applicable to all SDRFI strategies. The authors provide a collection of simplifying assumptions drawn from practice, that ensures the formulae are understandable and quantifiable. The formulae provided are well justified by records of best practice and established rules of thumb and present a method to calculate the opportunity cost of risk transfer, ex-ante reserve funds, contingent credit, ex-post budget reallocation and ex-post borrowing.
Grenham & Foster (Reference Grenham and Foster2014) consider the issues that arise owing to the fact that decisions to implement a specific SDRFI strategy or to decide between alternative options will be made by ministers, senior policy makers or civil servants. These decision makers may not be experts in all or any of the areas that have been used in calculating the projected cost and benefits of the various SDRFI strategies. This may lead to incomplete communication of alternative strategies and the misinterpretation of presented costs and benefits. Grenham & Foster (Reference Grenham and Foster2014) set out how data should be presented to aid interpretation and where necessary challenge the recommendations made by the technical experts.
3. Conclusion
While challenging, evaluating SDRFI programmes from a humanitarian perspective is a necessary precondition to enable future investment in this area from governments and donors. The impact appraisal methodology aims to provide crucial information to enhance the capacity of policy makers to make critical decisions concerning SDRFI. The methodology aims to identify when SDRFI programmes are effective components of a comprehensive approach to managing the financial risk associated with disasters.
This paper summarises the initial findings of the technical background papers in the first phase of the Impact Appraisal Project. The key synthesis of this result is the development of an approach to the evaluation of SDRFI strategies. Although the evaluation process relies on modelling and assumptions, the approach ensures that benefits and costs, as well as the impact of an SDRFI strategy, can be mostly quantified. This quantification in terms of key development indicators brings SDRFI into the realm of other development interventions and allows a comparison against alternative disaster responses. This is important when a government is faced with deciding between risk mitigation, reduction, response and management.
The operational framework provided by this set of papers begins to provide answers on how insights, analysis and evidence from a diverse range of technical disciplines can be combined within a coherent quantitative ex-ante framework to evaluate the impact of SDRFI programmes on poverty reduction and development efforts. An important aspect of the background papers is the involvement of a range of experts from various disciplines. It is clear that to create a cohesive and appropriate approach to SDRFI evaluation, risk modellers, microeconomists, macroeconomists, actuaries and governments will need to work together relying on their professional experience and technical knowledge.
Evidence on the impact of SDRFI strategies is lacking. This paper has brought together the results of 16 background papers on the topic, seeking to provide a possible pathway towards evaluation. Nevertheless, there are still research gaps that will be considered in the next phases of the project including the political impacts of SDRFI and finding better methods to integrate risk models with economic models.
It has been noted by several researchers that expenditure on risk mitigation is much less politically attractive than expenditure on risk response (Healy & Malhotra, Reference Healy and Malhotra2009). It is crucial to the success of SDRFI to understand how expenditure on SDRFI compares to these other disaster-related actions and the political aspects of such expenditure. In the end, even if the impact of SDRFI is largely beneficial, this may not result in implementation if the political impetus is not available.
The necessary research into integrating risk models and economic models is linked to five country case studies. These case studies will provide an initial test of the impact appraisal methodology to allow for further refinement. In several cases, it is anticipated that extensive work will be required to collate the data required to complete the analysis. Although this may be an early initial hurdle for the impact appraisal methodology, such obstacles can provide insight into the improvements required for the modelling component of the evaluation. In particular, the case studies offer a unique opportunity to understand the intricacies of utilising risk models in combination with economic models.
The impact appraisal methodology represents an interdisciplinary approach to the evaluation of SDRFI strategies. SDRFI strategies are a relatively new disaster risk management approach. Nonetheless, low- and middle-income countries have shown increasing enthusiasm to engage in SDRFI. This enthusiasm has driven a need to develop an impact appraisal methodology for SDRFI evaluation and greater research to provide evidence and analysis for decision makers. This project and the background papers presented in this paper aim to directly address this demand.
Acknowledgement
This research has been funded by the UK Department for International Development (DFID), the World Bank Group and the Global Facility for Disaster Reduction and Recovery (GFDRR). The author would like to thank Daniel Clarke, Richard Poulter and all members of the sovereign DRFI research team.
