Evidence on result-based financing in maternal and child health in low- and middle-income countries: a systematic review

Abstract

Introduction

Result-Based Financing (RBF) is an umbrella term for financial mechanisms that link incentives to outputs or outcomes. International development agencies are promoting RBF as a viable financing approach for the realization of universal health coverage, with numerous pilot trials, particularly in low- and middle-income countries (LMICs). There is limited synthesized evidence on the performance of these mechanisms and the reasons for the lack of RBF institutionalization. This study aims to review the evidence of RBF schemes that have been scaled or institutionalized at a national level, focusing on maternal, newborn, and child health (MNCH) programming in LMICs.

Methods

A systematic literature review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The authors identified and reviewed country-level RBF evaluation reports for the period between January 2000 and June 2019. Data were extracted from both published and gray literature on RBF application in MNCH using a predesigned matrix. The matrix headers included country of application; program setting; coverage and duration; evaluation design and methods; outcome measures; and key findings. A content thematic analysis approach was used to synthesize the evidence and emerging issues.

Results

The review identified 13 reports from 11 countries, predominantly from Sub-Saharan Africa. Performance-based financing was the most common form of RBF initiatives. The majority of evaluation designs were randomized trials. The evaluations focused on outputs, such as coverage and service utilization, rather than outcomes. RBF schemes in all 11 countries expanded their scope, either geographically or accordingly in terms of performance indicators. Furthermore, only three studies conducted a cost-effectiveness analysis, and only two included a discussion on RBF’s sustainability. Only three countries have institutionalized RBF into their national policy. On the basis of the experience of these three countries, the common enabling factors for institutionalization seem to be political will, domestic fund mobilization, and the incorporation of demand-side RBF tools.

Conclusion

RBF evidence is still growing, partial, and inconclusive. This limited evidence may be one of the reasons why many countries are reluctant to institutionalize RBF. Additional research is needed, particularly regarding cost-effectiveness, affordability, and sustainability of RBF programs.

Introduction

Result-Based Financing (RBF) is an umbrella term covering a number of financing instruments that align incentives to outcomes [1]. Common types of RBF include performance-based financing (PBF), usually referred to as “pay for performance” or P4P; user fees exemptions; voucher programs; and conditional cash transfers (CCTs). These innovative financing instruments utilize the provision of incentives to healthcare providers and/or users to improve health outcomes.

The World Bank is leading the promotion and implementation of RBF projects in maternal, newborn, and child health (MNCH) in low- and middle-income countries (LMICs). The World Bank is also managing the Health Results Innovation Trust Fund (HRITF), a multi-donor trust fund. This fund is supported by the governments of Norway and the United Kingdom [2]. As of September 2016, the HRITF had committed US $385.6 million for 35 RBF programs in 29 countries [3]. Increasingly, other bilateral, multilateral, and philanthropic agencies are channeling some of their funding via RBF [4].

From around the 2000s, the donor community has been funding RBF pilot projects in LMICs, particularly those experiencing a slow progress in the Millennium Development Goals (MDGs) related to maternal and child mortality [5]. RBF is now seen as a strategic health care financing mechanism with the potential to contribute to the achievement of universal health coverage (UHC) [6, 7]. UHC aims to enable all people to access the full spectrum of health care services while protecting them from financial risks associated with seeking these services [8].

Maternal mortality is unacceptably high with the vast majority of the deaths (94%) occurring in low-resource settings [9]. Existing literature suggests that low utilization of MNCH services is due to financial barriers, particularly among the poor [10,11,12]. Leveraging on effective and efficient health financing models, such as RBF, can potentially increase utilization on the demand side, enhance quality on the supply side, and improve health outcomes. Furthermore, RBF approaches used in MNCH have demonstrated significant increase in coverage and utilization of services [13]. By channeling resources directly to the point of use, RBF mechanisms equip frontline health care providers and managers with the financial capacity and autonomy to institute structural improvements required at the health facilities level, which can eventually improve health outcomes.

Many countries, however, have not institutionalized RBF by integrating such schemes into their national health systems [14, 15]. The reasons for the lack of integration are poorly understood. The aims of this study are to review the RBF schemes that have been scaled from an initial pilot – either geographically or by increasing the scope – and assess the evidence on effectiveness and cost effectiveness, including whether there are documented lessons on potential barriers and enablers to institutionalization. While strong evidence in favor of RBF may not necessarily translate into RBF institutionalization, an emerging body of literature from rigorous large-scale randomized trials has shown that policymakers are indeed receptive to such evidence [16]. Therefore, documenting evidence on the effect of country-level efforts can be an important step in determining the extent to which development agencies should continue to advocate for the institutionalization of RBF.

Methods

Study design

The authors conducted a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guide. The review is registered and published on PROSPERO, an international registry of systematic reviews (ID: CRD42019133119).

Study setting

This study focused on published and gray literature on country-level RBF evaluation reports. The authors reviewed evaluation reports for various RBF mechanisms being applied in MNCH. Sources were limited to reports from LMICs, which were defined based on the World Bank’s income-based classification [17]. The evaluation reports were predominantly from Sub-Saharan Africa.

Study period

RBF in MNCH in LMICs is a relatively new concept. Therefore, the authors reviewed RBF studies published between January 2000 and June 2019.

Search strategy

The authors retrieved published country-based RBF evaluation reports using the Web of Science, PubMed, and Google scholar databases following a PRIMSA guideline template (Fig. 1). Relevant records were obtained using the following predetermined search terms: (RBF “OR” Incentives schemes) “AND” (Maternal and Child Health Care “OR” Health Care “OR” Health) “AND” (RBF “OR” Output Based Strategies) “AND” (Impact in MNCH “AND” (RBF programs “OR” RBF projects “OR” Incentives based mechanics “OR” Health Financing “OR” PBF) “AND” (Low- and Middle-Income Countries “OR” Developing Countries).

Fig. 1

PRISMA flow diagram for the evidence on RBF mechanisms on maternal, neonatal, and child health in low- and middle-income countries

Inclusion and exclusion criteria

The inclusion criteria were country-based evaluation reports published between January 2000 and June 2019 for any RBF type in LMICs targeting MNCH and sources being available in English. The exclusion criteria were RBF study evaluation protocols, RBF mechanisms targeting sectors other than MNCH, and studies conducted in non-LMICs settings.

Data extraction and synthesis

The data from eligible evaluation reports [13] were extracted into a predesigned matrix table. The data included country of application; program setting; program coverage and duration; type of evaluation and methods used; outcome measures; and key findings. The first author (NJ) drafted the consolidated matrix, with the remaining two authors (YA and KL) assessing for consistency and accuracy. In order to evaluate the quality of the reports, the authors adopted the Cochrane Risk of Bias Assessment tool to assess potential selection, performance, and reporting bias. The first author assessed the level of bias (high, low, or unclear), and the co-authors reviewed the assessment. The overall level of bias reported for each study is based on the consensus of all three authors. The framework developed by Shroff and colleagues’ on RBF scale-up was adapted to assess each country’s institutionalization and scale-up progress [15].

Results

Sample of studies

The review retrieved 1489 records through the database search (Fig. 1). Of these, 802 were assessed for eligibility. Out of these 802 records, 713 were either not relevant to the research question or did not meet the inclusion criteria, leaving 89 records. Seventy-six of these 89 studies were based on projects that targeted areas other than MNCH or were not conducted in a LMIC, yielding 13 studies for the current review. Of the 13 records, three were based on Rwanda and the remaining 10 country-reports were from Afghanistan, Argentina, Benin, Burundi, Cameroon, Democratic Republic of Congo, Mozambique, Zambia, Zimbabwe and Nigeria (Table 1). Table 2 shows each country’s program scale-up level defined as either generation, adoption, or institutionalization. So far, Rwanda, Cameroon, and Burundi have institutionalized RBF as a national health financing policy.

Table 1 Description of RBF evaluation reports including evaluation methods and key findings, by country

Table 2 RBF scale-up framework

General features

The program’s implementation duration varied from two to five years. Zimbabwe and Benin reports were mid-line evaluations whereas the rest were end-line evaluations. Most of the studies were conducted as randomized trials; exceptions were those from Benin, Burundi, Mozambique, and Zimbabwe. In the absence of evaluation protocols to check selective reporting, the authors inferred the likelihood of bias based on whether the evaluation team seemed independent from the financing agency. Generally, the level of bias was low to medium (Table 3). The remaining sub-sections provide details on countries’ typical RBF types, evaluation methods, and evidence on cost-effectiveness.

Table 3 Risk of bias assessment

Common RBF approaches

RBF tools can be broadly classified into three categories: supply-side with a demand-side component (focus on provider), demand-side with a supply-side component (focus on provider and consumer), and demand-side with no supply-side component (focus on consumer) [18]. Previous reviews have assessed RBF evidence on one or more of these categories [32]. The country-level studies in the current review predominantly fell under the first category. All 13 studies implemented PBF-type programs that had incentives tied to volume, quality, or both.

Typical program setting involved the contracting of health facilities to offer MNCH services with an incentive tied to quantity, quality, or both. Afghanistan’s PBF intervention targeted health care providers in 230 health facilities, paying bonus payments of up to 10% of existing facility contracts to health facilities based on quantity and quality checklists [19, 33]. Argentina had a similar PBF model, except that payments were made through a national health insurance program that allocated funding to provinces based on enrolment of beneficiaries [20]. Health facility payments in DRC being tied to volume of services provided and not quality was the main difference between the PBF in DRC and those in Argentina, Benin, and Cameroon [20, 21, 23, 24].

The scheme in Rwanda, which was gradually expanded over time, provided both supply- and demand-side incentives. It provided: (i) in-kind incentives (gifts) to women, (ii) performance-based incentives to providers, and (iii) performance-based incentives to community health workers cooperatives for mobilizing mothers to access health services [27, 29]. Nigeria had a unique hybrid of RBF and Decentralized Financing Facility (DFF). In both RBF and DFF approaches, the recipient received direct funding and had autonomy over utilization of those funds. However, in the Nigeria’s DFF, the funds were not linked to quantity or quality of services delivered and the staff did not receive any performance bonuses [26].

Evaluation methodologies

Evaluation methods differed from country to country. The methods ranged from simple before-and-after comparisons to randomized control trials (RCTs). Of the 13 studies, eight were RCTs, one was a repeated cross section analysis, one was quasi-experimental, one was pre-post comparison, and one was case control. In most studies, randomization was at the level of the facility or higher, and the effects of the interventions were estimated using a difference-in-difference framework (Table 1).

The vast majority of the studies concentrated on output indicators such as antenatal care (ANC) booking rates and percentage of institutional deliveries, with little or no emphasis on quality or impact measures [5, 22]. Eleven countries reported an increase in utilization or coverage because of RBF. For example, a 34% increase in early ANC bookings was recorded in Argentina [20]. Rwanda recorded a 23% increase in institutional deliveries and a 56% and a 132% increase in preventive care visits for children age 0–23 months and 23–59 months, respectively [27].

The effect of RBF on health worker motivation in Zambia and the DRC was mixed. In Zambia, there was no significant improvement in staff motivation, whereas there was a 14% increase in the DRC. However, the effect in the DRC dropped by 25% 4 months after the incentives were removed [24, 30]. RBF in Rwanda and Cameroon had a significant incentive effect in increasing utilization and quality of care for the key MNCH indicators [23, 27]. There was no difference between the PBF and DFF approaches in Nigeria in terms of their effect on the quality of care. However, there were modest differences in the coverage of key services in favor of the PBF approach [26].

Economic evaluation

The authors analyzed and presented the economic evaluation results following the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) model [34] (Table 4). Three out of the 13 reports (Argentina, Zambia, and Nigeria) included an economic analysis [20, 22, 29]. All three reports provided a Cost Effectiveness Analysis (CEA). The cost-effectiveness estimates were derived from relatively short program implementation periods (2 years and 3 months in Zambia and 4 years in Argentina and Nigeria).

Table 4 Economic evaluation results - CHEERS model

When estimating costs, all studies factored in both fixed and variable costs incurred in program design; planning and management; and implementation. The total program costs for programs in Argentina and Zambia were US $106 million and $13.26 million, respectively [20, 30]. The hybrid PBF-DFF program in Nigeria cost US $132.9 million [26].

The reports based on Zambia and Nigeria calculated incremental cost effectiveness ratios (ICERs) comparing PBF to two comparison groups in each case (input financing and no intervention in the case of Zambia and DFF and no intervention in the case of Nigeria) [26, 30]. Depending on the comparison group, ICERs ranged between $809 per QALY gained and $1324 per QALY gained in Zambia (the corresponding range without adjusting for the quality of care was $999 to $1642). Likewise, ICERs ranged between $300 and $458 in Nigeria (between $698 and $796 without adjusting for the quality of care). For Argentina, cost effectiveness was estimated by dividing disability-adjusted life years (DALYs) saved due to RBF by incremental costs of the program. The estimated costs per DALY averted were $814, which was compared to the 2005–2008 per capita GDP of $6075 [20]. All three studies found RBF to be cost effective based on the countries’ annual GDP per capita [20, 26, 30]. This comparison between DALYs or QALYs against the country’s GDP per capita follows the World Health Organization guidelines on the evaluation of public health interventions [35].

Discussion

Although the development agencies have been encouraging many LMICs to adopt RBF as an important step toward UHC, RBF’s institutionalization remains limited. This study reviewed 13 country-specific RBF evaluation reports from 11 LMICs. In an earlier review similar to this review, Witter et al. [32] concluded that almost all dimensions of RBF impact were understudied for both intended and unintended outcomes. Unlike the earlier review, this review focused on country-level evaluations. While substantially more evidence exists now, the country-level evaluations have primarily focused on outputs rather than outcomes. In the logical framework often used for program evaluation, outputs are the immediate results that are delivered by a program whereas outcomes are the next level of effects resulting from the outputs [36].¹ Although both measures are useful in understanding the performance of RBF mechanisms, outcomes are more informative since they reveal higher level effects and are more useful for assessing return on investment of the mechanisms.

The improvement observed in structural quality indicators (outputs) at the health facility level is not surprising because RBF mechanisms channel resources to the point of use and foster local autonomy and capacity building.

Only three out of the 13 reviewed reports conducted a cost effectiveness analysis. Given the insufficient evidence on RBF mechanism’s cost effectiveness, the low number of countries to have institutionalized RBF is not surprising. The three studies with a CEA followed the World Health Organization’s GDP per capital threshold method to determine cost effectiveness. Some researchers have argued that this method may not be very useful to decision makers because it might not reflect national budget priorities, values, and country-specific contexts [37]. Nonetheless, evidence from Argentina, Zambia, and Nigeria suggests that RBF yields better returns on investments than traditional input-based financing strategies.

The current RBF implementation arrangements are complex and have high overhead costs, which can jeopardize the affordability and sustainability of RBF mechanisms even if they are deemed to be cost-effective [38]. Witter et al. [32] argue that paying for performance may not always be a good use of resources, even when it is effective, because the potentially small effects are achieved at high costs.

Only two out of the 13 reports in this review included a discussion on sustainability. In Mozambique, on average, it took 18 months of implementation for PBF to show effects, and the impact was generally sustained thereafter [25]. The mobilization of domestic financial resources was central to the sustainability of Burundi’s program [22]. The World Bank, the key proponent of RBF, recommends starting at a low and sustainable level of incentives and gradually increasing them based on a robust financial analysis. The World Bank further recommends that RBF should not be isolated from broader health systems reforms. Instead, it should be viewed as an entry point to tackling system-wide issues [31]. Beyond providing these general directions, the existing literature lacks a meaningful assessment of sustainability of RBF.

Relatedly, most RBF schemes piloted so far are donor funded [39]. Funding agencies view RBF as a good way to reduce the risk of investing funds when there is a possibility of the results not being achieved [39]. Unfortunately, the resulting dependency of the recipient countries on donors compromises the sustainability of RBF programs. If RBF is to make long-lasting impacts in LMICs, an appetite for reform needs to be created within the country. Simultaneously, the capacity to mobilize domestic resources for RBF needs to be built.

On the basis of the experience of the countries that have institutionalized RBF, the common enabling factors for institutionalization seem to be political will, domestic fund mobilization, and incorporation of demand-side RBF tools. For example, in Burundi, the government allocated 1.4% of its budget to PBF each year [22]. Rwanda expanded its PBF program to include a demand-side component that incentivized users [29]. In Cameroon, the government doubled its health sector budget to materialize RBF [23]. Insufficient political will and lack of domestic resources seem to be important challenges to institutionalizing RBF [15], which, of course, may be a reflection of a lack of local ownership and insufficient consideration of resource requirements when RBF is first prescribed to countries.

These findings should be understood in light of a number of caveats. Some of the evaluation studies included in the analysis were not conducted by independent evaluators. Rather, they were conducted by the funding agencies themselves, which raises concerns about the level of bias. The positive effects of RBF are likely weaker than reported in this study. Furthermore, the review only studied sources in English and may have missed relevant studies in other languages. Finally, the interventions analyzed were predominantly on the supply-side, leaving the vast number of financial protection-oriented RBF tools, such as user fees exemptions, voucher schemes, and conditional cash transfers to the users. The latter are important ingredients toward the achievement of UHC [3, 29, 40].

Despite these limitations, the policy implications of these study findings are clear. While political factors may be important in institutionalizing initiatives such as RBF in any country, the evidence on the effectiveness and effects of RBF is so far insufficient. Future research, at a pilot and country level, needs to continuously evaluate RBF schemes, and include qualitative and quantitative research to help define the conditions for successful scale-up, including affordability and sustainability.

Conclusion

RBF is being promoted as an innovative vehicle toward the achievement of UHC. This review has shown that, while the evidence on the effect of RBF is growing, this evidence is still limited and inconclusive, particularly in areas of cost-effectiveness, sustainability, and system-wide long-term impacts. This limited evidence and low local ownership may be some of the reasons behind countries being reluctant to institutionalize RBF. Additional research is needed, particularly on cost-effectiveness, health system-wide impacts, and sustainability of RBF programs.