Table 1 Description of RBF evaluation reports including evaluation methods and key findings, by country
Country, Authors, Year | Program Setting | Program coverage | Evaluation timing and program duration | Evaluation method and main outcome measures | Key evaluation findings |
|---|---|---|---|---|---|
Afghanistan Cyrus et al. (2015) [19] | A supply side P4P on selected MNCH indicators in 11 out of 34 provinces. Incentives tied to quantity of care delivered were provided quarterly to healthcare workers | 422 health facilities (230– Intervention, − 212–Control) | End-line evaluation Sept 2010- Dec 2012 | Method: Cluster randomized trial Outcomes: Contraceptive prevalence, proportion of deliveries with at least one antenatal care visit, skilled birth attendant, pentavalent 3 vaccination and service utilization | -No substantial effect in any of the five MCH coverage indicators (modern contraception, antenatal care, skilled birth attendance, postnatal care, and childhood vaccination, or in the equity measures -Substantial increases in the quality of history and physical examinations index and the client counselling index, as well as time spent with patients -The inattention to demand-side factors and difficulty in communicating to health workers about the intervention may have undermined the potential effects of the P4P intervention - More attention needs to be given to these factors in the design, management, and implementation of P4P programs |
Argentina Gertler et al. (2014) [20] | Supply side P4P national program based on an insurance program that allocated funding to provinces based on enrolment of beneficiaries and adding incentives based on indicators of the use and quality MNCH services | Nationwide | End-line evaluation 2004–2008 | Method: Cluster randomized trial Outcomes: Prenatal care visits, tetanus toxoid vaccine, caesarean section, APGAR score at 5 min | -19% lower chances of low birth weight -74% lower chances in hospital neonatal mortality -Early booking was 34% higher in treatment group with incentives |
Benin RBFHealth (2014) [21] | Supply side P4P linked to quantity and quality in 8 out of 34 districts | Four health facilities assigned to intervention arm and one health facility assigned to control | Mid-line evaluation 2010–2011 | Method: Mixed methods design, consisting of a randomized control trial and qualitative data Outcomes: Health worker motivation, ANC services utilization | -Thoroughness of physical examination and history taking in ANC higher in intervention group compared to the control groups -On average, four additional minutes per patient spent on ANC services -No significant effect on productivity or presence of staff in their posts -Greater level of client satisfaction on staff attitude and competence |
Burundi Bonfrer et al. (2013) [22] | A supply side PBF program that started off in one province, scaled up to nine before finally being rolled out nationwide | 3200 randomly sampled households 75 randomly selected health facilities from intervention and control provinces | End-line evaluation 2006–2008 (Phase 1) 2008–2010 (Phase 2) | Method: Repeated cross-sectional survey, analyzed in a difference-in-difference framework Outcomes: Institutional delivery, ANC services, vaccination coverages, ITN coverage, child illness episodes, waiting time | -PBF increased the probability of institutional deliveries by 21%, utilization of antenatal care by 7%, and the use of modern family planning methods by 5% -No effect on vaccination rates and user satisfaction -Government committed to allocate 1.4% of its budget to performance-based financing and related health financing strategies each year |
Cameroon De Walque et al. (2017) [23] | Payment of health facility bonus linked to volume and quality of services delivered in 14 districts in East, South West and North West regions | 14 health districts in the region randomized into four arms as follows: T1- P4P plus autonomy C1- Incentive not attached to performance plus autonomy C2- No incentives at all but autonomy C3- No incentive, no autonomy | End-line evaluation 2012–2015 | Method: Randomized control trial Outcomes: Child and maternal vaccinations, use of modern family planning, antenatal care visits, facility-based deliveries, patient satisfaction | -P4P efficient in bringing payments and funding to provider level, leading to an increased coverage of MNCH and structural measures of quality of care -Decreased OOP payments -No difference in MNCH outcomes between T1 and C1 -No effect observed on skilled deliveries and ANC visits - There was a clear effect of additional financing, irrespective of whether it was linked to incentives |
DRC World Bank (2015) [24] | Performance-based payments to health centres and referral centres using a “point system” linked to the volume of targeted services in post conflict Haut- Katanga District- DRC | One out of eight health district zones | End-line evaluation 2009–2013 | Method: Randomized control trial Outcomes: Cost to patients, health workers’ satisfaction, work-related stress and motivation, service utilization, patient satisfaction | -Increased tendency to over report on volumes, but the tendency fell with increased verification -Patient records and data quality better in intervention facilities -Greater transparency and equity in resource allocation among staff -Significant reduction in absenteeism -Increased community-based outreach effort -No change in quality of services in either targeted or non-targeted services -No effect in service utilization -Reduction in job satisfaction -Increased health worker motivation initially, which ultimately reduced intrinsic motivation post intervention |
Mozambique Rajkotia et al. (2017) [25] | Phased PBF programs in two provinces Nampula (North) and Gaza (South) targeting 18 MNCH and / HIV-PMTCT services | 134 matched facilities health facilities (84 in North, 50 in South) | End-line evaluation 2011- Sept 2013 | Method: Retrospective data (analyzed using propensity score matching) Outcomes: PMTCT, Paediatric HIV indicators vaccination coverages | - The majority of the 18 indicators responded to PBF, with at least half of the indicators showing at least 50% improvement from baseline -Pregnant women indicators (HIV-infected pregnant women initiating ART and family planning consultations for HIV-infected women) were the only adult HIV indicators that responded to PBF -No adverse effects on non-incentivized indicators |
Nigeria Kandpal et al., 2019 [26] | PBF and DFF hybrid approach to increase the delivery and utilization of high impact maternal and child health services in three states- Adamawa, Ondo, and Nasarawa | 52 Local Government Agencies-LGAs randomised into PBF or DFF and compared with traditional input financing matched states | End-line project evaluation 2012–2016 | Method: Randomized control trial (for PBF vs DFF comparison) or a quasi-experimental design (for PBF-DFF vs ‘business as usual’ comparison) Outcomes: Skilled birth attendance, fully immunized child, modern contraceptive prevalence, pentavalent 3 immunization, institutional delivery, antenatal care visits, equity, CEA | -Significant impact of PBF and DFF on key MCH services as well as quality of care (QOC) (relative to ‘business as usual’). For example, 14 percentage point increase in fully immunized child coverage and 4.5 percentage point increase in use of modern contraceptives -Limited difference in terms of QOC indicators and only a modest difference in terms of MCH services between PBF and DFF -Both interventions found to be cost-effective and likely to be successful due to decentralization of funds, autonomy given to the facilities, improved supervision, and investments in health systems management |
Rwanda Basinga et al. (2011) [27] | National supply side PBF program implemented at health facility level. | 166 district level facilities randomly selected. (intervention group n = 80), (control group n = 86) | 2006–2010 End line evaluation | Method: Randomized control trial Outcomes: Prenatal care visits, institutional deliveries, quality of prenatal care, child preventive care visits, immunization | - 23% increases in institutional deliveries in intervention group -56% increase in preventive care visits for 0–23 months age group 132% increase in preventive care visits for 23–59 months age group. - No improvement in the number of women completing four ANC visits or the number of children receiving full immunization - Increased prenatal care quality measured by Rwandan prenatal clinal guidelines - Financial performance incentives can improve quantity and quality of MNCH services and can be in accelerating global development goals |
Rwanda (b) Gertler & Vermeersch (2013) [28] | National supply side PBF program implemented at health facility level | 166 of Rwanda’s 401 primary care facilities, 80 in treatment districts and 86 in comparison districts. | End-line evaluation 2006–2010 | Method: Nested randomized control trial Outcomes: Health worker productivity, child health outcomes | - Substantial improvements in child health outcomes (weight-for-age and height-for-age z-scores) - Provider incentives led to a 20% increase in productivity - Evidence of complementarity between the incentive and the knowledge (skill) of health care providers |
Rwanda (c) Shapira et al. (2017) [29] | Complementary community PBF program that rewarded community health worker cooperatives for the utilization of five targeted maternal and child health ser-vices by their communities | End-line evaluation 2010–2014 | Method: Randomized control trial Outcomes: Nutritional status, use of modern contraceptive methods, ANC and PNC services utilization | -9.6% increased likelihood to attend ANC within 4 months gestational age. -7.2% increased likelihood to attend PNC within 10 days post delivery -Financial rewards to the community health workers did not impact on outcome indicators -No multiplicative effect on outcomes when demand and supply incentives were combined | |
Zambia Friedman et al. (2016) [30] | Performance based contracting of health centres to deliver a specified package of essential MNCH services. | T1: P4P incentives and medical equipment starter packs C1: Input based grants and medical equipment starter packs C2: nothing was provided. | End-line evaluation 2008–2014 | Method: Randomized control trial Outcomes: Vaccination coverage, job satisfaction, status of infrastructure-drugs and medical equipment, health services coverage-ANC | - T1 and C1 increased in institutional delivery and skilled birth attendances compared to C2. However, more marginal increase was between C1 and C2 -ANC visits were 2 weeks earlier in T1 and C1 compared C2 -Immunization coverage remained the same in T1 but significantly declined in C1 and C2 (P4P – protective factor) -In contrast, PNC was better in C1 compared T1 -Significant structural quality increase in T1 -Health workers in T1 significantly spent more time with their patients during consultations -Patients trusted more T1 services compared to C1 and C2 -Job satisfaction and staff retention were increased in T1 and C1 compared to C2; however, job satisfaction was marginally higher in T1 -No impact on staff motivation in both T1 and C1 |
Zimbabwe World Bank (2016) [31] | P4P and PBC contracting started in two districts, and in March 2012 was expanded to 16 additional pilot districts, then to 44 country districts | The sample included 16 RBF districts to 16 counterfactual districts (control districts) | Mid-line Evaluation 2011–2014 | Method: Quasi experimental design, with data analysed in a difference-in-difference framework Outcomes: Skilled birth attendance, MNCH service utilization, family planning, vaccination coverages, nutritional status, client satisfaction, OPP, task shifting | -Improvement in skilled providers, in facility deliveries and caesarean sections outcomes; however, this was also the situation generally across Zimbabwe -Program did not have negative effect on non-incentives services -RBF districts had improved autonomy and decentralized decision making -RBF administrative linked tasks aggravated shortage and high workload situation in HF |