SEARCH

SEARCH BY CITATION

Keywords:

  • Conditional payments;
  • output-based aid;
  • pay-for-performance;
  • performance-based funding;
  • results-based financing

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Background Results-based financing (RBF) refers to the transfer of money or material goods conditional on taking a measurable action or achieving a predetermined performance target. RBF is being promoted for helping to achieve the Millennium Development Goals (MDGs).

Methods We undertook a critical appraisal of selected evaluations of RBF schemes in the health sector in low and middle-income countries (LMIC). In addition, key informants were interviewed to identify literature relevant to the use of RBF in the health sector in LMIC, key examples, evaluations, and other key informants.

Results The use of RBF in LMIC has commonly been a part of a package that may include increased funding, technical support, training, changes in management, and new information systems. It is not possible to disentangle the effects of financial incentives as one element of RBF schemes, and there is very limited evidence of RBF per se having an effect. RBF schemes can have unintended effects.

Conclusion When RBF schemes are used, they should be designed carefully, including the level at which they are targeted, the choice of targets and indicators, the type and magnitude of incentives, the proportion of financing that is paid based on results, and the ancillary components of the scheme. For RBF to be effective, it must be part of an appropriate package of interventions, and technical capacity or support must be available. RBF schemes should be monitored for possible unintended effects and evaluated using rigorous study designs.

inline image

Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Promoting the use of results-based financing (RBF) is one of five actions being taken as part of the Global Campaign for the Health Millennium Development Goals (1, 2). This is based on an assumption that “the evidence suggests that small financial incentives targeted at the right level … are enough to change behavior significantly and achieve results” (3).

RBF can be defined as the transfer of money or material goods conditional on taking a measurable action or achieving a predetermined performance target (4). While this is a simple concept, it includes a wide range of interventions that vary with respect to the:

  • • 
    Level at which the incentives are targeted– recipients of healthcare, individual providers of healthcare, healthcare facilities, private sector organizations, public sector organizations, sub-national governments (municipalities or provinces), national governments, or multiple levels.
  • • 
    Targeted results– health outcomes, delivery of effective interventions (e.g. immunization), utilization of services (e.g. prenatal visits or birth at an accredited facility), quality of care, provision of facilities, human resources or supplies, or development goals (e.g. building institutional capacity).
  • • 
    Indicators used to measure results– what is measured, how it is measured, and who measures it, including the use of independent assessments and monitoring.
  • • 
    Choice of targets– who sets the targets (the provider of the incentives, the recipient of the incentives, or both) and the type of target (pay per result (e.g. per immunization) or pay only if a target is achieved (e.g. 90% coverage).
  • • 
    Type and magnitude of the incentive– the amount of cash, vouchers, or material goods provided for achieving results and the frequency of transfers.
  • • 
    Proportion of financing that is paid for based on results, and how the rest of the financing is allocated, including the proportion of the payer's financing based on results, the proportion of the total financing based on results, and how flexible the financing is.
  • • 
    Ancillary components of RBF schemes, such as increasing the availability of resources, education, supplies, technical support or training; monitoring and feedback, other quality improvement strategies, increasing salaries, construction of new facilities, improvements in planning and management or information systems; changes in governance (e.g. decentralization), priority setting and rationing (e.g. establishment of essential drug lists or services covered by insurance), and involving stakeholders.

We have critically reviewed selected evaluations of RBF schemes in LMIC with the aim of informing decisions about when to use RBF and how to design, implement and evaluate RBF schemes in LMIC. In a companion paper we have summarized the findings of systematic reviews of RBF in both high and low income countries (5).

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

We undertook a critical appraisal of selected evaluations of RBF schemes in the health sector in LMIC. In addition, we interviewed key informants (see acknowledgements) by email or telephone to identify key literature relevant to the use of RBF in the health sector in LMIC, evaluations and other key informants.

We included evaluations of RBF schemes in the health sector in LMIC that are used as examples by the Global Campaign for the Health Millennium Development Goals (1), the World Bank (6) in the discussion paper of the Working Group on Performance-Based Incentives (4), or key informants. The evaluations that were identified by our three main sources and the reasons for excluding evaluations are summarized in Table 1. RBF schemes that were included in a recent systematic review (e.g. conditional cash transfers and contracting) (7, 8) or for which we could not find an evaluation were excluded. For each included example we extracted and summarized the key characteristics of the RBF scheme, the evaluation, and the main findings.

Table 1.  LMIC RBF evaluations considered for inclusion
CountryRBF schemeSourceReason for exclusion
Targeted at recipients of care
Mexico, NicaraguaCCTWorld Bank (6)Included in systematic review of CCT
 P4P Working Group (4) 
YemenVouchers for maternal careWorld Bank (6)Evaluation not found
Targeted at recipients and providers of care
IndiaIncentives for mothers and health workers to give birth in a health facilityGlobal Campaign (3)INCLUDED
 World Bank (6) 
Targeted at providers
Democratic Republic of CongoIncentives for supervisors, inspectors and service providers to deliver public health and healthcare servicesP4P Working Group (4)Results not available
RwandaIncentives for primary care centres to increase the delivery of health servicesWorld Bank (6)Evaluation not found (Some results reported by both the World Bank and P4P Working Group; randomized trial ongoing)
 P4P Working Group (4) 
Targeted at NGO
AfghanistanContracting NGO to deliver basic health servicesWorld Bank (6)Systematic review of contracting included; evaluation not found
CambodiaContracting NGO to deliver healthcare servicesWorld Bank (6)Included in systematic review of contracting
 P4P Working Group (4) 
GuatemalaContracting NGO to deliver primary healthcare servicesP4P Working Group (4)Systematic review of contracting included; results have not been reported
HaitiContracting NGO to deliver healthcare servicesP4P Working Group (4)INCLUDED
Targeted at sub-national governments
ArgentinaIncentives for provincial governments’ health insurance programs to improve maternal and child health outcomesWorld Bank (6)Evaluation not found
RwandaIncentives for municipalities to increase the use of bed nets for children (and other activities)Global Campaign (3)Evaluation not found (No reference provided by Global Campaign. Reference to “June 2007” evaluation by World Bank)
 World Bank (6) 
Targeted at national governments
GAVIIncentives for national governments to increase immunization coverageGlobal Campaign (3)INCLUDED
Targeted at diverse levels
DiverseIncentives for tuberculosis detection and treatmentP4P Working Group (4)INCLUDED

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Of the 13 examples of RBF schemes identified (Table 1), four evaluations met our inclusion criteria (Table 2) (9–12). The four selected evaluations include RBF at different levels, and include two single country cases and two multi-country studies. They are described in Tables 3 and 4.

Table 2.  Evaluations of results-based financing in LMIC
First authorYearCountryProvider of incentivesRecipient of incentivesTargeted results
CORT (9)2007IndiaGovernment of IndiaMothers & community health workersInstitutional delivery
Beith (12)200716 countriesDiverseDiverseTuberculosis detection and treatment success
Eichler (10)2007HaitiUSAIDNGOImmunization coverage, attended deliveries, pre and post-natal care
Chee (11)200752 countriesGAVINational governmentsImmunization coverage
Table 3.  Descriptions of results-based financing (RBF) schemes in LMIC
 ASHA/JSY Scheme in India: RBF for mothers and community health workers (9)Haiti: RBF for NGO (10)GAVI: RBF for national governments (11)
Study designMixed quantitative (survey) and qualitative (interviews) methods.Case study, including analysis of indicator data over 5 years for NGO reimbursed for expenditures and ones that went over to the RBF scheme, and interviews.Quantitative (regression models for 52 countries that received ISS funds from 1995 to 2005) and in depth qualitative studies in 6 countries (3 matched pairs of countries with similar circumstances and starting baseline coverage, and different results).
FundingJSY is 100% centrally sponsored by the Government of India.USAIDGAVI
Flow of fundsThe flow of money appears to be from the Government of India to the State Health Society to the District Health Society to a United Fund to an auxiliary nurse midwife, medical officer, or staff at an instate accredited for deliveries to the ASHA. Cash assistance for delivering to mothers flowed through a Medical Relief Society or the United Fund, to auxiliary nurse midwives, community health centres, primary health care doctors, health centre staff, or others to mothers. Only 1% flowed through ASHA.USAID to Management Sciences for Health (a US based NGO) to NGO in HaitiFrom GAVI to countries. Governments can spend ISS funds in any manner they deem appropriate.
Service providersASHA, auxiliary nurse midwives, staff at accredited institutionsFrom 3 NGO in 2000 to 21 in 2005Varies
Service recipientsMothers giving birth at an accredited institutionFrom 534,000 in 2000 to 2.7 million in 2007Children
LevelIndividual providers (ASHA) and recipients of health care (mothers)NGOCountries
Targeted results and indicatorsBirth at an accredited institutionImproved performance of NGO. Initially seven performance indicators were used (below), and a private survey research firm was hired to measure these. Subsequent changes described in text.Immunization (DTP) coverage. The number of children receiving DTP serves as the primary performance indicator for routine immunization. The system for reporting the number of children immunized with DTP is validated through a one-time Data Quality Audit (DQA) conducted by GAVI-retained external auditors. Reward funding is contingent upon both increasing the number of children immunized with DTP3 and on achieving a verification factor of 80 percent on the DQA. If a country did not achieve the 80 percent verification factor on its DQA, it may work to improve data quality and receive reward funding if it passed a subsequent DQA.
Choice of targetsPayment is per institutional delivery. It is not clear to what extent the target for institutional deliveries was decided by the Government of India or Rajasthan or what the basis was for setting the target. There does not appear to have been any involvement of the recipients of care or ASHA.Targets and budgets are negotiated with the NGO as part of their contract negotiation, based on the previous years’ budget and performance and taking into account other factors such as migration.ISS “investment” funding was paid in instalments over three years based on each country's self-projected number of children to be immunized with DTP3 in the first year after application. Thereafter, additional ISS “reward” funding was paid for immunising additional children above the projected first year targets. The reward funding is calculated at $20 per additional child receiving DTP3 above the number of children targeted the first year after application.
Type and magnitude of incentive200 to 1400 rupees ($4.94 to $34.58) for institutional delivery for mothers and ASHA, adjusted for rural and urban areas.RBF up to 10% of the previous expenditure-based budgetISS funding is a performance-based strategy that makes continued funding conditional upon improved performance and high quality coverage data to encourage countries to make the necessary allocations and immunization investments to vaccinate more children. Funding in later years is based on increases in the number of immunized children.
Proportion of financingASHA's are volunteers. 100% of what they earn through the JSY program is through RBF. It is not clear what proportion of the financing of the ASHA program or the JSY program is RBF.A fixed price contract was equivalent to 95% of the expenditure-based budget.As of June 2006, 145 million USD of ISS funds have been disbursed to 53 countries. It is estimated that GAVI funds increased immunization program funding 15% from pre-GAVI levels.
Ancillary componentsIncreased funding; detailed guidelines; cascade training of ASHA; supervision and regular meetings of the ASHA with auxiliary nurse midwives, provision for transport including referral and escort to an accredited institution for deliveries; investment in improving public health institutions and services; flexibility for state governments to use public-private partnership mechanisms and accredit private health institutions for providing institutional delivery services.Increased funding, aggressive technical assistance, data validation, participation in a network of NGO with shared learning activities.To receive GAVI support, eligible countries were required to submit an application to GAVI and demonstrate three conditions: 1) an inter-agency coordinating committee (ICC) of partners in immunization, operating at the national level; 2) a review of its immunization program conducted within three years of the application year; and, 3) a multi-year plan for its immunization program.
ComparisonTwo before-after comparisons are made in this report regarding the potential impact of the JSY program on institutional deliveries. The first is the proportion of institutional deliveries among a sample of 166 JSY beneficiaries compared to their last previous birth prior to the establishment of the program. The second compared the change in the number of deliveries in public sector institutions before and after establishment of the program to the change one year prior to the establishment of the program.NGO reimbursed for expenditures, most of which subsequently switched to the RBF scheme.Immunization coverage rates with ISS funding were compared with immunization rates in the same countries prior to ISS funding. Several potential modifying factors were considered, including macroeconomic and political factors, health funding and other health priorities, immunization program activities, ISS management and planning, and ISS expenditures by category.
OutcomesIn addition to institutional deliveries, the report considers a number of other outcomes including the satisfaction of beneficiaries and ASHA. The delivery of other services, quality of care, health outcomes, and costs and cost-effectiveness were not addressed in the evaluation.The indicators above are the primary outcomes analysed in the evaluation.The primary outcome that was considered was improvements in immunization coverage. Other outcomes that were considered include impact on overall immunization financing, the cost per additional child vaccinated, and equity.
Table 4.  Results-based financing (RBF) schemes targeted at patients to improve tuberculosis (TB) control in LMIC (from Beith 2007) (12)
RBF targeted at patients
Country/Organization implementing the incentiveIncentive type and population coveredFinancing mechanism and management responsibilityResults*
  1. *DOTS = Directly Observed Treatment Strategy. Most results reported here are from DOTS monitoring data as reported by survey respondents. In many cases other interventions are going on simultaneously; therefore the individual impact of the incentive has not been isolated.

Russian Federation/Orel Oblast GovernmentFood parcel, hot meal, hygienic kits, and bus tickets are part of a package of interventions for all TB patients in the oblast who adhere to treatment norms (N =∼1200 since initiation)WHO/Russia and USAID financed the scheme at initiation; now local government has complete funding responsibility. The Russian Red Cross managed the scheme at initiation but now this is also the responsibility of the local administrationDefault rates dropped from 15–20% to 2–6%
  Default rates dropped from 15–20% to 2–6%
Russian Federation/Vladimir Oblast GovernmentFood parcel (for outpatients only), travel expenses, clothing and hygienic articles (for all patients) are provided to patients who do not interrupt treatment (N =∼3200 since initiation)The scheme was initially financed by WHO and local administration with management by the local Department for Social Affairs and TB service; since 2005, management and financing has been fully transferred to the local oblast administrationLittle evidence of perverse effects was reported. This may be due to strict monitoring and reporting. In rare cases, patients tried to sell the food parcel in order to buy alcohol.
Tajikistan/Project HOPEFood support if provided to DOTS* patients who adhere to treatment and their families who are determined to be vulnerable using standard WFP criteria (N =∼6700 total since initiation)Food is provided by the WFP while funding comes from USAID and Project HOPE. PH and the WFP manage the schemeCure rates were higher for the vulnerable group that received food support: 89.5% vs. 59.4%
In practice, very few TB patients qualified as “not vulnerable”. However, the program felt that many patients who were classified as “not vulnerable” based on WFP criteria were vulnerable, and the decision was made expand the program to cover almost all TB/DOTS patients. Treatment failure was 3.9% in the food support group vs. 15.6% in the comparison cohort
  2.9 percent of patients in the food support group died, vs. 12.5% in the comparison group
  Default rates were lower for the food support cohort: 3.7% vs. 9.4%
  Given small numbers a larger-scale study is necessary to confirm these findings
Kazakhstan/American Red Cross (ARC)Monetary payment versus hot meals versus nurse home visit to TB patients in 20 DOTS corners in one oblast. Patient must complete treatment; if s/he defaults, s/he is responsible for refunding benefits for all drugs taken (N = 449)USAID and the ARC fund the scheme while management is the responsibility of ARC, Oblast national TB program and DOTS corner staffNo intervention was significantly more effective, though the combined contribution of the three interventions improved treatment success 4.7%.

Asha/JSY Scheme in India: RBF for mothers and community health workers

Janani Suraksha Yojana (JSY) is a safe motherhood intervention for reducing maternal and neo-natal mortality launched by the Indian Prime Minister in April 2005, as an integral component of the National Rural Health Mission. The scheme aims to promote institutional deliveries amongst poor pregnant women. Accredited Social Health Activists (ASHA) are female honorary volunteers. One volunteer for every village with a population of 1000 is proposed to act as an interface between the community and the public health system. ASHA receive performance-based compensation for promoting a variety of primary healthcare services in general and reproductive and child health services in particular such as universal immunization, referral and escort services for institutional deliveries, construction of household toilets, and other healthcare delivery interventions. The government scheme (JSY) includes a package of interventions of which RBF is a small component.

The program was evaluated using a mix of quantitative (survey) and qualitative (interviews) methods. The proportion of institutional deliveries increased from 32.5% to 65.1% among the 166 interviewed JSY-beneficiaries, and the number of institutional deliveries in the public sector in Rajasthan state increased by 36% the year after the JSY was established compared to a slight decrease (−0.25%) the previous year (9).

Among the 173 interviewed ASHA, only 72 received some cash remuneration while the majority were yet to receive any despite having assisted in promoting institutional deliveries. The ASHA who had received money earned an average of about 400 rupees ($9.88) for cases motivated over three months while the projected estimate of the maximum an ASHA could earn was three times that. Forty-three percent of ASHA were satisfied and 36% were somewhat satisfied with the remuneration received mainly because ‘they could earn extra money’ (39%) or because being an ASHA gave an opportunity to learn many new things and work within the village. ASHA were unsatisfied with the cash assistance because it involved ‘too much work for too little money’ (21%), the money was not given on time (15%), and the feeling that some ASHA were being favored.

The most commonly reported motivation for institutional delivery reported by beneficiaries (mothers) was money available under JSY (56% of respondents), better access to institutional delivery services in the area (44%), and the support provided by ASHA (22%). ASHA played a limited role in facilitating and arranging transport. On average, the beneficiaries spent 280 rupees ($6.92) on transport to reach the place of delivery. Nine out of 10 beneficiaries paid money for the transport expenses on their own, and an insignificant proportion were reimbursed later. Eighty-eight percent of the beneficiaries interviewed received JSY cash assistance for delivery in an institution and 76% of those who delivered at home. The majority of women (64%) had to pay for services at the institution where they delivered; including medicines and IV fluids (94%), delivery, caesarean or operation charges (60%), and food and accommodation charges (11%).

The quantitative and qualitative data collected for this evaluation suggest that the cash assistance for mothers likely played an important role in motivating institutional deliveries, but it is not possible to quantify the effectiveness of RBF targeted at mothers. The evidence for the effects of RBF for ASHA is less compelling and suggests that RBF for ASHA probably played a small, if any, role in motivating institutional deliveries.

The evaluation indicates that the JSY program has had some positive effects in reducing inequities, and RBF targeted at mothers may have contributed to that. The evaluation provides only limited information about limitations and potential unintended consequences of RBF. These include:

  • • 
    problems with auxiliary nurse midwives handling substantial amounts of money for the first time
  • • 
    problems with delays in payment
  • • 
    potential problems with nepotism

Haiti: RBF for non-government organizations

The objective of the evaluation of RBF for non-government organizations (NGO) in Haiti was to assess whether paying for results is effective as well as the many “nuts and bolts” details that can be used to inform others considering implementing performance based incentives (10). The evaluation was based on an analysis of indicator data over 5 years for NGO reimbursed for expenditures and the ones that went over to the RBF scheme, as well as interviews.

Although it is not possible to isolate the effects of RBF, regression analysis suggests that the new payment incentives contributed to improvements in both immunization coverage and attended deliveries. Results for prenatal and postnatal care were less clear. The project went from using an assessment guideline to assess the eligibility of NGO for the RBF scheme, to all NGO being in the RBF scheme. It is not clear how well the assessment tool worked.

Several changes were made to the indicators. Waiting time was dropped because it was a poor indicator of quality, and child visits were dropped because they were difficult to measure. Management indicators were added, motivated by concern that attention to short term improvements resulted in neglect of key management functions. Additional technical output indicators were added. Some indicators were developed jointly with the NGO, and some performance indicators were added that could be evaluated throughout the year with immediate payments.

Performance went from being measured by an independent firm to self-report with random audits. To encourage NGO to focus on all services included in the essential package and reduce costs of verifying performance, the project switched to randomly choosing indicators from an expanded list. Then to encourage NGO to focus efforts on improving the quality of all services in the basic package, one of two packages of indicators was randomly chosen for evaluation. Then the project switched to random audits of two common indicators across NGO and an additional randomly chosen indicator from a list of seven. Feedback from the NGO suggests that the switch from community surveys conducted by an external firm to self-report with random audits not only reduced costs but also encouraged NGO to strengthen their information systems.

The results in this evaluation are difficult to interpret since the comparison groups were not equivalent, and the switch to the RBF scheme is completely confounded with switching from 100% reimbursement-based financing to 95% fixed price financing, as well as the other ancillary components. The increased autonomy, flexibility, and reduced reporting were identified as major motivators. All of these are largely due to the shift from reimbursement-based financing to a fixed price contract, rather than to RBF. It is likely that the combined fixed price contract and RBF financing motivated NGO to utilize technical assistance. On this basis it could be argued that there is a synergistic effect of combined fixed price, RBF financing, and the availability of technical assistance.

GAVI: RBF for national governments

The GAVI Alliance (formerly the Global Alliance for Vaccines and Immunization) provides support to country immunization programs in the form of Immunization Services Support (ISS) funding. Continued ISS funding following an investment period is conditional upon improved performance and high quality coverage data. GAVI commissioned an evaluation of its ISS funding from 2000–2005 that was released in December 2007 (11). The objectives were to assess the experience of the ISS scheme, the implementation of ISS funding at the country level and its relation to overall immunization financing, and to identify the relationship between the allocation of ISS funding and immunization coverage rates (DTP3). The evaluators utilized a regression model for 52 countries that received ISS funds from 1995 to 2005 and in-depth qualitative studies in six countries (3 matched pairs of countries with similar circumstances and starting baseline coverage and different results).

A relationship was found between ISS funding and increased immunization coverage. The imputed cost of immunizing an additional child was approximately $23 at the lowest coverage rates. Once coverage rates were above 60 to 70%, the cost per child immunized increased exponentially. Gross domestic product, political instability, and current conflict were found to reduce the effect of ISS funding. Specific immunization program activities, ISS planning and management, and expenditures in different categories were not found to have an impact on immunization coverage.

It is not clear whether ISS funding displaced other sources of immunization funding. The majority of funds were used for recurrent expenses (83%) and at a sub-national level (77%). ISS was well-integrated within national immunization programs, but harmonization across health programs and administrative levels was much more challenging, reflecting the general level of harmonization within the health system.

Only limited data was available to explore the extent to which RBF motivated changes. The only factor that was significantly related to approval of reward funding was population growth rate, and only that and baseline DTP coverage rate were related to disbursement of reward funds. Similarly, only limited data was available to test whether the size of the reward affected performance. Neither the size of the reward relative to pre-ISS immunization expenditure per child or relative to government health expenditures were found to be related to performance.

No correlation was found between rewards and geographic equity or stability of coverage. Receiving rewards had little effect on performance, although the data for evaluating this was also limited. Low-income countries under stress (LICUS) were less likely to receive rewards. Rewards were based on the number of children initially projected to be immunized the first year potentially allowing countries to manipulate their projections and making achievement of projections difficult for countries with declining birth rates.

LICUS countries, more politically unstable countries, and countries with lower population growth rates were less likely to benefit from ISS funding. Countries with higher baseline DTP coverage rates were also less likely to benefit. ISS rewards do not cover the actual cost of increasing coverage for the hardest to reach children in these countries and, thus, may not motivate efforts to increase coverage among those disadvantaged populations. There was no evidence of negative impacts on measles coverage rates, which was used as an indicator of performance with other vaccines.

It is not possible to isolate the effect of RBF from the effect of increased funding for immunization coverage. In other words, it is not known whether the same amount of funds provided in a different way would have achieved different results in terms of immunization coverage. Qualitative data suggest that the flexibility of the funding (which is not specific to RBF) was valued by recipients, and may have facilitated good use of the funds under some circumstances.

Tuberculosis detection and treatment: RBF for patients and providers

In a review of evaluations of RBF in tuberculosis (TB) control programs (12), the authors identified a variety of incentives that have been used to try to motivate improvements in the detection and treatment of TB. Most of these appear to have been targeted at patients, including:

  • • 
    Direct payment
  • • 
    Deposit return
  • • 
    Food (hot meals, dry rations, vouchers)
  • • 
    Transportation subsidies
  • • 
    Vouchers for material goods
  • • 
    Packages of personal hygiene products

For individual providers incentives have included:

  • • 
    Direct Payment
  • • 
    Food packages
  • • 
    Vouchers
  • • 
    Other material goods
  • • 
    Free drugs to private providers

Direct payments have also been used as incentives for teams, organizations, and local governments. It is difficult to isolate the effects of RBF using routine TB data and there are few rigorous evaluations.

RBF targeted at both patients (Table 4) and providers has been used in several countries to improve TB control (12). In Bangladesh, a community-based approach to DOTS (Directly Observed Treatment Strategy) that included an incentive for community health workers achieved higher detection rates than the rest of the country (90 vs 82%). It is not possible to assess the contribution of RBF to this apparent improvement. In Pune, India, a private provider payment scheme for referral of suspects to microscopy centers and subsequent directly observed treatment (DOT) found improvements in detection and cure rates. These findings were attributed to a variety of factors that include RBF.

The limited evaluations of the use of RBF for TB detection and treatment suggest that RBF may be one element of a broad strategy to achieve TB control goals. This experience highlights the importance of stakeholder involvement and undertaking an appropriate assessment of obstacles to the desired behaviors to inform the design of an RBF scheme. Careful incentive design and monitoring are also needed to minimize unintended effects. Unintended effects of RBF for patients included engaging in practices that enable them to continue to qualify for benefits such as avoiding medicines to continue to receive monthly payments, pressuring providers to transfer them to an area with benefits, selling food to buy alcohol, creating false patients to get food for non-TB patients, demoralizing health workers who feel that it is unfair that they are not provided incentives, and health workers stealing food and money.

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

In all of these evaluations it is difficult, if not impossible, to disentangle the effects of RBF from other components of the intervention packages, including increased funding, technical support, training, and new management structures and monitoring systems. Interviews with mothers in India suggest that payments to mothers may have contributed to an increase in facility-based births, whereas there is little support for the contention that payments to ASHA to encourage facility-based births contributed to the increase. It is not possible to know, based on the available evidence, to what extent paying NGO for performance contributed to improvements in care in Haiti. Similarly, it is not possible to know whether the same amount of funds provided in a different way by GAVI would have achieved different results in terms of immunization coverage, and the same is true for funding by the Global Fund (13). It also is not certain what the impact of the GAVI or Global Fund RBF schemes is on disparities within or across countries, although the Global Fund's scheme does not appear to penalize poorer countries that first receive grants or those with weaker health systems (14). While the use of RBF for TB detection and treatment may be effective as one element of a broad strategy to achieve TB control goals, the available evidence is limited, and RBF can have unintended effects.

Although it is plausible that RBF can help to achieve the MDGs, at present there is at best limited evidence to support this contention. More rigorous evaluations are needed to evaluate the impacts of RBF, and to inform decisions about when to use it and how to design, implement, monitor, and evaluate RBF when it is used. Nonetheless, some key messages have emerged regarding the design of RBF schemes from the evaluations we have reviewed and our overview of systematic reviews reported in a companion article (5). These are summarized here.

How should RBF be designed?

RBF schemes require very careful design with respect to when they are used, the level at which they are targeted, the choice of targets and indicators, the type and magnitude of incentives, the proportion of financing that is paid based on results, and the ancillary components of the scheme. Suggested steps in designing an RBF scheme, adapted from Eichler and NorthStar (4, 15), include:

  • 1
    Identify which stakeholders will be involved.
  • 2
    Specify the health system's problems, underlying causes, and goals.
  • 3
    Decide whether the problem is a priority.
  • 4
    Specify specific performance problems, how RBF will motivate the desired performance, and what other interventions are needed to address the underlying problems.
  • 5
    Design the intervention package, including RBF, and design the RBF, including:
    • a. 
      Who will receive the incentives
    • b. 
      The conditions for receiving the incentives (targets)
    • c. 
      How performance will be measured (indicators)
    • d. 
      The type and magnitude of the incentives, what other funding is needed and how it will be disbursed
    • e. 
      A budget
  • 6
    Assess the feasibility of the scheme, including costs, political consequences, availability of personnel and supplies, adequacy of information to measure performance, readiness and capacity to manage the process, and adequate technical support.
  • 7
    Ensure that there is political and institutional support for the RBF scheme.
  • 8
    Develop detailed service specifications, including the flow of money operational procedures for managing the scheme and contractual arrangements.
  • 9
    Ensure there is capacity for managing the RBF scheme.
  • 10
    Prepare a plan for monitoring the scheme, including possible unintended effects, and evaluating it, using as rigorous a design as possible to address important uncertainties.

Level of the incentives

It is important to ensure that incentives go to those who need motivation to change their behavior. Incentives that go to the wrong people are unlikely to motivate desired changes, unless there are mechanisms in place or reasons to assume that the incentives will have indirect effects; for example if they are passed on by managers of organizations to health professionals or motivate managers to change the behaviors of professionals in other ways.

With government, organization, or group level incentives, individual health workers cannot collect the full returns on their individual efforts to improve quality and may not be motivated by incentives. Thus, the potential for some to “free-ride” on the efforts of others may reduce the efforts of all. Alternatively, the problem with rewarding physicians and not organizations or groups is that needed organizational changes may not be motivated. For example, studies evaluating chronic care suggest that multidisciplinary teams produce better patient outcomes. Provider group level or organization level incentives (if substantial enough) may provide the impetus to create infrastructure changes that are needed (16).

Choice of targets and indicators

The best process-of-care measures are those for which there is good quality evidence that better performance leads to better health outcomes. Also, it is important to note that process of care measures may be more sensitive to quality differences than measures of outcomes, because a poor outcome does not necessarily occur every time there is a quality problem. Therefore, one way to change behavior so that both quality and documentation improve may be to base the incentive on the combination of a process of care measure (for example, documentation of diagnosis and treatment) and the outcome of interest (for example, cure rates). This approach may avoid the pitfalls of process of care measures alone that encourage gaming, as well as the disadvantage of basing incentives solely on outcomes that may be relatively rare or difficult to achieve and somewhat beyond the control of the provider. Thus, a combined approach could capitalize on the advantages and complementary nature of both types of quality of care measures.

Whether the incentive target should be designed as an absolute performance goal (that is, a defined threshold, such as 75% of patients with up-to-date immunization status), a relative performance goal (for example, 30% improvement from baseline), or a payment for each instance of a service regardless of the overall performance, is an important question. A systematic review of pay for performance for improving the quality of care found four studies that used an absolute performance target, two that used relative performance targets, and three that used a combination of relative and absolute performance targets (16). Two studies showed that individuals or groups with the lowest baseline performance improved the most. If threshold performance targets are used, however, they may garner the least performance pay. This suggests the need to consider combined incentives for both overall improvement and achievement of a threshold, if thresholds are used. Policymakers must consider whether their goal is improving performance at the lower end of the spectrum, achieving a target for performance, or both.

Type and magnitude of the incentives

The size of incentives needed in different settings requires careful attention due to two sources of inefficiency. On the one hand, RBF can yield very high costs per marginal change in behavior that is induced if the incentive is given to all targeted individuals regardless of their possible previous compliance with the desired behavior. Consequently, potential benefits of RBF must be weighted against their cost-effectiveness (and any potential undesirable effects), in particular when incentives and initial compliance in the target population are high. On the other hand, the existence of possible threshold effects of incentives levels may lead to inefficiency because the incentive will either be too high (reducing efficiency) or too low to induce the desired behavior. Small incentives may motivate little or no change, and it is not known what size of an incentive is needed to yield large or sustained effects. It is also possible that those most in need of behavioral change and most recalcitrant to change may not be as easily swayed by incentives, and that the size of incentives (or other strategies) needed to reach them may be still higher.

It is possible that the lack of effect or small effects in some studies of RBF, particularly for providers, was due the small size of the bonus. One qualitative study in the USA suggested that a bonus of at least 5% of a physician's income may influence behavior (16). Similarly, when providers are paid by multiple sources, the incentive may affect too few patients, effectively diluting the size of the incentive.

“End-of-year” compensation may not influence provider behavior as much as a concurrent fee or intermittent bonuses. This is because lack of awareness of the incentive and infrequent performance feedback may be barriers to incentive effectiveness.

Undesirable effects and development goals

When designing RBF schemes, it is important to anticipate and avoid incentives that may foster undesirable behaviors (5). In addition, it is important to take into consideration medium to long-term development goals, such as building institutional capacity, as well as immediate targets and indicators.

How should RBF be monitored and evaluated?

It can be argued that RBF at the government or organizational level contributes to transparency and accountability (14). However, given the lack of good quality evidence about the effects of financial incentives at all levels, and the risk of unintended effects, ongoing monitoring of RBF schemes is critical to determine whether incentives are working, and whether they are having unintended effects. To discern the effects of financial incentives from the package of interventions of which they normally are one part, rigorous evaluations are needed, such as ongoing randomized trials of conditional versus unconditional funding mechanisms in Indonesia and the Philippines (17, 18). When possible, randomized trials are ideal because they can control for the many possible confounders and they may give answers more quickly as well as more reliably (18–20). In addition, both quantitative and qualitative process evaluations are needed, given the complexity of most interventions, behaviors, and systems (21).

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

The report on which this article is based was commissioned by the Norwegian Agency for Development Cooperation (Norad). ADO and AF are employed by the Norwegian Knowledge Centre for the Health Services. The views expressed herein are those of the author and do not necessarily represent the views of the project's funder or the Norwegian Knowledge Centre for the Health Services.

The authors thank the following individuals who provided background information for this report or commented on an earlier version: Jennie Barugh, DFID; Amie Batson, World Bank; Sara Bennett, Alliance for Health Policy and Systems Research; Abdallah Bchir, GAVI Alliance; Logan Brenzel, World Bank; Rena Eichler, CGD Working Group on Performance-Based Incentives; Tessa Tan-Torres Edejer, WHO; Timothy Evans, WHO; Matt Gordon, DFID; Daniel Low-Beer, Global Fund; Ingvar Olsen, Norad; Jean Perrot, WHO; Don de Savigny, Swiss Tropical Institute; Susan Stout, World Bank.

Conflict of interests: None.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  • 1
    The Global Campaign for the Health MDGs. Challenges, opportunities, and the imperative of shared learning. Lancet 2007; 370: 101820.
  • 2
    Stoltenberg J. Delivering for women and children. Lancet 2008; 371: 12302.
  • 3
  • 4
    Eichler R. Can “Pay for Performance” Increase Utilization by the Poor and Improve the Quality of Health Services? Discussion paper for the first meeting of the Working Group on Performance-Based Incentives. Washington , DC : Center for Global Development, 2006: 5.
  • 5
    Oxman AD, Fretheim A. Can paying for results help to achieve the MDGs? 1. An overview of the effectiveness of results-based financing. Journal of Evidence-Based Medicine 2009; 2: 7083.
  • 6
    World Bank. Health Results Innovation Grant Proposal to the Government of Norway. Washington , DC : World Bank, October 24, 2007.
  • 7
    Lagarde M, Haines A, Palmer N. Conditional cash transfers for improving uptake of health interventions in low- and middle-income countries: a systematic review. JAMA 2007; 298: 190010.
  • 8
    Lagarde M, Palmer N. The impact of contracting out on access to health services in low and middle income countries. Cochrane Database Syst Rev. In press.
  • 9
    CORT, Vadodara. Report on assessment of ASHA/JSY scheme (Rajasthan). Commissioned by Ministry of Health & Family Welfare, Government of India. Draft, June 2007.
  • 10
    Eichler R, Auxila P, Antoine U, Desmangles B. Performance-based incentives for health: six years of results from supply. side programs in Haiti. CGD Working Paper #121. Washington , DC : Center for Global Development, 2007.
  • 11
    Chee G, His N, Carlson K, Chankova S, Taylor P. Evaluation of the first five years of GAVI immunization services support funding. Prepared for the GAVI Alliance. Behtesda , MD : Abt Associates Inc, 2007.
  • 12
    Beith A, Eichler R, Weil D. Performance-based incentives for health: A way to improve tuberculosis detection and treatment completion? CGD Working Paper #122. Washington , DC : Center for Global Development, 2007.
  • 13
    Low-Beer D, Afkhami H, Komatsu R, Banati P, Sempala M, Katz I, et al . Making performance-based funding work for health. PLoS Med 2007; 1 4(8): e219.
  • 14
    Low-Beer D, Afkhami H, Komatsu R, Banati P, Sempala M, Katz I, et al . Making performance based funding work: progress and challenges of the Global Fund. PLoS Med 4(8): e219. Supporting Information. doi:DOI: 10.1371/journal.pmed.0040219.sd001.
  • 15
    NorthStar – how to design and evaluate healthcare quality improvement interventions. The ReBEQI Collaboration 2005: http://www.rebeqi.org/northstar.
  • 16
    Petersen LA, Woodard LD, Urech T, Daw C, Sookanan S. Does pay-for-performance improve the quality of health care? Ann Intern Med 2006; 145: 26572.
  • 17
    Oiken B, Onishi J, Pradhan M. Conditional community cash transfers. Poverty Action Lab. http://www.povertyactionlab.com/projects/project.php?pid=49 (accessed 3 May 2008).
  • 18
    Shimkhada R, Peabody JW, Quimbo SA, Solon O. The Quality Improvement Demonstration Study: An example of evidencebased policy-making in practice. Health Res Policy Syst 2008; 6: 5.
  • 19
    Savedoff WD, Levine R, Birdsall N. When will we ever learn? Improving lives through impact evaluation. Report of the Evaluation Gap Working Group. Washington , DC : Center for Global Development, 2006. http://www.cgdev.org/content/publications/detail/7973/
  • 20
    Schneeweiss S, Maclure M, Carleton B, et al . Clinical and economic consequences of a reimbursement restriction of nebulised respiratory therapy in adults: direct comparison of randomized and observational evaluations. BMJ 2004; 328: 5607.
  • 21
    Oakley A, Strange V, Bonell C, Allen E, Stephenson J, RIPPLE Study Team. Process evaluation in randomized controlled trials of complex interventions. BMJ 2006; 332: 4136.