Setting priorities for health interventions in developing countries: a review of empirical studies
Summary
enObjective To assess and summarize empirical studies on priority-setting in developing countries.
Methods Literature review of empirical studies on priority-setting of health interventions in developing countries in Medline and EMBASE (Ovid) databases.
Results Eighteen studies were identified and classified according to their characteristics and methodological approaches. All studies were published after 1999, mostly between 2006 and 2008. Study objectives and methodologies varied considerably. Most studies identified sets of relevant criteria for priority-setting (17/18) and involved different stakeholders as respondents (11/18). Studies used qualitative (8/15) or quantitative (3/15) techniques, or combinations of these (4/15) to elicit preferences from respondents. In a few studies, respondents deliberated on results (3/18). A minority of studies (7/18) resulted in a rank ordering of interventions.
Conclusions This review has revealed an increase in the number of empirical studies on priority-setting in developing countries in the past decade. Methods for explicit priority-setting are developing, being reported and are verifiable and replicable and can potentially lead to solutions for ad hoc policy-making in health care in many developing countries.
Abstract
frDéfinir les priorités pour les interventions en santé dans les pays en développement: une revue des études empiriques
Objectif: Analyser et résumer les études empiriques sur la définition des priorités dans les pays en développement.
Méthodes: Revue de la littérature des études empiriques sur la définition des priorités d’interventions en santé dans les pays en développement dans les bases de données Medline et EMBASE (Ovid).
Résultats: 18 études ont été identifiées et classées en fonction de leurs caractéristiques et leurs approches méthodologiques. Toutes ces études ont été publiées après 1999, la plupart entre 2006 et 2008. Les objectifs des études et les méthodes variaient considérablement. La plupart des études ont identifié des critères pertinents pour la définition des priorités (17/18) et ont impliqué les différents acteurs concernés comme répondants (11/18). Les études ont utilisé des techniques qualitatives (8/15) ou quantitatives (3/15) ou la combinaison des deux (4/15) pour déceler les préférences des répondants. Dans peu quelques, les répondants ont délibéré sur les résultats (3/18). Une minorité d’études (7/18) ont résulté en un ordre de priorité des interventions.
Conclusions: Cette revue a révélé une augmentation du nombre d’études empiriques sur la définition des priorités dans les pays en développement au cours de la dernière décennie. Des méthodes pour l’établissement de priorités explicites sont en développement, sont rapportées et sont vérifiables, sont reproductibles et peuvent potentiellement conduire à des solutions ad-hoc pour l’élaboration de politiques dans les soins de santé dans de nombreux pays en développement.
Abstract
esEstableciendo prioridades para intervenciones de salud en países en vías de desarrollo: una revisión de estudios empíricos.
Objetivo: Evaluar y resumir estudios empíricos sobre el establecimiento de prioridades en países en vías de desarrollo.
Métodos: Revisión de la literatura de estudios empíricos sobre el establecimiento de prioridades en intervenciones sanitarias en países en vías de desarrollo, utilizando las bases de datos de Medline y EMBASE(Ovid).
Resultados: Se identificaron 18 estudios y se clasificaron de acuerdo a sus características y enfoques metodológicos. Todos los estudios fueron publicados después de 1999, la mayoría entre el 2006 y el 2008. Los objetivos y las metodologías de los estudios variaban considerablemente. La mayoría de los estudios identificaban un conjunto de criterios relevantes para el establecimiento de prioridades (17/18), e involucraban diferentes stakeholders como entrevistados (11/18). Los estudios utilizaron técnicas cualitativas (8/15) o cuantitativas (3/15), o una combinación de ambas (4/15) para averiguar las preferencias de los entrevistados. En unos pocos estudios, los entrevistados discutían sobre los resultados (3/18). Una minoría de los estudios (7/18) dieron como resultado un rango ordenado de intervenciones.
Conclusiones: Esta revisión ha revelado un aumento en el número de estudios empíricos sobre el establecimiento de prioridades en países en vías de desarrollo durante las últimas décadas. Nuevos métodos para el establecimiento explícito de prioridades están siendo desarrollados, reportados, y son verificables y replicables, y potencialmente podrían llevar a soluciones para la realización ad-hoc de políticas sanitarias en muchos países en vías de desarrollo.
Introduction
Priority-setting of health interventions is one of the most challenging and difficult issues faced by health policy decision-makers around the world. It is a process that is inevitably value-laden and political (Ham 1997; Klein 1998; Buse 1999; Bryant 2000; Goddard et al. 2006), requiring credible evidence, strong and legitimate institutions and fair processes (Daniels & Sabin 1997; Klein & Williams 2000; Cappenlen & Norheim 2006; Norheim 2008).
Priority-setting is especially important in developing countries, where resources are limited and government expenditures on health are less than US$20 per capita per year (World Health Organization 2008). As Kapiriri and Martin (2007) argue, this is further complicated by: (i) the burden of underdevelopment in these countries which increases the gap between the health needs and resources available to respond to them; (ii) the many uncertainties in priority-setting because of lack of dependable information; (iii) the multiple players with various agendas; (iv) few systematic processes for decision-making; and (v) many obstacles to implementation such as political instability, inadequately developed social sectors, weak institutions and marked social inequalities, which make the implementation of systematic priority-setting processes difficult (Bryant 2000). As a result, priority-setting in developing countries is often ad hoc or history-based (Baltussen & Niessen 2006; Kapiriri & Martin 2007).
There have been a number of international efforts to promote rational priority-setting by addressing the information gaps, such as studies on burden of disease (BoD) (Lopez et al. 2006) and cost-effectiveness analysis (CEA) (Jamison et al. 1993;Evans et al. 2005). Many such studies have also been carried out at the national level, e.g. in Mexico, India, and a set of east and northern African countries (Baltussen et al. 2005). Although these initiatives may have improved the evidence-base for priority-setting, it was also observed that the resulting information is only input to complex process of priority-setting and that ‘simple technical solutions’ are insufficient (Naylor 1995; Holm 2000; Haudemaekers & Dekkers 2003; Benatar 2003; Evans et al. 2005; Teerawattananon et al. 2007).
In response to this, a growing number of empirical studies have explored more comprehensive approaches to priority-setting in developing countries in the past decade. For example, researchers have tested different strategies to involve all relevant stakeholders in the priority-setting process (Makundi et al. 2007), or to identify the relative importance of CEA and severity of disease as criteria for priority-setting (Kapiriri et al. 2004). While these studies provide valuable information with potential benefit to policy-makers and researchers, a review is lacking and the options and limitations of the various approaches are difficult to assess.
We reviewed empirical studies on priority-setting of health interventions in developing countries, classified their methodological approaches and defined methodological suggestions for future studies. Thus we aimed at stimulating discussion on the options and limitations of the various approaches. This paper defines priority-setting as the process of rank ordering interventions with the aim of informing decision-makers on the implementation of these interventions.
Methods
We carried out a literature search in November 2008 using the Medline and EMBASE (Ovid) databases. In a first step, we performed a search using the following keywords: ‘health’ and ‘priority-setting’ or ‘prioritization’ or ‘resource allocation’, in combination with the names of developing countries according to World Bank (2008) definitions. We limited the search to studies published in English available from January 1997 to October 2008. Next we (i) included studies in the review if they reported empirical data to guide future priority-setting of interventions in health care and (ii) excluded studies from the review if they reported on a single criterion for priority-setting only (this excludes CEA or BoD studies). In this step, we initially screened study abstracts on these criteria and subsequently obtained full-text formats for studies that seemed relevant. The final inclusion of studies in the review was based upon a detailed assessment of the full-text formats (studies for which no full-text format was available were excluded). All abstracts and full-text formats were reviewed independently by both authors. In case we disagreed, a discussion was arranged to reach a consensus. As step 3, a snowballing technique was used to identify related articles in this context and these were also assessed using the same inclusion and exclusion criteria.
We then classified identified studies according to four general characteristics: (i) country(ies) where the studies were conducted; (ii) area of prioritization; (iii) decision-making level(s); and (iv) study objectives. In addition, we classified identified studies according to their methodological approaches to the priority-setting process. Here, we distinguished: (i) which type of respondents (or stakeholders) was involved; (ii) how criteria were identified; (iii) which criteria were identified; (iv) how preferences for the criteria were elicited; and (v) how results were presented.
Results
The first step in the literature search resulted in a total of 1291 studies. In the second step, these studies were initially screened on their abstract and 1235 studies were excluded. The remaining 56 studies were assessed on the basis of the full-text formats and of these, 39 articles were excluded. Studies were excluded because, e.g. they merely assessed the participation of stakeholders in the priority-setting process in the past (e.g. Mubyazi et al. 2007); assessed the relevance of a single criterion in priority-setting (e.g. Kapiriri et al. 2003); or assessed the priorities in targeting of diseases (thus not interventions) (e.g. Makundi et al. 2005; Rosato et al. 2006). A total of 18 studies were finally selected (Table 1).
| Study | Characteristics | |||
|---|---|---|---|---|
| Country | Area of prioritization | Decision-making level | Objectives | |
| Teerawattananon and Russell (2008) | Thailand | Several disease area (two hypothetical case scenarios) | National | To explore policy actors’ justifications for their decisions on the two case scenarios |
| Vargas and Poblete (2008) | Chile | Health system | National | To examine the introduction of a prioritized list of 56 health conditions in Chile by using multiple criteria |
| Lasry et al. (2008) | South Africa | HIV/AIDS | Organization (primary healthcare clinic) | To apply the system for HIV/AIDS resource allocation to a primary healthcare clinic |
| Ottersen et al. (2008) | Tanzania | Health system | National | To explore distribution preferences among health planners |
| Mshana et al. (2007) | Tanzania | Health system | District | To describe an initiative in Tanzania to improve priority-setting using ‘accountability for reasonableness’ (A4R) |
| Makundi et al. (2007) | Tanzania | Several disease area | National | To test out a model for priority-setting which incorporates both scientific evidence and public values |
| Rubinstein et al. (2007) | Argentina | Health system | National | To determine whether economic evaluations are considered and used by decision-makers and report the criteria decision-makers used for resource allocation |
| Baltussen et al. (2007) | Nepal | Several disease area | National | To identify the various criteria for priority-setting, and rank-ordering health interventions |
| Husain et al. (2007) | Pakistan | HIV/AIDS | National | To identify perceptions of decision-makers about the process of resource allocation within the National AIDS Control Programme |
| Madi et al. (2007) | Burkina Faso | Safe motherhood programme | National | To describe a process to elicit and prioritize evaluation needs for safe motherhood programme |
| Ghana | ||||
| Indonesia | ||||
| Kapiriri et al. (2007) | Uganda (Canada and Norway) | Health system | Organization (publicly funded hospital) | To describe the process of healthcare priority-setting and evaluate the description using the framework of A4R |
| Kapiriri and Martin (2006) | Uganda | Health system | Organization (1500 bed-public hospital) | To describe priority-setting process in a hospital and evaluate the description using A4R |
| Baltussen et al. (2006) | Ghana | Several disease area | National | To identify the various criteria for priority-setting, and rank ordering health interventions |
| Kapiriri and Norheim (2004) | Uganda | Health system | National | To explore the acceptance of priority-setting criteria for healthcare system |
| Kapiriri et al. (2004) | Uganda | Health system | National | To establish the relative preferences regarding cost-effectiveness regarding cost-effectiveness of interventions and severity of disease as main criteria for setting priorities |
| Reichenbach (2002) | Ghana | Reproductive health (breast and cervical cancer) | National | To examine the influence of political and organizational factors on national priority-setting |
| Hrabačet al. (2000) | Bosnia and Herzegovina | Health system | National | To provide an overview of the methodology for designing a basic package of health entitlements |
| Söderlund (1999) | South Africa | Health system | National | To define package of essential hospital care |
All studies were published after 1999; and 13 in the period 2006–2008. Four studies were conducted in Uganda, three in Tanzania, two each in South Africa and Ghana and one each in Bosnia-Herzegovina, Ghana, Pakistan, Nepal, Argentina, Chile and Thailand. One study researched priority-setting in three developing countries (Burkina Faso, Ghana and Indonesia). The studies covered a wide range of priority-setting areas: 10 studies prioritized interventions across the healthcare system, four studies across several disease areas and four studies concentrated on particular disease areas. Most of the identified studies (14/18) focused on priority-setting at the national level. One study in Tanzania evaluated the priority-setting process at the district and community levels and three considered priority-setting process at the institutional level, i.e. a hospital. In terms of study objectives, 13 studies primarily aimed at identifying criteria for setting priorities in health care. Three studies explored the acceptability of using economic evaluation or burden of disease information in decision-making health priorities. One study examined the introduction of ‘Accountability for reasonableness’ to improve the priority-setting process and one study described the priority-setting process as experienced by stakeholders.
Table 2 describes the methodological approaches of the reviewed studies. In terms of respondents, 11 studies included more than one type of stakeholder (with policy-makers being most often included). Among these, Makundi et al. (2007) involved four types of respondents – policy-makers, health workers, general population and people living with HIV/AIDS. Kapiriri et al. (2004) included the largest number of respondents (413 respondents in Uganda). In terms of approaches to identify criteria, 10 studies organized group discussions or held interviews. Eight studies identified criteria from a literature review. In terms of identified criteria, cost-effectiveness was the most common important criterion considered (in 12 of the 17 studies that identified criteria), followed by severity of disease (6/17). Other criteria included burden of disease, age of target group, poverty reduction, effectiveness/benefit of treatment and health effects.
| Study | Study methodology | |||||
|---|---|---|---|---|---|---|
| Respondents/ participants (number) | How to identify criteria | Identified criteria | Preference eliciting techniques | Presenting results | ||
| Teerawattananon and Russell (2008) | Policy actors (policy-makers, hospital director, health worker and academics) | (38) | Literature review | Cost-effectiveness, severity of disease and treatment alternatives, equity of access improvement and financial impact on government budget | Semi-structured interview | Distribution of choices between the two cases scenarios by type of respondents |
| Vargas and Poblete (2008) | None | Literature review | Burden of disease, inequity, effectiveness, delivery capacity of systems, costs, people’s preference and cost-effectiveness | NA (secondary data analysis) | Ranking of 56 priority diseases and treatments | |
| Lasry et al. (2008) | Policy-maker (national, provincial, local) NGOs and academia | (35) | Literature review | (1) Prescriptive and current priority; (2) equity; (3) optimization, e.g. cost-effectiveness, total budget constraint, budget levels | Group discussion and interview | Ranking of HIV/AIDS interventions based on each of three approaches |
| Ottersen et al. (2008) | Policy-maker | (63) | Group discussion including participants of the study | Life expectancy gains and average life-year benefit per patient | Group discussion and questionnaire (deliberative process) | Ranking of the important reason in priority-setting |
| Mshana et al. (2007) | District health planner | (116) | NA | NA | Group discussion | Describe the participants’ views |
| Senior health staff | ||||||
| General population | ||||||
| Patients | ||||||
| Makundi et al. (2007) | Policy-maker | (31) | Literature review | Prevalence, disease burden, coverage of selected, conditions, severity of disease, efficacy, equity, and cost-effectiveness | Individual rating, group discussion and balance sheet method (deliberative process and reach consensus) | Rank ordering of the nine selected interventions (from an essential healthcare intervention package in Tanzania) |
| Health worker | (23) | |||||
| General population | (21) | |||||
| People living with HIV/AIDS | (10) | |||||
| Rubinstein et al. (2007) | Policy-maker (macro-, meso- and micro-level) | (20) | Focus group and interviews | Evidence-based clinical guidelines, individual impact and benefit, social impact and benefit, cost and consequences, available of resources and financial incentives | Focus group and interviews | List of criteria |
| Baltussen et al. 2007 | Policy-maker | (7) | Literature review and group discussion including participants of the study | Severity of disease, health benefits, age of target group, positive poverty reduction and cost-effectiveness | Individual rating and discrete choice experiment | Rank ordering of 33 interventions which address an important part of burden of disease in Nepal |
| Health professionals | (66) | |||||
| Husain et al. (2007) | Policy-maker | (10) | In-depth interview | Equity and efficiency | Interview | Describe policy-maker’s views |
| Madi et al. (2007) | Policy-maker | (21) | Self-administered questionnaire and group discussion including participants of the study | National priorities, marternal mortality, quality of maternity care services, effectiveness and cost-effectiveness | Group discussion, questionnaire with rating questionnaire (Deliberative process and reach consensus) | The first three most important of safe motherhood programme characterisitics and the priority evaluation questions |
| Head of organization | (15) | |||||
| Health worker | (45) | |||||
| Kapiriri et al. (2007) | Policy-maker | (81) | Interviews | Macro-level: political pressure, advocacy and international priorities | Semi-structured interview | Describing criteria for priority-setting |
| Health worker | (103) | Meso-level: historical budgets, volume of activity, emergencies and need | ||||
| Micro-level: medical and social worth (health state, expected benefit) | ||||||
| Kapiriri & Martin 2006 | Policy-maker | (14) | One-on-one interviews | Strategic plan, evidence and need (in terms of the number of beds per directorate, medical emergencies and the patient load) | Interview and document analysis | Describing criteria for priority-setting |
| Health worker | (56) | |||||
| Baltussen et al. 2006 | Policy-maker | (30) | Group discussion including participants of the study | Cost-effectiveness, poverty reduction, age of target group, severity of disease, health effects and total budget impact | Individual rating and discrete choice experiment | Rank ordering of health interventions |
| Kapiriri and Norheim (2004) | Policy-maker | (28) | Literature review and self-administered questionnaire | Age of patients, cost-effectiveness, treatment costs, severity of disease and equity of access | Questionnaire with 6-point raing scale | List of criteria and their weights according to number of respondents who agreed criterion was important |
| Health worker | (320) | |||||
| General publication | (59) | |||||
| Kapiriri et al. (2004) | Policy-maker | (37) | Group discussion including participants of the study and interviews with international development partners and national level government officers | Cost-effectiveness, severity of disease, equality, costs of care, effectiveness of treatment/intervention, number affected, affects children, availability of effective intervention and preventable | Brain storming, interview and questionnaire (with 6-point rating scale and three scenarios) | Rank ordering of the criteria and respondent’s choices in the three scenarios |
| Health worker | (326) | |||||
| General population | (50) | |||||
| Reichenbach (2002) | Policy-maker | (115) | Literature review, interview and analysis of media attention | Direct attention: incidence, mortality and morbidity data, disability-adjusted life years, actual costs, cost-effectiveness | Interview and secondary data | Demonstrating the influence of political and organizational factors on priority |
| Programme managers | Process attention: direct and indirect measures of social and organizational | |||||
| Academic, scientific and Non-Government Organization communities | Capacity (e.g. budget, trainings, reports) | |||||
| Political attention: committee created, regional and district activities, funding sources, involvement of private/ NGO sectors and media attention | ||||||
| Hrabačet al. (2000) | None | NA | Vulnerable groups, ensuring safe motherhood and pregnancy, life-threatening conditions and the health of the entire population, preventive healthcare, prevention of high degree of disability, cost-effectiveness and efficiency | NA (secondary data analysis) | Describing criteria for priority-setting | |
| Söderlund (1999) | None | Literature review | The extent to which there was another appropriate responsible party who should pay for treatment, the urgency of required treatment, cost-effectiveness | NA (Secondary data analysis) | Ranking of possible interventions included in an essential hospital package | |
- NA, not available; NGO, Non-Government Organization.
In terms of eliciting preferences for those criteria, a wide range of approaches were used. Eight studies relied solely on (combinations of) qualitative approaches to elicit participants’ preferences, i.e. by semi-structured interviews, group discussions and key informant interviews. Another three studies relied solely on quantitative approaches to elicit participants’ preferences, i.e. by discrete-choice experiments (DCE) and questionnaires involving a rating scale. Four studies combined qualitative and quantitative techniques, i.e. Makundi et al. (2007) employed individual rating and group discussions with a balance sheet to test a model of combining evidence and public values in priority-setting and Ottersen et al. (2008), Madi et al. (2007) and Kapiriri et al. (2004) used group discussions and questionnaires with rating questions to explore respondents’ preferences regarding cost-effectiveness and severity of disease. It is to be noted that three studies applied an explicit deliberative process to address both quantitative and non-quantitative concerns (such as ethical considerations) (Makundi et al. 2007; Madi et al. 2007; Ottersen et al. 2008) and they did so to reach a consensus by the stakeholders involved.
In terms of presentation of results, seven studies rank-ordered health interventions, three studies rank-ordered identified criteria, five studies listed the criteria for setting priorities and three studies described participants’ views.
Discussion
This review has revealed an increase in the number of empirical studies on priority-setting in developing countries in the past decade. Methods for explicit priority-setting are developing, being reported and are verifiable and replicable. In combination with increasingly available evidence of all sorts on diseases and related interventions, these methods can potentially be solutions for the ad hoc policy-making in health care in many developing countries. Yet, most of the studies included in our review are small pilot studies and do not include an evaluation of the impact of its finding on actual priority-setting. Only when such information becomes available, clear recommendations to scale up certain methods can be given.
Nevertheless, the review allows us to provide a number of suggestions on the various aspects of the methodological approaches, on the basis of a comparison of our findings to observations on good priority-setting practice in the literature. First, most of the studies in this review involved policy-makers, health workers and general population in their priority-setting process. This concurs with observations in the literature (Fleck 1994; Ham & Coulter 2001; Martin et al. 2003; Vuorenkoski et al. 2008) that stress the need to involve the views of other stakeholders in addition to those of policy-makers, especially that of the public, in debates on rationing to enhance the legitimacy and fairness of decision-making. We therefore suggest future studies to involve relevant stakeholders.
Secondly, a number of studies involved only a limited number of quantitative criteria, whereas observations in the literature (Musgrove 1999; Baltussen & Niessen 2006) stress that many other criteria, including medical (e.g. effectiveness of interventions and severity of disease) and non-medical (e.g. economic efficiency, ethical reasons and political circumstances) criteria, may also be important and relevant. In addition, some studies identified criteria through literature review, whereas the definitions of criteria are likely to be dependent on culture and perspective. Identifying these criteria through focus group discussions with relevant stakeholders is probably a better approach to obtain a suitable set of criteria.
Thirdly, a number of studies relied solely on quantitative techniques, such as discrete choice experiments, to elicit preferences of respondents. Where the advantage of such techniques is that its results can be applied across disease areas/interventions, their disadvantage is that not all criteria that are relevant to priority-setting are amenable to quantification (not only ethical and social acceptability but also more practical considerations like intervention complexity) and these techniques then fall short of capturing these (Gonzalez-Pier et al. 2006). A number of studies have used qualitative techniques such as deliberative processes. Such techniques have the advantage that they are also able to address non-quantitative concerns and that they explicitly allow the inclusion of views of different stakeholders (Green & Gerard 2008) and the reaching of consensus (Gonzalez-Pier et al. 2006). The disadvantage is that its results are only relevant to the disease area/interventions under study and cannot be generalized across disease areas/interventions. Our suggestion then, is that quantitative techniques such as DCE may be relevant to situations where general guidance on priority-setting is required and that qualitative techniques may be more apt in situations where more specific decisions are required on, e.g. implementation of certain interventions (cf. Murray et al. 2000 on the need for generalized vs. highly contextualized CEA). The added value of quantitative techniques such as DCE in the latter situation, e.g. to make decisions more transparent and explicit, is a topic for further research.
Finally, a number of studies presented their results in mere descriptive format such as identified criteria or respondents’ preferences, whereas studies on priority-setting have the intrinsic aim to rank-order interventions, or more specifically, to identify interventions that should be included or excluded from, e.g. public reimbursement (Baltussen & Niessen 2006). To the extent study objectives allow, we suggest studies to (also) present the impact of their findings in this respect.
Our study has a number of limitations. First, our review only included studies in English and incorporated in Medline and EMBASE (Ovid) databases. This may mean that ‘grey literature’ (such as government reports, unpublished reports, academic theses and conference proceedings) and publications in other languages were not identified from the search. Secondly, our classification of study methodologies may not be comprehensive and other methodological issues can also be important.
