SEARCH

SEARCH BY CITATION

Keywords:

  • health services research;
  • maternity services;
  • emergency obstetric care;
  • millennium development goals;
  • health services accessibility;
  • geographical information systems
  • recherche sur les services de santé;
  • services de maternité;
  • soins obstétriques d’urgence;
  • Objectifs de Développement du Millénaire;
  • accessibilité aux services de santé;
  • systèmes d’information géographique
  • Investigación en servicios sanitarios;
  • servicios de maternidad;
  • urgencias obstétricas;
  • objetivos de desarrollo del milenio;
  • accesibilidad a los servicios sanitarios;
  • sistemas de información geográfica

Summary

  1. Top of page
  2. SummarySuivi des progrès vers une maternité sans risque: Atteindre le repère mais rater l’objectif? Un appel pour une meilleure utilisation des indicateurs des résultats du système de santé avec les exemples de la Zambie et du Sri LankaSeguimiento del Progreso hacia una Maternidad Segura: ¿Alcanzando la cota pero no el objetivo? Llamado para un mejor uso de los indicadores de resultados del sistema sanitario con evidencia proveniente de Zambia y Sri Lanka
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions and recommendations
  8. Acknowledgements
  9. References
  10. Supporting Information

Objectives  Indicators of health-system outputs, such as Emergency Obstetric Care (EmOC) density, have been proposed for monitoring progress towards reducing maternal mortality, but are currently underused. We seek to promote them by demonstrating their use at subnational level, evaluating whether they differentiate between a high-maternal-mortality country (Zambia) and a low-maternal-mortality country (Sri Lanka) and assessing whether benchmarks are set at the right level.

Methods  We compared national and subnational density of health facilities, EmOC facilities and health professionals against current benchmarks for Zambia and Sri Lanka. For Zambia, we also examined geographical accessibility by linking health facility data to population data.

Results  Both countries performed similarly in terms of EmOC facility density, implying this indicator, as currently used, fails to discriminate between high- and low-maternal-mortality settings. In Zambia, the WHO benchmarks for doctors/midwives were met overall, but distribution between provinces was highly unequal. Sri Lanka overshot the suggested benchmarks by three times for midwives and over 30 times for doctors. Geographical access in Zambia – which is much less densely populated than Sri Lanka – was poor, less than half the population lived within 15 km of an EmOC facility.

Conclusions  Current health-system output indicators and benchmarks on EmOC need revision to enhance discriminatory power and should be adapted for different population densities. Subnational disaggregation and assessing geographical access can identify gaps in EmOC provision and should be routinely considered. Increased use of an improved set of output indicators is crucial for guiding international efforts towards reducing maternal mortality.

Suivi des progrès vers une maternité sans risque: Atteindre le repère mais rater l’objectif? Un appel pour une meilleure utilisation des indicateurs des résultats du système de santé avec les exemples de la Zambie et du Sri Lanka

Objectifs:  Les indicateurs des résultats du système de santé tels que la densité des soins obstétriques d’urgence (SOU), ont été proposés pour le suivi des progrès vers la réduction de la mortalité maternelle, mais sont actuellement sous-utilisés. Nous cherchons à les promouvoir en démontrant leur utilisation au niveau infranational, en évaluant s’ils diffèrent entre un pays à mortalité maternelle élevée (Zambie) et un pays à mortalité faible (Sri Lanka) et en évaluant si les repères ont été posés au bon niveau.

Méthodes:  Nous avons comparé la densité nationale et infranationale des établissements de santé, les services de SOU et les professionnels de la santé par rapport aux repères actuels, en Zambie et au Sri Lanka. Pour la Zambie, nous avons également examiné l’accessibilité géographique en reliant les données des établissements de santéà des données de la population.

Résultats:  Les deux pays avaient une performance similaire en termes de densité des services SOU, ce qui implique que cet indicateur, tel qu’il est actuellement utilisé, ne parvient pas à discriminer entre les zones à mortalité maternelle faible et celles à mortalitéélevée. En Zambie, la référence de l’OMS pour les médecins/sages-femmes était respectée dans l’ensemble, mais la répartition entre les provinces était très inégale. Le Sri Lanka a dépassé de trois fois les objectifs de référence proposés pour les sages-femmes et plus de 30 fois ceux pour les médecins. L’accès géographique en Zambie – beaucoup moins densément peuplée que le Sri Lanka –était faible, moins de la moitié de la population vivait à moins de 15 km d’un centre de SOU.

Conclusions:  Les indicateurs actuels de résultats du système de santé et les repères pour les SOU nécessitent une révision afin de renforcer leur pouvoir discriminatoire et devraient être adaptés pour différentes densités de population. La désagrégation infranationale et l’évaluation de l’accès géographique peuvent identifier les lacunes dans la prestation des SOU et devraient être systématiquement prises en considération. L’utilisation accrue d’un ensemble d’indicateurs améliorés des résultats est cruciale pour guider les efforts internationaux à réduire la mortalité maternelle.

Seguimiento del Progreso hacia una Maternidad Segura: ¿Alcanzando la cota pero no el objetivo? Llamado para un mejor uso de los indicadores de resultados del sistema sanitario con evidencia proveniente de Zambia y Sri Lanka

Objetivos:  Se han propuesto indicadores de resultados del sistema sanitario, tales como la densidad de las urgencias obstétricas, para monitorizar el progreso hacia la reducción de la mortalidad materna, que actualmente están siendo subutilizados. Buscamos promoverlos demostrando su uso a nivel subnacional, evaluando si hay diferencias entre los de un país con una alta mortalidad materna (Zambia) y otro con una baja mortalidad materna (Sri Lanka), y evaluamos si las cotas alcanzadas están establecidas en el nivel adecuado.

Métodos:  Hemos comparado las densidades a nivel nacional y subnacional de los centros sanitarios, de centros de urgencias obstétricas y de profesionales sanitarios, frente a las cotas actuales para Zambia y Sri Lanka. Para Zambia, también examinamos la accesibilidad geográfica, conectando los datos de centros sanitarios con datos sobre la población.

Resultados:  Ambos países tuvieron un desempeño similar en términos de la densidad de las urgencias obstétricas, lo que implica que este indicador, tal y como se utiliza actualmente, no logra discriminar entre lugares con una alta y baja mortalidad materna. En Zambia, las cotas de la OMS para doctores/comadronas se cumplieron en general, pero la distribución entre provincias era muy desigual. Sri Lanka sobrepasó en tres veces las cotas sugeridas para comadronas, y en 30 veces las de doctores. El acceso geográfico en Zambia – que tiene una densidad poblacional mucho menor que Sri Lanka – era pobre; menos de la mitad de la población vivía dentro de un radio de 15 km de unas instalaciones de urgencias obstétricas.

Conclusiones:  Los actuales indicadores de resultados del sistema sanitario y las cotas para urgencias obstétricas tienen que revisarse para mejorar su poder discriminatorio, y deberían adaptarse para diferentes densidades poblacionales. Un desglose a nivel subnacional y la evaluación del acceso geográfico pueden identificar brechas en las prestaciones de las urgencias obstétricas, y deberían tenerse en cuenta de forma rutinaria. Un mayor uso de un conjunto mejorado de indicadores de resultados es crucial para guiar los esfuerzos internacionales hacia la reducción de la mortalidad materna.


Introduction

  1. Top of page
  2. SummarySuivi des progrès vers une maternité sans risque: Atteindre le repère mais rater l’objectif? Un appel pour une meilleure utilisation des indicateurs des résultats du système de santé avec les exemples de la Zambie et du Sri LankaSeguimiento del Progreso hacia una Maternidad Segura: ¿Alcanzando la cota pero no el objetivo? Llamado para un mejor uso de los indicadores de resultados del sistema sanitario con evidencia proveniente de Zambia y Sri Lanka
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions and recommendations
  8. Acknowledgements
  9. References
  10. Supporting Information

The main indicators proposed for tracking Millennium Development Goal 5 (MDG5) via the UN and Countdown to 2015 include a health impact indicator (maternal mortality), several outcome indicators (use of antenatal care or skilled attendance at delivery) and some health-system output indicators (namely Emergency Obstetric Care facility density at national and subnational levels). This is demonstrated in Figure 1, using a framework suggested by the UN and the International Health Partnership (United Nations Development Group 2005; IHP+ 2008).

image

Figure 1.  Indicators for tracking progress towards MDG5. EmOC, Emergency Obstetric Care. Financial, material, training and managerial input into the health system should lead to outputs in the form of sufficiently staffed facilities capable of providing EmOC. If women use these for giving birth, skilled attendance at delivery is ensured and maternal and newborn survival can be achieved.

Download figure to PowerPoint

Most countries collect empirical data on outcome indicators, generally garnered from population-based surveys such as the Demographic and Health Surveys (MEASURE DHS) or Multiple Indicator Cluster Surveys (UNICEF). These are tabulated and made available by UN agencies, for example the World Health Organization Statistical Information System and Global Health Observatory (WHO; WHOSIS). Yet these outcome indicators are often crudely measured (Hussein et al. 2004; Harvey et al. 2007) and may be misleading: The indicator of skilled attendant at birth has been criticized for not capturing quality of care and therefore not aligning well with maternal mortality (Shankar et al. 2008).

Empirically measured health impact data are harder to come by, despite substantial interest in tracking maternal mortality both as a hard measure of success and as the main target indicator for MDG5. The UN publish maternal mortality estimates every 5 years and produce predicted statistics for countries without data (WHO et al. 2010). In the absence of good vital registration data, maternal mortality is costly to measure in surveys and consequently is carried out infrequently and thus does not easily lend itself to monitoring short-term progress (Graham et al. 2008).

Indicators of health-system outputs are the least available or used despite having been promoted as ‘process indicators’ as early as the 1997 ‘UN guidelines for monitoring the availability and use of obstetric services’ and subsequently in the World Health Report (WHR) 2005 and the 2009 UN handbook ‘Monitoring emergency obstetric care’. Emergency Obstetric Care (EmOC) facility density is not recorded on the Global Health Observatory (WHO) or the UN MDG indicator site (United Nations Statistics Division). Countdown to 2015 only reports it for 40% of its 68 countries (UNICEF 2008). Also, despite recommendations to collect this indicator in ‘areas smaller than the country as a whole’ (UNICEF et al. 1997), subnational EmOC density ‘remains globally underused’ (Fauveau & Donnay 2006), and tabulation is restricted to a few reports and articles (AMDD). Doctor and midwife density and geographical accessibility of EmOC services have hardly been used at all for tracking progress towards safe motherhood in low-income countries.

This is regrettable because health-system output data are comparatively easy to collect and provide valuable information by pointing towards required actions. Countries with inadequate provision of services cannot expect maternal mortality to decline unless services are upgraded and accessibility improved. Measuring maternal mortality through expensive survey approaches before gaining a basic understanding of what kind of care is accessible wastes resources (Graham et al. 1996; UNICEF et al. 1997).

This article aims to encourage the use of health-system output indicators to track progress towards MDG5. To achieve this, we (i) illustrate for Zambia, a setting with high maternal mortality, how these indicators can be studied nationally and subnationally, (ii) assess the indicators’ discriminatory power by comparing Zambia’s indicator performance to that of Sri Lanka (where maternal mortality is far lower) and (iii) assess whether the proposed benchmarks appear to be set at the right levels.

Methods

  1. Top of page
  2. SummarySuivi des progrès vers une maternité sans risque: Atteindre le repère mais rater l’objectif? Un appel pour une meilleure utilisation des indicateurs des résultats du système de santé avec les exemples de la Zambie et du Sri LankaSeguimiento del Progreso hacia una Maternidad Segura: ¿Alcanzando la cota pero no el objetivo? Llamado para un mejor uso de los indicadores de resultados del sistema sanitario con evidencia proveniente de Zambia y Sri Lanka
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions and recommendations
  8. Acknowledgements
  9. References
  10. Supporting Information

Current EmOC benchmarks from the UN handbook (WHO et al. 2009), which updated the UN guidelines (UNICEF et al. 1997), and from the WHR 2005 (WHO 2005) are presented in Table 1.

Table 1.   Output indicators and benchmarks (implied numbers in italics)
Indicator sourceWorld Health Report 2005UN handbook 2009
  1. EmOC, emergency obstetric care (BEmOC or CEmOC); BEmOC, basic emergency obstetric care; CEmOC, comprehensive emergency obstetric care.

  2. *Preliminary/supplementary indicator.

Reference births (per year)360020 000
Facilities
 EmOC facilities per reference births 2–3 (or more) At least 5
 CEmOC facilities per reference births1At least 1
 BEmOC facilities per reference births1–2 (or more) At least 5 minus CEmOC facilities
Staffing
 Midwives per reference births20 
 Doctors per reference births3 part-time 
 Midwives per EmOC facilitye.g. 10 in CEmOC, 5 in each of 2 BEmOC 
 Doctors per CEmOC facility3 part-time 
Geographical accessibility*
 Travel time to EmOC facility for most women ≤2–3 h

Zambia is a large, land-locked, sparsely populated low-income country in sub-Saharan Africa, and Sri Lanka is a densely populated, lower middle–income South Asian island. Maternal mortality is very high in Zambia, whereas Sri Lanka is a maternal health success story, having reduced maternal mortality continuously over the last decades. Table 2 presents some basic country information, with Sweden as a high-income comparator.

Table 2.   Basic country information on Zambia and Sri Lanka (Sweden for comparison) as provided by WHO
 ZambiaSri LankaSweden
  1. WHOSIS, World Health Organization Statistical Information System (now incorporated into the Global Health Observatory); GDP, gross domestic product; PPP, purchasing power parity exchange rate.

  2. *From Maternal Mortality 2005 estimates (WHO et al. 2007).

  3. Sri Lanka MoH 2006, not on WHOSIS.

Land area (km2)740 00065 000410 000
Population (WHOSIS 2006)11.7 million19.2 million9.1 million
Crude birth rate (UNDATA 2000–2005)45 per 100019 per 100011 per 1000
Maternal deaths per year (WHO 2005)*39001903
Maternal mortality ratio (WHO 2005)*830583
Infant mortality rate (WHOSIS 2006)100 per 100010 per 10003 per 1000
Doctors per 10 000 population (WHO 2004)1633
Nurses and midwives per 10 000 pop. (WHO 2006)717116
Skilled attendance at birth (WHOSIS 2002, 2000)43%97%100%
Caesarean section rate (WHOSIS 2001)2%24%17%
GDP per capita in PPP $ (WHOSIS 2006)1100370037 000
Total health expenditure per capita in PPP $ (for 2002, from WHR 2005)511312500

We used data from the Zambian Census of Population and Housing 2000, the Zambian Health Facility Census 2005 (Health Facility Assessment Technical Working Group 2005) (containing information on all the country’s health facilities, excepting some private-for-profit facilities), the Sri Lankan Needs Assessment ‘Women’s Right to Life and Health Project’ reports 2001 and 2003, and published data on births, health facilities and staffing from Sri Lankan government websites (Department of Census and Statistics Sri Lanka).

To evaluate EmOC density, we first considered all facilities offering delivery care and then only the subgroup providing certain EmOC signal functions: injectable antibiotics, injectable oxytocics, injectable anticonvulsants, manual removal of placenta, manual removal of retained products, assisted vaginal delivery, Caesarean section and blood transfusion. Comprehensive EmOC (CEmOC) is characterized by performance of all eight functions and basic EmOC (BEmOC) by the first six; actual provision in the previous three months should be established by checking registers, drugs, equipment and staffing (UNICEF et al. 1997; Paxton et al. 2006).

The EmOC assessments conducted in 2001/2003 in Sri Lanka included all higher-level hospitals and a sample of lower-level hospitals. To estimate the number of CEmOC facilities in 2006, we multiplied the number of higher-level hospitals in 2006 by the proportion found to provide all eight signal functions in the assessments of 2001/2003, separately for each province. Similarly, the number of BEmOC facilities in 2006 was estimated by multiplying the number of lower-level hospitals by the proportion found to provide the six basic signal functions.

The Zambian Health Facility Census did not check actual performance of signal functions, but only reported theoretical capability, which is known to overestimate functioning (UNICEF et al. 1997; Paxton et al. 2006). Therefore, our EmOC classification also used criteria on opening hours (24- h presence of a midwife or doctor), staffing (at least three doctors registered and one found on duty for CEmOC; at least three doctors, nurses, midwives or clinical officers registered and one found on duty for BEmOC), availability of electricity (for CEmOC) and referral capacity (for BEmOC).

Facility density was computed per 20 000 births (to match with the literature) (UNICEF et al. 1997; Paxton et al. 2006) and compared to current benchmarks, for EmOC facilities overall (CEmOC and BEmOC) and for CEmOC facilities.

To study health workers, we considered all professional cadres with potential to conduct deliveries who were registered at facilities providing delivery care (namely doctors, clinical officers, nurses and midwives). In Sri Lanka, disaggregated figures for midwives were available by province and work task. Sri Lankan public health midwives mainly provide preventive ante- and post-natal care and only conduct deliveries very occasionally; hospital midwives are the principal providers of delivery care. In the Zambian Health Facility Census, nurse and midwife numbers were aggregated; we used disaggregated national figures from the Global Atlas of the Health Workforce (WHO) to estimate the proportion of midwives in the provinces. Health-professional density was computed per 3600 births (as recommended by WHO) (WHO 2005) and compared to current benchmarks.

In rural Zambia, only 1% of households possess motorized transport and public transport is scarce (Central Statistical Office et al. 2009). Thus, geographical accessibility was estimated as the proportion of births within 15 km of services set to match with a benchmark of 3- h travel time, assuming a walking speed of 5 km/h. We mapped health facilities and ward areas in the geographical information system platform ArcGIS and created circular buffers of 15 km radius around all delivery facilities and around EmOC facilities to calculate the proportion of total ward area covered. Birth density by ward was estimated by projecting annual district population growth rates to 2005 and applying provincial crude births rates. An even spatial distribution of births was assumed.

Results

  1. Top of page
  2. SummarySuivi des progrès vers une maternité sans risque: Atteindre le repère mais rater l’objectif? Un appel pour une meilleure utilisation des indicateurs des résultats du système de santé avec les exemples de la Zambie et du Sri LankaSeguimiento del Progreso hacia una Maternidad Segura: ¿Alcanzando la cota pero no el objetivo? Llamado para un mejor uso de los indicadores de resultados del sistema sanitario con evidencia proveniente de Zambia y Sri Lanka
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions and recommendations
  8. Acknowledgements
  9. References
  10. Supporting Information

Health facilities

Numbers of births, health facilities and EmOC facilities, as well as facility densities per 20 000 births for Zambia and Sri Lanka, are shown in Table 3. Assuming all delivery facilities offer EmOC, both Zambia and Sri Lanka substantially exceeded the ‘5 EmOC of which 1 CEmOC’ benchmark suggested in the UN handbook, both nationally and in all provinces. The WHR 2005 benchmark (of 2 EmOC facilities per 3600 births, or 11.1 per 20 000 births) was also met in both countries and all provinces. However, even under this generous assumption of counting all facilities, both countries fell slightly short of the WHR 2005 CEmOC benchmark of 5.6 facilities per 20 000 births, as did most Sri Lankan and half of the Zambian provinces (Figure 2a,b).

Table 3.   Births, facilities and health professionals in Zambia and Sri Lanka Thumbnail image of
image

Figure 2.  EmOC facility indicator performance in Zambia and Sri Lanka. EmOC, Emergency Obstetric Care (BEmOC or CEmOC); BEmOC, Basic Emergency Obstetric Care; CEmOC, Comprehensive Emergency Obstetric Care. Benchmarks represented by vertical lines: UN handbook benchmarks per 20 000 births: 5 EmOC (dark blue) of which 1 CEmOC (orange); WHR 2005 benchmarks per 20 000 births: 11.1 EmOC (light blue) of which 5.6 CEmOC (yellow). When considering all delivery facilities (left panels), indicator performance in Zambia (a) and Sri Lanka (b) was far in excess of the UN benchmarks. The WHR 2005 EmOC benchmark was also easily met, but not its CEmOC benchmark nationally and in most provinces. When considering only facilities truly offering EmOC (right panels), both Zambia (c) and Sri Lanka (d) fell far short of the WHR 2005 benchmarks and several provinces also of the UN EmOC benchmark. Sri Lanka had far more CEmOC facilities than Zambia, easily meeting the UN CEmOC benchmark, while several Zambian provinces failed it.

Download figure to PowerPoint

When assumptions were tightened to assess EmOC functioning, both countries fell far short of the WHR 2005 benchmarks, and several provinces in both countries also failed the UN handbook’s EmOC benchmark. While all of Sri Lanka’s provinces had a CEmOC facility density well in excess of the UN CEmOC benchmark, several Zambian provinces failed this benchmark (Figure 2c,d).

Health professionals

Health professionals are a critical component of EmOC provision. Total numbers as well as densities per 3600 births are presented in Table 3. The WHR 2005 benchmark of three doctors per 3600 births was met in Zambia overall, with a national average of eight doctors per 3600 births. There was, however, substantial variation between regions, and the benchmark was not met in four of the nine provinces (Figure 3a) and half of all districts (data not shown). Sri Lanka overshot the benchmark by over 30 times, with a national average of 99 doctors per 3600 births (Figure 3b).

image

Figure 3.  Health-professional indicator performance in Zambia and Sri Lanka. *The ratio of nurses to midwives for the provinces was estimated from the national figures. Benchmarks represented by vertical lines: WHR 2005: three doctors (orange) and 20 midwives (blue) per 3600 births. Doctor density is shown on the left panels and density of other health workers on the right panels. Zambia (a) met the doctor benchmark nationally, but not in all provinces. Doctor density in Sri Lanka (b) was 10 times higher than in Zambia and far above the benchmark. Midwife density in Zambia (c) just met the benchmark, but was unequal between provinces. In Sri Lanka, midwife density (d) was much higher, and the benchmark was still met when only counting midwives working in delivery care (hospital midwives, dark blue).

Download figure to PowerPoint

The total number of nurses, midwives and clinical officers in both Zambia and Sri Lanka greatly exceeded the benchmark of 20 per 3600 births. When counting only midwives, Zambia exactly met the benchmark nationally, but again distribution was unequal and six of nine provinces failed it. In Sri Lanka, the country and all its provinces were well above the 20 midwives per 3600 births benchmark. This remained true in the country overall and most provinces when counting only hospital midwives, the chief providers of delivery care in Sri Lanka (Figure 3c,d).

Geographical accessibility

For Zambia overall, 86% of births were within 15 km of a facility offering delivery care, but only 44% were within 15 km of a facility offering EmOC (Figure 4). There were large differences between provinces and districts, with much poorer EmOC access in rural areas: 95% of urban but only 16% of rural births were within 15 km of EmOC (data not shown).

image

Figure 4.  Geographical access to delivery care and EmOC in Zambia. EmOC, Emergency Obstetric Care. The bars show the percentage of the population living within 15 km of services, nationally and by province. Access to delivery care (light blue) was reasonable with more than 60% of the population being within 15 km of delivery care in all provinces. Access to EmOC (dark blue) was much worse, with <20% of the population within 15 km of EmOC in some provinces, and a high proportion only in the two urbanized provinces Copperbelt and Lusaka.

Download figure to PowerPoint

Comparing performance of Zambian districts in terms of geographical accessibility to EmOC services with performance against the indicator of EmOC facility density (Figure 5) revealed a discrepancy: while some predominantly rural districts met the UN EmOC facility benchmark, hardly any had good geographical accessibility, with mostly less than half the population living within 15 km of an EmOC facility.

image

Figure 5.  Performance comparison of Zambian districts for two indicators: EmOC facility density and geographical access to EmOC. EmOC, Emergency Obstetric Care. Benchmarks for EmOC facility density represented by horizontal lines: UN handbook benchmark: 5 EmOC per 20 000 births (dark blue); WHR 2005 benchmark: 11.1 EmOC per 20 000 births (light blue). Benchmark for geographical access represented by purple vertical line: UN handbook recommendation is within 3 h for most women (translating into 15- km distance if assuming a walking speed of 5 km per hour). The 72 Zambian districts are represented by circles with size according to population and colour according to urbanization. Districts meeting both a density benchmark and the geographical benchmark are located in the top right and districts meeting neither in the bottom left corner. Several rural districts (green) met the UN EmOC benchmark (dark blue horizontal line), but hardly any had good geographical access to EmOC. Accessibility in most urban districts (orange) was good.

Download figure to PowerPoint

Benchmarks and birth density

Figure 6a elaborates on how EmOC facility and staffing benchmarks may be specified for settings with different population densities, as proposed in the WHR 2005. Assuming a facility’s birth load consists of all births within a 15 km radius (an area of approximately 700 km2) and 20 midwives are needed for a district of 3600 births (as suggested in the WHR 2005), it becomes obvious that in areas with birth densities below 500 per 700 km2, the number of deliveries required to keep three midwives busy is not achieved. Consequently, either some areas will not have easy access, more midwives will be deployed than necessitated by the birth load (with the concomitant risk of losing infrequently practised skills), or facilities will be staffed by fewer midwives, thus compromising 24-h care.

image

Figure 6.  (a) Distribution and size of EmOC facilities for different birth density scenarios; (b) District birth densities in Zambia and Sri Lanka. EmOC, Emergency Obstetric Care. EmOC facilities are symbolized by red crosses; the small pink crosses signify a lower level of care. Catchment areas around facilities are symbolized by circles with a radius of 15 km (maximum walking distance) resulting in areas of around 700 km2. Panel (a) depicts EmOC provision in hypothetical districts of 3600 births for four different scenarios: In very densely populated districts, one can efficiently station all midwives in one facility. Districts where births are distributed over larger areas require several smaller facilities to ensure geographical accessibility. If three midwives per facility is the minimum to ensure 24- h quality care and a midwife can deliver 175 births annually (WHO 2005), birth densities below 500 births per 700 km2 pose difficulties in provision of both accessible and quality services. Panel (b) shows maps of Zambia (left) and Sri Lanka (right), proportional to size and the corresponding birth densities by district, demonstrating that the majority of Zambian districts have birth densities below the threshold.

Download figure to PowerPoint

In Zambia, over half the 72 districts (comprising 35% of the country’s births) had a birth density below 500 births per 700 km2, about a quarter of districts generated between 500 and 1000, and another quarter generated over 1000 births per 700 km2. In Sri Lanka, no district had <500 births per 700 km2 (Figure 6b).

Discussion

  1. Top of page
  2. SummarySuivi des progrès vers une maternité sans risque: Atteindre le repère mais rater l’objectif? Un appel pour une meilleure utilisation des indicateurs des résultats du système de santé avec les exemples de la Zambie et du Sri LankaSeguimiento del Progreso hacia una Maternidad Segura: ¿Alcanzando la cota pero no el objetivo? Llamado para un mejor uso de los indicadores de resultados del sistema sanitario con evidencia proveniente de Zambia y Sri Lanka
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions and recommendations
  8. Acknowledgements
  9. References
  10. Supporting Information

We showed that health-system output indicators on EmOC, namely indicators of facility density, health-professional density and geographical access, can be created on national and subnational levels from existing country data. Furthermore, we evaluated the discriminatory power of the indicators by comparing performance between Zambia and Sri Lanka and assessed the appropriateness of their benchmarks.

Good discriminatory power means that a setting with low mortality should perform better on an indicator (e.g. have higher doctor density) than a setting with high mortality. Meeting the ‘minimum acceptable level’ (UNICEF et al. 1997) benchmark for an indicator (e.g. having at least three doctors per 3600 births) should translate into low mortality, ideally as both a necessary and sufficient condition. In reality, there are trade-offs between high sensitivity (all high-mortality settings fail the benchmark) and high specificity (all low-mortality settings meet the benchmark), because indicators rarely align completely with their outcome and other factors confound and modify the relationship. Therefore, it is crucial to consider several indicators together.

The overall number of delivery facilities in Zambia and Sri Lanka far exceeded all EmOC and CEmOC benchmarks. When counting only those facilities fulfilling EmOC criteria, both countries failed the WHR 2005 benchmarks but met or almost met the UN benchmarks nationally, although not in all provinces (Figure 2). The much lower performance when applying EmOC criteria confirms the importance of evaluating ‘how facilities are actually functioning, and not […] how they are supposed to function’ (UNICEF et al. 1997).

This similarity of both countries’ performance on EmOC facility density implies low discriminatory power because maternal mortality is high in Zambia and low in Sri Lanka. Although Zambian EmOC facility density may be somewhat overestimated because signal function performance was not actually assessed, the similarity casts doubt on the usefulness of this indicator if used in isolation and at the national level as tracked in Countdown to 2015.

The discriminatory power of CEmOC facility density seems better: Sri Lanka had much higher levels than Zambia. Both countries, and almost every country examined to date (Paxton et al. 2006), including many high-maternal-mortality countries, are well above the UN benchmark of 1 CEmOC facility per 20 000 births – suggesting this benchmark is set too low and needs to be raised. The WHR 2005 benchmark of 5.6 CEmOC facilities per 20 000 births, on the other hand, seems too high, as it was not even met by the total of Sri Lanka’s high-level hospitals, which successfully perform over 80% of the country’s deliveries. As these hospitals usually have two or three obstetric units, each with a consultant obstetrician and 4–6 junior doctors, it may be more appropriate to count obstetric units instead of facilities. If doing this, Sri Lanka would meet the WHR 2005 CEmOC benchmark (Figure S1), supporting the use of obstetric units instead of facilities as a metric.

Interpreting facility density in the absence of facility size is difficult and is probably the main reason why EmOC facility density does not perform well as an indicator, because it treats large hospitals with thousands of deliveries per year the same as small facilities with few beds. Sri Lanka’s EmOC facilities are large, while Zambia’s are small – possibly explaining the similar EmOC facility densities despite vastly different maternal mortality ratios. Indicator performance between provinces is similarly mismatched. The provinces where the capital cities are located (Lusaka in Zambia and Western Province in Sri Lanka) are likely to have the lowest maternal mortality in the country, yet perform badly in terms of number of EmOC facilities per birth load (Figure 2) because they have fewer but larger facilities.

The WHR 2005 provides indicators and benchmarks on delivery staff, thus considering health-system capacity. Health-professional density in Zambia and Sri Lanka was strikingly different (Figure 3), suggesting this indicator may have better discriminatory power.

Zambia had over double the WHR 2005 benchmark of three doctors per 3600 births, and Sri Lanka over 30 times more. The benchmark referred to doctors working (at least part-time) in delivery care, while our numbers refer to all doctors, but nevertheless, this suggests the doctor benchmark may be set too low and needs to be raised.

Concerning midwives, Zambia exactly met the WHR 2005 benchmark of 20 midwives per 3600 births, while Sri Lanka had nearly four times that figure. Again, this suggests the benchmark should be raised, although, if the indicator is interpreted as referring only to midwives working in delivery care, the benchmark may be appropriate. However, this definition may be less practicable as few countries divide work tasks so clearly as to allow ‘delivery midwife’ numbers to be calculated.

The WHR 2005 benchmark on midwife density was estimated assuming an average annual work load of 175 births per midwife (WHO 2005), a figure based on the observed median of certain sub-Saharan district hospitals (Van Lerberghe et al. 1992). Other African hospitals report birth loads ranging from 28 deliveries per midwife to 464 (Health Systems Trust and Department of Health South Africa 1997; Levin et al. 2003). Rather than basing recommendations on observed birth loads from settings where maternal mortality may be high, it would be preferable to base them on ideal birth loads, though these have not been clearly established and different estimates are used (Ogunbekun et al. 1996; Health Systems Trust and Department of Health South Africa 1997; Levin et al. 2000). Data from well-functioning low-mortality settings (such as Sri Lanka) could be used to identify the ideal workload per midwife in different situations (hospital, health centre). For settings where disaggregated figures by health-professional cadre are lacking, it would be helpful to estimate the approximate proportion of health professionals with midwifery skills of the total. This proportion should be higher for settings with high crude birth rate.

Given the uneven distribution of health services and staff inside countries, it has been repeatedly highlighted that national averages may be misleading. Disaggregation into smaller units has been strongly recommended (UNICEF et al. 1997), but is rarely practiced (Fauveau & Donnay 2006). We show, using existing data, that output indicator performance can be assessed subnationally, and we revealed highly inequitable distributions, particularly for doctors in Zambia.

Disaggregating below district level is problematic as populations become so small that less than one facility (with the accompanying staff) is needed to meet the benchmark. Furthermore, facilities may be used by women from across the administrative borders, thus distorting estimates. Where geographical coordinates of facilities are available, as in Zambia, it is possible to calculate the proportion of the population or births within a certain distance of services instead, thus overcoming the problem of estimating access in small units accurately. With the increasing availability of digital mapping and geographical information systems, use of indicators of geographical access to EmOC should increase (WHO et al. 2009).

Applying this method to Zambia revealed that geographical access to EmOC was very low in most provinces (Figure 4). We also demonstrated that districts could meet the benchmark for EmOC facility density without achieving reasonable geographical access for their populations (Figure 5). For Sri Lanka, we did not have the necessary data to calculate geographical access, but given its much smaller area and higher population density, access is likely to be far better – and probably contributes to the low maternal mortality seen there. Geographical accessibility is thus a potentially very useful indicator and should be much more promoted.

Health-system output indicators are also useful in terms of health-system planning (Figure 6). The WHR 2005 recognizes that ‘the problem is to decide on the optimal level of decentralization – the compromise between access and efficiency’ and suggests possible facility sizes and staffing for settings with different population density.

We have developed this further for EmOC in general (ignoring that one of the EmOC facilities should be CEmOC and may need more midwives) by introducing a metric of 700 km2 circular areas of 15 km radius around facilities. This exercise highlighted that for a sparsely populated country like Zambia, balancing the need for geographical accessibility with efficiency and quality of services is challenging, especially in a context of severe health-worker shortages. In areas with very low birth densities, innovative solutions could include improved transportation links, telemedicine, maternity waiting homes and teaming a single midwife with assistant midwives.

Our analyses are obviously limited by the types and quality of data available. Comparing two countries offers only limited proof, but rather demonstrates what can be carried out and highlights potential areas for further improvement. The geographical analysis could be refined by using population figures for areas smaller than wards and considering geographical data on roads, rivers and altitude. While assuming even spatial population distribution within wards will have underestimated accessibility, our choice of 15- km maximum distance probably overestimates it, as it requires three hours of brisk walking from heavily pregnant women. The benchmark for travel time and/or distance can be set differently if desired, depending on the local transport situation. Work also remains to be carried out on better operationalizing measures of EmOC functionality and quality and in integrating neonatal emergency functions.

In conclusion, this analysis showed that current health-system output indicators and their benchmarks may need revision to make them more practically useful and that several indicators should be used in combination. The WHR 2005 suggestions on facility size and staffing for different population densities went into the right direction, but the opportunity to develop these further was missed in the recent handbook update to the UN guidelines. The development of ‘a basic set of indicators of health-system functions and of scientifically sound, practical and user-friendly tools’ (Boerma et al. 2009) should be a high priority in our efforts to achieve MDG5.

This article has focussed on Zambia and Sri Lanka not for their particular results, but rather to show examples of what could be carried out to encourage similar and better analyses. Despite minor limitations (such as failing to report midwives separately, numbers of deliveries and actual signal function performance in the Zambian Health Facility Census, and the lack of geographical coordinates in the Sri Lankan Health Management Information systems), both countries have rich existing data sources that can be used to better understand health-system constraints in providing EmOC.

While facility surveys focussing on EmOC provision can provide detailed and nationally representative data, a national facility census with geographical coordinates, as carried out in Zambia, has three main advantages. First, subnational disaggregation and comparison between provinces and districts is straightforward as there is no issue of whether the sample is representative at lower levels; secondly, it can be linked to population census data to calculate geographical accessibility; and thirdly, it can be used for district level planning to target interventions to underserved areas.

Similar facility censuses should be encouraged in other countries, integrated into routine Health Management Information Systems and put in the public domain. A recent ‘call for action on health data from eight global health agencies’ (Chan et al. 2010) stated that tracking progress towards the MDGs is ‘constrained by limited data availability, quality and use’. Investing in collecting a set of health-system output indicators is a particularly promising approach which can directly guide intervention planning, especially if ‘active management with data’ is carried out with local stakeholders in a participatory way (Shankar et al. 2008).

Conclusions and recommendations

  1. Top of page
  2. SummarySuivi des progrès vers une maternité sans risque: Atteindre le repère mais rater l’objectif? Un appel pour une meilleure utilisation des indicateurs des résultats du système de santé avec les exemples de la Zambie et du Sri LankaSeguimiento del Progreso hacia una Maternidad Segura: ¿Alcanzando la cota pero no el objetivo? Llamado para un mejor uso de los indicadores de resultados del sistema sanitario con evidencia proveniente de Zambia y Sri Lanka
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions and recommendations
  8. Acknowledgements
  9. References
  10. Supporting Information

1. EmOC indicators should have discriminatory power, i.e. better performance in the indicator should be associated with lower maternal mortality.

2. EmOC facility density, as currently used, seems to have low discriminatory power, but could be improved by focussing on CEmOC and by considering facility size.

3. Numbers of doctors and midwives per birth load seems to have better discriminatory power, but current benchmarks may need to be raised.

4. Subnational disaggregation of health-system output indicator performance elucidates inequalities and should be routine.

5. Geographical accessibility should be assessed more frequently as it adds crucial information for adequate service provision.

6. To help local planning, output indicator benchmarks should be adapted to allow for birth density and transport infrastructure.

7. With modification, health-system output indicators have great potential to track progress and guide interventions towards achieving MDG5.

Acknowledgements

  1. Top of page
  2. SummarySuivi des progrès vers une maternité sans risque: Atteindre le repère mais rater l’objectif? Un appel pour une meilleure utilisation des indicateurs des résultats du système de santé avec les exemples de la Zambie et du Sri LankaSeguimiento del Progreso hacia una Maternidad Segura: ¿Alcanzando la cota pero no el objetivo? Llamado para un mejor uso de los indicadores de resultados del sistema sanitario con evidencia proveniente de Zambia y Sri Lanka
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions and recommendations
  8. Acknowledgements
  9. References
  10. Supporting Information

We want to thank the Zambian Ministry of Health for the kind permission to use the Health Facility Census 2005 data set and the Central Statistical Office for the provision of the population and geographical data from the Zambian Census of Population and Housing 2000. Furthermore, we acknowledge JICA for their assistance with the Health Facility Census in Zambia and DFID for their help in establishing the contact with the Zambian Ministry of Health and Central Statistical Office. We thank Jon Cox for support with the GIS analysis, and Anna Goodman and Volker Winkler for helpful comments on the final manuscript.

References

  1. Top of page
  2. SummarySuivi des progrès vers une maternité sans risque: Atteindre le repère mais rater l’objectif? Un appel pour une meilleure utilisation des indicateurs des résultats du système de santé avec les exemples de la Zambie et du Sri LankaSeguimiento del Progreso hacia una Maternidad Segura: ¿Alcanzando la cota pero no el objetivo? Llamado para un mejor uso de los indicadores de resultados del sistema sanitario con evidencia proveniente de Zambia y Sri Lanka
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions and recommendations
  8. Acknowledgements
  9. References
  10. Supporting Information

Supporting Information

  1. Top of page
  2. SummarySuivi des progrès vers une maternité sans risque: Atteindre le repère mais rater l’objectif? Un appel pour une meilleure utilisation des indicateurs des résultats du système de santé avec les exemples de la Zambie et du Sri LankaSeguimiento del Progreso hacia una Maternidad Segura: ¿Alcanzando la cota pero no el objetivo? Llamado para un mejor uso de los indicadores de resultados del sistema sanitario con evidencia proveniente de Zambia y Sri Lanka
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions and recommendations
  8. Acknowledgements
  9. References
  10. Supporting Information

Figure S1. EmOC facility indicator performance in Sri Lanka, counting obstetric units instead of facilities.

FilenameFormatSizeDescription
TMI_2741_sm_f1.doc27KSupporting info item

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.