‘We are data rich, but information poor!’ So said one of the senior Ministry of Health officials in Fiji when a team from the International Union Against Tuberculosis and Lung Disease and the University of Fiji was discussing the start of an operational research capacity building initiative in that country two years ago. This statement probably sums up the situation for many Ministries of Health and Non-Governmental Organizations (NGOs) in low- and middle-income countries (LMIC), where large amounts of data are regularly collected and collated from health facilities, filed away or shelved and never used, analysed or interpreted to drive improved performance of health services, the scale up of interventions or better outcomes for patients. This paradigm needs to change. Furthermore, such data could be used in the context of operational research to ask more far-reaching questions about quality, coverage, safety and effectiveness of programmes or health systems.
National tuberculosis control programmes (NTPs) throughout the world serve as a model for data collection. One of the essential components of the WHO Stop TB Strategy is a standardised monitoring, evaluation and reporting system to account for all TB cases diagnosed along with their treatment outcomes through cohort monitoring (WHO 2006). Within well-functioning NTPs, TB officers review patient data every three months in terms of who and how many are accessing treatment, whether treatment outcomes are satisfactory, and whether adverse outcomes such as death or loss to follow-up are being kept to a minimum. These data are used to assess programme performance, and they also serve as the foundation for district and national drug forecasting and procurement.
The routine data can also form the basis of operational research, which we define as research into strategies, interventions, tools or knowledge that can enhance the quality, coverage, effectiveness or performance of the health system or disease programme in which the research is being conducted (Zachariah et al. 2009). Between 1996 and 2005, the Malawi NTP formally incorporated operational research into its routine activities, and a large number of research projects were undertaken to improve policy and practice and reduce morbidity and mortality of patients (WHO 1999).
One particular example concerned high case-fatality rates in HIV-infected TB patients before the advent of antiretroviral therapy (ART). The country undertook a series of district-based operational research studies to determine whether it was feasible to offer a package of HIV testing and cotrimoxazole preventive therapy (CPT) to TB patients registered under routine programme conditions (Harries et al. 2011). Using routinely collected data, and paying particular attention during this time to ensuring that mortality data were reliable and complete, the studies showed a significant reduction in death rates of between 20–25%. This evidence persuaded the Malawi Ministry of Health to support the scale up of HIV testing and CPT countrywide for TB patients, and activities started soon after completion of the research in 2003. Between 2002 and 2008, routinely collected data showed a significant increase in HIV testing amongst TB patients with the majority of HIV-positive patients being started on CPT. Treatment outcomes in new smear-positive pulmonary TB patients gradually improved, and by 2008, the global cure rate target of 85% was reached for the first time in 20 years since the start of the HIV/AIDS epidemic (Harries et al. 2011).
The same principles have been used to scale up and deliver antiretroviral therapy (ART) in Malawi. At the start of ART scale up in 2004, a strong monitoring and evaluation system was set up to document socio-demographic and clinical details of patients enrolled for therapy, their treatment regimens and their treatment outcomes. These data are used for quarterly cohort reporting of cases and treatment outcomes, with data always checked during supervision and then collated every three months for national reporting. By 31st December 2012, there were data from 651 static ART sites in the country documenting the number of new patients initiated on ART in the three months from October to December 2012 (N = 24,168), the number ever started on ART since 2004 (N = 560,325), the number alive and retained in care (N = 404,905) and the numbers dead, lost-to-follow-up and stopped treatment (Government of Malawi 2012). These data can be further dissected and analysed to evaluate programme performance. For example, over the last 8 years, the proportion of patients in WHO Clinical Stage 4 starting ART every quarter has declined from 25% to 2.7% with deaths in the first three months of therapy decreasing from 12% to about 3%, indicating earlier access of patients to therapy and consequent better outcomes.
In a similar vein to the TB programme, operational research was carried out during the early years of ART scale up in Malawi using routine programme data to inform about important issues such as who was accessing ART, reasons for loss to follow-up, use of adjunctive CPT to reduce early death after starting ART and impact of ART in decreasing mortality at the population level (Harries et al. 2012). The main source of these data was ART patient treatment cards and registers, but other sources were also used either from within the health sector (hospital inventories of healthcare workers) or outside the health sector (the register of army deaths, traditional village registers, registers for coffin sales and registers for church funerals). In an ART programme such as Malawi, only a fraction of the routinely collected data is used for monitoring and evaluation purposes, while operational research by asking key questions about healthcare delivery can tap into this rich data source to provide important additional and specific information on the healthcare systems or on healthcare delivery. Most of the data in the operational research studies published between 2004 and 2010 would never have seen the light of day had it not been for research questions being generated and papers published.
The experience of using programme data to support the scale up of interventions for chronic communicable diseases such as TB and HIV/AIDS is also being used to inform the development of monitoring systems for non-communicable diseases (NCD). The successful use of cohort monitoring and reporting has been reported for hospital outpatients with diabetes mellitus in Malawi (Allain et al. 2011) and for patients with hypertension and diabetes mellitus in primary care clinics for Palestine refugees in Jordan (Khader et al. 2012a,b). In Jordan, these cohort data inform the health facility and NCD programme about case detection, disease burden, comorbid complications, treatment outcomes such as losses-to-follow-up and failure to attend clinic appointments.
Based on field experience of the value of routinely collected data in disease control programmes and with a view to expanding this paradigm on a more global scale, in March 2009 The Union and Medecins sans Frontieres (MSF-Luxembourg) developed a modular-based, product-orientated course to train people working in health in LMIC to use retrospectively routinely collected data for operational research. The courses are run over 8–12 months, with Module 1 (protocol development) leading to Module 2 (data collection and analysis) and to Module 3 (paper writing). Because of the tight time frame and ethics considerations, participants are usually only selected if routine data from government or NGO health facilities or health management information systems are already available for collection, cleaning and analysis.
By 1st May, 2014, 14 courses had been or are being run, with 164 participants enrolled from 57 countries in Africa (22), Asia (15), Europe (10), Oceania (8) and Latin America (2). Of 170 different projects undertaken (some participants take on two projects per course), 121 (71%) are related to TB and HIV/AIDS, but there is an increasing focus on other areas such as malaria, maternal and child health, malnutrition, smoking, non-communicable disease and the prevention and treatment of neglected tropical diseases. The research based on use of programme data is already having benefits for policy, practice and advocacy in diverse areas such as monitoring retention of patients on ART through pharmacy records, screening TB patients for diabetes mellitus, advocating for better tobacco control regulations, improving the care of victims of sexual violence and reducing maternal mortality.
Recent years have seen a growing interest in operational research as a means of supporting improvements in health in LMIC. However, despite this appreciation, many still regard operational research as the ‘poor cousin’ of the randomised trial, criticising such research that uses routine data as being potentially unreliable due to confounding, biases, missing variables and poor quality data (Ford & Maher 2013). There is definitely a need to improve quality of routinely collected data, although our collective, unpublished experience is that regular and systematic use of these data to answer relevant programme-related questions leads to better data being collected and recorded, resulting in a gradual win–win situation for both routine reporting and operational research. Programme managers may also see a number of potential benefits of routinely collected data being used for operational research as highlighted in Panel 1.
Panel 1. Potential benefits of using routine programme data for operational research in low- and middle-income countries
- Data for routine monitoring become more accurate and reliable which in turn increases the validity of the next generation of research studies
- Programme managers and staff who are unfamiliar with research rapidly appreciate that it need not be a parallel activity that competes or diverts resources from the core job of programme implementation
- Using routine data requires little or no additional human and/or material resources particularly when conducted within the routine system of care delivery
- Simple research methodologies are often used that can be learnt by programme staff with limited technical support
- Use of retrospective data is quick and takes less time for the generation of results compared with prospective studies or controlled trials
- Field workers who collect the data that are used for research become convinced that this work is valued and relevant to the programme
- Ethics approval for the use and reporting of routine data is less cumbersome and faster than ethics approval for studies involving human subjects
- Publications that arise from research conducted on programme data can serve as a useful tool for dissemination of operational knowledge: this improves programme credibility, is useful for advocacy and can attract additional funding.
There is a growing belief that all countries should be producers as well as consumers of research and that this production should not just be the prerogative of academic institutions, but should also be carried out by Ministries of Health and public health programmes, close to where health service delivery takes place (Zachariah et al. 2009). In this environment, a culture of enquiry should be promoted and the capacity to undertake high-quality research using routine data should be developed. There seems to be no lack of demand for these skills, as for every place on one of the operational courses offered by the Union and MSF, there are up to ten applicants. It is important to foster this enthusiasm and turn around the paradigm in LMICs so that in becoming ‘data rich and information rich’, health programme performance and patient outcomes can be improved for a wide range of diseases.