SEARCH

SEARCH BY CITATION

Keywords:

  • computers;
  • handheld;
  • malaria;
  • management information systems;
  • programme evaluation;
  • Tanzania
  • malaria;
  • Tanzanie;
  • évaluation des programmes;
  • systèmes d’information de la gestion;
  • ordinateurs;
  • portable
  • Malaria;
  • Tanzania;
  • evaluación de programa;
  • manejo de sistemas de información;
  • ordenadores de bolsillo

Summary

  1. Top of page
  2. SummaryAméliorer le système d’information sanitaire de routine dans les zones rurales dans le sud de la Tanzanie: succès, défis et leçons apprisesMejorando los sistemas de información sanitaria rutinarios en zonas rurales del sur de Tanzania: éxitos, retos y lecciones aprendidas
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Objective  To describe and evaluate the use of handheld computers for the management of Health Management Information System data.

Methods  Electronic data capture took place in 11 sentinel health centres in rural southern Tanzania. Information from children attending the outpatient department (OPD) and the Expanded Program on Immunization vaccination clinic was captured by trained local school-leavers, supported by monthly supervision visits. Clinical data included malaria blood slides and haemoglobin colour scale results. Quality of captured data was assessed using double data entry. Malaria blood slide results from health centre laboratories were compared to those from the study’s quality control laboratory.

Results  The system took 5 months to implement, and few staffings or logistical problems were encountered. Over the following 12 months (April 2006–March 2007), 7056 attendances were recorded in 9880 infants aged 2–11 months, 50% with clinical malaria. Monthly supervision visits highlighted incomplete recording of information between OPD and laboratory records, where on average 40% of laboratory visits were missing the record of their corresponding OPD visit. Quality of microscopy from health facility laboratories was lower overall than that from the quality assurance laboratory.

Conclusions  Electronic capture of HMIS data was rapidly and successfully implemented in this resource-poor setting. Electronic capture alone did not resolve issues of data completeness, accuracy and reliability, which are essential for management, monitoring and evaluation; suggestions to monitor and improve data quality are made.

Améliorer le système d’information sanitaire de routine dans les zones rurales dans le sud de la Tanzanie: succès, défis et leçons apprises

Objectif:  Décrire et évaluer l’utilisation d’ordinateurs portables pour la gestion des données du système d’information de la gestion de la santé (SIGS).

Méthodes:  La saisie électronique des données a eu lieu dans 11 centres de santé sentinelles dans les zones rurales dans le sud de la Tanzanie. L’information sur les enfants fréquentant le département des patients ambulatoires (DPA) et la clinique du Programme Elargi de Vaccination a été capturée par des élèves locaux formés, soutenus par des visites de supervision mensuelles. Les données cliniques comprenaient les lames de frottis sanguin pour la malaria et les résultats d’échelle de couleur de l’hémoglobine. La qualité des données saisies a étéévaluée en utilisant une double saisie des données. Les résultats des frottis sanguins pour la malaria dans les laboratoires du centre de santé ont été comparés à ceux du laboratoire de contrôle de qualité de l’étude.

Résultats: Le système a requis cinq mois pour sa mise en œuvre et quelques problèmes logistiques et de personnel ont été rencontrés. Au cours des 12 mois suivants (avril 2006 à mars 2007), 7056 visites ont été enregistrées chez 9880 nourrissons âgés de 2 à 11 mois, 50% atteints de malaria clinique. Les visites de supervision mensuelles ont révélé des enregistrements incomplets de l’information entre les registres du DPA et ceux du laboratoire, où en moyenne 40% des visites de laboratoire ne comportaient pas un record de leur visite DPA correspondante. La qualité de la microscopie des laboratoires de centre de santéétait globalement plus faible que celle du laboratoire d’assurance qualité.

Conclusions:  La saisie électronique des données SIGS a été rapidement implémentée et avec succès dans ce contexte à ressources limitées. La saisie électronique seule n’a pas résolu les questions de complétude des données, la précision et la fiabilité, qui sont essentielles pour la gestion, le suivi et l’évaluation. Des suggestions pour surveiller et améliorer la qualité des données sont procurées.

Mejorando los sistemas de información sanitaria rutinarios en zonas rurales del sur de Tanzania: éxitos, retos y lecciones aprendidas

Objetivo:  Describir y evaluar el uso de ordenadores de bolsillo para el manejo de datos gestión de los sistemas de información sanitaria HMIS.

Métodos:  La captura de datos electrónicos se realizó en 11 centros sanitarios centinela, en zonas rurales del sur de Tanzania. Se obtuvo información de los niños que se presentaban en las consultas externas (CE) y en el centro de vacunación del Programa Ampliado de Inmunizaciones. Los datos fueron introducidos por personal local entrenado para ello y apoyado por visitas mensuales de supervisión. Los datos clínicos, incluían los resultados de las láminas para malaria y de la escala de color de hemoglobina. La calidad de los datos capturados se evaluó mediante la entrada doble de datos. Los resultados de las láminas para malaria provenientes de los laboratorios de los centros sanitarios se compararon con aquellas leídas en el laboratorio de control de calidad del estudio.

Resultados:  Se tardó cinco meses en implementar el sistema y se encontraron algunos problemas logísticos y de personal. Durante los siguientes 12 meses (Abril 2006-Marzo 2007), se registraron 7056 visitas de 9880 niños con edades entre los 2-11 meses, un 50% con malaria clínica. Las visitas de supervisión mensual dejaron en evidencia un registro incompleto de la información entre las CE y los datos de laboratorio, de manera que en promedio a un 40% de las visitas de laboratorio le faltaba algún dato de la visita a CE correspondiente. La calidad de la microscopía de los laboratorios de los centros sanitarios era en general menor que la del laboratorio de control de calidad.

Conclusiones:  La captura electrónica de datos HMIS fue implementada rápida y exitosamente en este emplazamiento con recursos limitados. La captura electrónica por si sola no resolvió los problemas de datos incompletos, precisión y fiabilidad, que son esenciales para el manejo, monitorización y evaluación. Se sugiere monitorizar y mejorar la calidad de los datos.


Introduction

  1. Top of page
  2. SummaryAméliorer le système d’information sanitaire de routine dans les zones rurales dans le sud de la Tanzanie: succès, défis et leçons apprisesMejorando los sistemas de información sanitaria rutinarios en zonas rurales del sur de Tanzania: éxitos, retos y lecciones aprendidas
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

The importance of well-functioning Health Management Information System (HMIS) has been emphasised by WHO (2008). It enables adequate planning by health managers and provides a framework within which the performance of the health system and that of intervention programmes, including malaria control programmes, may be evaluated. In many countries in Africa, including Tanzania, the HMIS depends on manual data collection; a time-consuming method that overburdens health staff, is prone to error and hampers summary and rapid retrieval of information. These shortcomings may produce unreliable information that can lead to planning inefficiency, resource wastage and inappropriate resource allocation at regional or national levels (Azubuike & Ehiri 1999). In comparison with manual data collection, the use of an electronic HMIS is more accurate, less time-consuming and more acceptable to health staff (Ndira et al. 2008).

In this paper, we describe the experience in using handheld computers [personal digital assistants (PDAs)] to collect routine information from the outpatient department (OPD), laboratory and reproductive and child health (RCH) vaccination clinics of 11 sentinel health centres in five rural districts of southern Tanzania (Figure 1). We (i) detail the implementation of this electronic data capture; (ii) document the training and supervision of PDA operators and health centre staff; (iii) describe temporal trends in data completeness and timeliness; and (iv) to assess the feasibility of using HMIS laboratory data to provide reliable estimates of parasitologically confirmed malaria cases, and report the quality of malaria blood slide microscopy in the health centre laboratories in comparison with that of a quality assurance (QA) laboratory.

image

Figure 1.  Study area in southern Tanzania (Lindi Rural, Ruangwa, Nachingwea, Tandahimba and Newala districts), showing sentinel health centres included in the morbidity monitoring study.

Download figure to PowerPoint

Methods

  1. Top of page
  2. SummaryAméliorer le système d’information sanitaire de routine dans les zones rurales dans le sud de la Tanzanie: succès, défis et leçons apprisesMejorando los sistemas de información sanitaria rutinarios en zonas rurales del sur de Tanzania: éxitos, retos y lecciones aprendidas
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Study area and the existing HMIS

The enhanced HMIS data collection and morbidity monitoring described took place in the context of a cluster-randomized control trial of Intermittent Preventive Treatment for malaria in infants (IPTi), assessed using household surveys and a community census, and health centre surveys (Armstrong Schellenberg et al. 2010). Additionally, in a morbidity monitoring substudy, HMIS data from 11 sentinel health centres were used to document feasibility of the delivery of IPTi through routine contacts with the health service and evaluate effectiveness of IPTi.

The area has been described in detail (Schellenberg Armstrong et al. 2008). Briefly, malaria is endemic, with transmission occurring throughout the year and is the most common primary cause for health centre or hospital admission among those aged under 5 years. The health system in the study area consists of six hospitals, 13 health centres and 149 dispensaries. Health centres on the whole have laboratory diagnostic facilities and are staffed by trained nurses, clinical officers and assistant medical officers. Health planning in Tanzania is decentralised to the districts, and the District Health Management Team, headed by the District Medical Officer, is empowered to plan and allocate resources. The Tanzanian HMIS consists of 12 registers, including one for OPD attendees and another for infants attending the Expanded Program on Immunization (EPI) at RCH clinics. The OPD register records date of consultation, diagnoses and treatments given, but not symptoms, while the EPI register records dates and doses of routine immunisations. Separate registers for different laboratory investigations are used, but these are not included in the routine HMIS. At the RCH clinics, when infants attend for routine immunisations or weighing sessions, information on immunisations, body weight and growth are recorded on the child’s health card, which contains some demographic information documented at birth. Mothers or carers are expected to bring the child’s health card to all RCH, immunisation and OPD appointments and visits.

Enhancement of the HMIS

As an enhancement to the existing HMIS system, we established a PDA-based system in all health centres with on site functional microscopes in the study area (11/13 centres). Implementation of this morbidity monitoring study was staggered, beginning in 2005, and data collection ran until January 2008. A period of full functionality was defined as 12 months between April 2006 and March 2007, and results refer to this period.

Identification codes

Children younger than 5 years were registered when they visited the selected health centre for sickness or for routine RCH visits, including EPI immunisation visits. Each registered child was given a 12-digit identity code (IDC) during registration to enable linkage of an individual between the separate registers (OPD attendance, laboratory results and EPI vaccinations), and between subsequent visits made by the same child. The IDC was created by combining the first two letters of the child’s first name, the first letter of the mother’s and father’s first names, the month and year of the child’s date of birth, the first three letters of the area of residence and finally either number 1 (singleton or first twin), 2 (second twin or triplet) or 3 (third triplet). The generated IDC was written on the child’s RCH card, and an extra column next to the child’s name was added to the end of each page in the OPD, laboratory and immunisation registers, in which the IDC could be added at each clinic visit. During monthly supervision visits, IDCs electronically entered were compared with the details on paper records to ensure accuracy of data entry.

Personal digital assistant operators and software

Personal digital assistant operators were selected by posting advertisements in villages surrounding the health centres. Applications from secondary school-leavers, who were residents of neighbouring villages and experienced in using mobile telephones for text messaging, were considered. In each health centre, three candidates were shortlisted and underwent a 3-day training course led by the study data manager (KS) and data entry clerk, which included a data entry exercise using dummy-filled RCH cards. Based on their performance during training, one of the three candidates was selected to be the PDA operator for the health centre. Supervision and support of PDA operators continued monthly throughout the study when the study data manager (KS) or data entry clerk observed the PDA operators while capturing data in their respective health centres. Summary reports of time and date patterns of data entry were automatically generated during PDA use, and results of these reports were discussed with PDA operators during supervision visits to emphasise the importance of timely data capture and to encourage good data entry practices.

Full details of PDA hardware and software are provided elsewhere (Shirima et al. 2007). In brief, we use Palm m130 (Palm Inc, Sunnyvale, USA) units with a coloured touch-sensitive screen, powered by a rechargeable internal lithium ion battery that lasted for at least 2 working days (recharged by solar power). Pendragon Forms 4.0 (Pendragon Software Corporation, Libertyville, IL, USA) software was used, which allowed logical checks to be attached to questions, and the use automated of skip patterns. Drop down menus of the most frequently used treatment and diagnosis categories were created for use by the PDA operators. Data from each register were entered separately, but linked by the unique IDC. Data were downloaded from the PDA to an IBM ThinkPad laptop (IBM Corporation, Armonk, NY, USA) using synchronising software provided with the PDAs (Palm TM Desktop for Microsoft Windows 4.0.1) and subsequently backed up onto CDs.

Health centre staff familiarisation, monthly health centre visits and health centre reports

Staff at health centres received a week’s on-the-job training at their health centre, conducted by the study’s medical officer (WM), data manager (KS), data entry clerk and an experienced laboratory assistant. Health centre clinicians received training for completion of OPD register and re-generation of IDCs. Local laboratory personnel received training for malaria blood slide staining using Giemsa stain (Warhurst & Williams 1996), classification of parasite positive blood slides [≥1 parasite/200 white blood cells (WBCs)] (Warhurst & Williams 1996; O’meara et al. 2006), haemoglobin analysis using the haemoglobin colour scale (Gosling et al. 2000), storage of malaria blood slides, as well as recording of malaria blood slide and haemoglobin results in the register.

Monthly supervision visits to all study sites were carried out by the study co-coordinator (WM), data manager (KS) and laboratory assistant. At this time, data from PDAs were synchronised with the study’s dedicated laptop and backed up onto CDs, stored blood slides were collected, and laboratory and other project supplies were delivered. As described previously, during supervision visits IDCs on PDAs were cross checked with details recorded in HMIS ledgers, and patterns of data entry were discussed with PDA operators. Visits were also an opportunity to discuss the issues of data completeness and quality with health centre staff. The basis for these discussions was the automatically generated monthly report (Figure 2). These compared numbers of OPD attendances and EPI vaccinations to those recorded during the previous month; highlighted whether all OPD attendances had corresponding laboratory records; and listed the top 10 diagnoses made at the centre in the past month, which previously had been prepared manually.

image

Figure 2.  Example of an electronically generated monthly health centre summary report.

Download figure to PowerPoint

Double data entry and laboratory blood slide microscopy quality control

Each PDA operator was working full-time in one of the 11 health centres. They were supported by the study’s data entry clerk who visited each health centre monthly throughout the study and completed double data entry for all data from OPD attendance and laboratory registers. Discrepant entries between those of the PDA operator and the study’s data entry clerk were identified, and a third independent person identified the definitive data, by referring back to the original paper registers.

Blood slides for malaria parasites were initially read on site in the health centre laboratory, except for one health centre (centre D) where the qualified laboratory personnel left shortly after introduction of our system. In this health centre, blood slides were stained and stored, without reading. Blood slides from all 11 centres were then read twice by two independent slide readers in the morbidity monitoring study’s QA laboratory. Parasite densities per 200 WBCs were counted, and if results were discrepant, a third slide reading was performed. Results were defined as discrepant if one was positive and one negative, or if the ratio of the difference in parasite density between two positive readings was >1.5 or <0.67. When a third reading was necessary, because of density discordance, the geometric mean of the three readings was used as the definitive blood slide result (Alonso et al. 1994; O’meara et al. 2005).

Statistical analysis

Data from PDAs were exported from Pendragon Forms software to Microsoft Access for data management and subsequently to Stata for statistical analysis. Stata 10.1 (Stata Corp., College Station, TX, USA) was used to calculate proportions, incidence rates, confidence intervals and relevant P values in infants aged 2–11 months presenting between 1 April 2006 and 31 March 2007. Blood slide malaria results between the health centre laboratories and the QA laboratory (gold standard) were investigated using calculated sensitivity (true positives/true positives + false negative) and specificity (true negatives/true positives + false positives), percentage positive for each laboratory, percentage agreement, as well as Cohen’s Kappa score (Cohen 1960), and degree of agreement above that expected by chance alone followed the classification suggested by Landis and Koch (1977).

Ethical clearance

The study was undertaken within the framework of the assessment of the community effectiveness of IPTi, part of the IPTi Consortium (Schellenberg et al. 2006). We received ethical approval from the following local and national institutional review boards: Ifakara Health Institute, National Tanzania Medical Research Coordinating Committee, London School of Hygiene & Tropical Medicine, and the Ethics Commission of the Cantons of Basel-Stadt and Basel-Land, Switzerland. The permission to execute the project was sought and obtained from the Tanzania Ministry of Health and Social Welfare and the respective district councils of the participating districts.

Results

  1. Top of page
  2. SummaryAméliorer le système d’information sanitaire de routine dans les zones rurales dans le sud de la Tanzanie: succès, défis et leçons apprisesMejorando los sistemas de información sanitaria rutinarios en zonas rurales del sur de Tanzania: éxitos, retos y lecciones aprendidas
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Implementation

It took 5 months to establish the morbidity monitoring framework in all 11 health centres in the study area. Few logistical problems were encountered during implementation, mainly as this method and technology had been previously used in a household survey in Tanzania (Shirima et al. 2007). For example, solar energy was used to power PDAs, reducing the impact of power cuts to the health centres, and no serious problems with PDA hardware or software were experienced. An initial equipment outlay of US $480 per health centre was required, including about US $100 per PDA. Running costs were estimated at approximately US $7800 per health centre per year, with over 60% of these costs due to personnel, principally supervision personnel. All but one of the 11 PDA operators who completed training remained with the project for the duration of the study. A new PDA operator was recruited, trained 6 months and allowed to replace the one who was performing poorly.

The use of IDCs was successful at uniquely identifying infants, with 0.08% duplication on average across all 11 health centres in infants aged 2–11 months (attending between April 2006 and March 2007). Although the IDC was constructed with the aim of being easily re-generated for occasions when the child health card was forgotten, in practice IDCs were rarely re-generated by health staff, and parents had difficulty recalling the information needed for the IDC.

Data entry

The drop down menus for treatment and diagnosis details within the PDA software were not well-used by PDA operators. Instead treatment and diagnosis details were entered manually, resulting in many spelling variations and some incomprehensible entries. For example, prior to data cleaning, 1889 categories were included in the treatment variable, with 2053 in the diagnosis variable, which was reduced to 109 individual (i.e. ungrouped) categories for treatment, and 222 for diagnosis after data cleaning and management were completed.

Double data entry was performed on all records from the OPD and laboratory registers, and discrepant entries were corrected. Of the entries, 19% from the laboratory register were discrepant, with errors in the spelling of the child’s name (54%) and failure to count WBCs (33%) being the most frequent. Almost 73% of entries from the OPD attendance register were discrepant. Again most were attributable to errors in the spelling of the child’s name (23%), spelling of treatment (17%) and spelling of diagnosis (10%) variables.

Data completeness and timeliness

During the year April 2006 to March 2007, a total of 7056 OPD attendances, and 5246 blood slides were recorded from 9880 infants aged 2–11 months, with an incidence of OPD attendance of about 2/infant/year. A malaria diagnosis was recorded in 50% of OPD visits, with an incidence of about 1/infant/year (Table 1).1

Table 1.   Distribution of outpatient department (OPD) visits, malarial diagnoses and blood slide readings recorded April 2006–March 2007 in infants aged 2–11 months, by health centre
Health centreNumber of infants*OPD visitsMalaria (clinical diagnosis)OPD visits with blood slide availableMalaria diagnoses with blood slide available
Number of visitsPYAR†IncidenceNumber of visitsPYARIncidenceNumber%Number%
  1. PYAR, person years at risk.

  2. *Infants aged 2–11 months attending between April 2006 and March 2007.

  3. †PYAR – Infants entering on latest of registration date, turning 2 months, starting of 12-month follow-up period (1st April 2006), and exiting on earliest of becoming 12 months of age or end of 12-month follow-up period (31st March 2007).

  4. OPD visits include repeat attendances, malaria cases also include repeat episodes. PYAR is lower for malaria cases as 28-day follow-up was removed after each malaria diagnosis to avoid double-counting the same malaria episode as two separate episodes.

  5. ‡Blood slides from health centre D were not read in the health centre.

A8406292832.223332511.3229747.220060.1
B4867021484.733031222.4925536.317557.8
C8693953231.221853110.5919850.19350.3
D‡5193012261.331282170.593311.01814.1
E11988593982.164423821.1671282.941293.2
F16108925561.604965350.9321724.316833.9
G8626583361.963853181.2123235.319650.9
H9995483341.643683271.1339371.729279.3
I9517393242.282823090.9133645.57928.0
J8685893471.702613140.8326745.316262.1
K6787442363.163502091.6716121.610329.4
Total9880705635122.01353332951.07310143.9189853.7

Feedback to health centre staff during supervision visits emphasised that data completeness between OPD and laboratory ledgers was important. The number of OPD attendees without laboratory records was monitored and changed little over time remaining at about 44% (average for all 11 centres). A more appropriate measure of data completeness may be the number of malaria diagnoses without a laboratory result, although it is important to note that MoH guidelines at the time of this study did not include parasitological confirmation of clinical malaria cases before treatment, as is the case now (WHO 2010). Of 3553 recorded visits with a clinical malaria diagnosis, 54% had a corresponding blood slide result available from the health centre laboratory. Availability of blood slide results showed wide variability by health centre (range 14–93%) (Table 1).

An additional measure of data completeness, not assessed during monthly supervision visits, is the proportion of health centre blood slide results without a corresponding OPD record – termed orphan slides. On average, 41% of blood slide readings were lacking a corresponding OPD record suggesting incomplete data recording, which showed variability by health centre (59–24% orphan slides) (Table 2).

Table 2.   Proportion of blood slide readings without the corresponding outpatient department (OPD) visit recorded April 2006–March 2007, by health centre
Health centreNumber of OPD visitsNumber of blood slide readingsBS readings without corresponding OPD visit recorded
Number%
  1. BS, blood slide.

A62960030350.5
B7023368124.1
C39537818047.6
D301804758.8
E859114943738.0
F89253131459.1
G65836513336.4
H54864325038.9
I73945912326.8
J58942415737.0
K74428112042.7
Total70565246214540.9

Feedback to PDA operators from the monitoring of time and motion activity of the PDA operator by study staff during supervision visits was useful and showed improvement in timeliness of data entry, for example, using an average across all 11 centres, the proportion of records entered on the same day as attendance at the health centre improved from 67% to 85%. Information on timeliness of data transfer from the periphery to the district was not available. Additionally, this study did not explicitly monitor the use of electronically captured HMIS data to modify health care practices. The only proxy measure of change in practices available – the number of oral polio vaccine and DPT vaccines administered at 2 months (which should be equal to one another, in line with EPI scheduling guidelines) – changed by only 1%, albeit from a high baseline, from 94% to 95%.

Quality of malaria microscopy

The sample size available for investigating the agreement between the blood slide results from the health centre and the QA laboratory was 5166 (Table 3). In comparison with the QA laboratory, the health facility blood slide results had a sensitivity of 89.6% and a specificity of 77.3% overall, though this varied by health centre with sensitivity ranging from 53.5% to 99.3% and specificity from 21.9% to 95.3%. Overall, a kappa score of 0.572 was documented, suggesting moderate agreement above that expected by chance alone between the health centre laboratory blood slide results and those from the QA laboratory, although this ranged between 0.865, suggesting almost perfect agreement above that expected by chance alone, and 0.098, suggesting slight agreement above that expected by chance alone (Table 3).

Table 3.   Kappa scores of agreement between blood slide results from health centre laboratories and the morbidity monitoring laboratory
Health centreHealth centre laboratoryMorbidity monitoring laboratorySensitivity & SpecificityAgreement between laboratories*
n blood slidesn positive%n blood slidesn positive%SensitivitySpecificity% agreementKappa scoreAgreement above that expected by chance alone†
  1. *Based on n = 5166.

  2. †Classification according to Landis & Koch (1977)).

  3. ‡Blood slides from health centre D were not read in the health centre.

A60032654.359823138.696.572.581.80.642Substantial
B33622065.53349929.670.640.051.20.132Slight
C37813335.23708422.792.881.884.30.626Substantial
D‡80781114.1
E114956649.3114444438.896.880.787.00.739Substantial
F5319117.14875010.396.091.391.80.662Substantial
G36530182.53638022.097.521.938.60.098Slight
H64317126.664314121.999.393.895.00.865Almost perfect
I4595612.24534710.478.795.393.60.683Substantial
J42412629.74169923.853.577.671.90.286Fair
K28111340.22807928.274.773.673.90.429Moderate
Total5246210340.15166136526.489.677.380.60.572Moderate

Discussion

  1. Top of page
  2. SummaryAméliorer le système d’information sanitaire de routine dans les zones rurales dans le sud de la Tanzanie: succès, défis et leçons apprisesMejorando los sistemas de información sanitaria rutinarios en zonas rurales del sur de Tanzania: éxitos, retos y lecciones aprendidas
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

We have described the implementation of electronic data capture from the HMIS in rural southern Tanzanian. Data from 11 sentinel health centres were collected by minimally trained local secondary school-leavers who were externally supervised monthly. Few serious logistic or staff problems were encountered, and use of this technology resulted in easily produced monthly summary reports of patient information and morbidity trends.

Challenges with using this method include the creation and use of an individually unique identification code, which can be re-generated. The creation of identification codes is challenging in settings like Tanzania, which often lack national identity numbers, and where birth dates may be inaccurate, and name spellings flexible. The IDC used in this study reliably, though not perfectly, produced a means for uniquely identifying individuals, but proved to be re-generated much less frequently than anticipated. Additional program design challenges include the need for more robust data entry for variables such as treatment and diagnosis, which could be addressed though piloting to determine frequently used categories, and PDA operator training to accurately allocate treatments and diagnoses to appropriate categories. The nature of errors detected during double data entry, principally spelling errors in names, and diagnosis and treatment variables, also highlights the necessity of unique IDCs and drop down menus for variables that may be easily misspelled. Improvements in technology since 2005 are likely to go some way to helping resolve such programming issues.

The use of HMIS data for monitoring programme effectiveness and evaluating the effectiveness and safety of interventions has been much emphasised (WHO Roll Back Malaria 2000, WHO Roll Back Malaria Monitoring and Evaluation Reference Group 2005). The implementation and running of this study highlighted the importance of quality control, monitoring and feedback. Results from the double data entry of HMIS data (OPD and laboratory registers) emphasise the usefulness of this technique for detecting errors (World Health Organization 2008). Results also suggest that complete data recording between different registers, and high agreement between health centre and control laboratories may be used as quality control indicators of health centre functioning in settings similar to those seen in Tanzania.

Over 40% of health centre laboratory results from these rural health centres were not recorded in the OPD register (orphan slides). A total of 35% of these orphan blood slide results came from the two largest and busiest centres, which saw almost 25% of OPD visits. One health centre, with few OPD visits, but very high levels of orphan slides, experienced serious laboratory staffing difficulties. We hypothesise that if this measure evaluating complete data recording had been explicitly monitored during data collection and discussed during monthly feedback with health centre staff as with other data completeness and timeliness variables, outcomes may have improved following constructive feedback to and support for staff on the importance of accurate and complete HMIS data collection.

National Tanzanian guidelines for diagnosis and treatment of malaria from 2006 recommend the performance of a malaria test in facilities with laboratory services, while the algorithm for a child aged <5 years presenting with fever (based on WHO’s Integrated Management of Childhood Illness) recommend treatment with the first-line anti-malarial drug (National Malaria Control Programme 2006). Therefore, the observation that over half of clinical malaria diagnoses had a corresponding blood slide result available from the health centre laboratory, but that this showed wide variability, is not unexpected. The revised WHO global malaria programme treatment guidelines recommend parasitological diagnosis before treatment of all malaria cases, regardless of age (World Health Organization 2010). This clear recommendation is likely to facilitate the completeness of laboratory blood slide results in consolidated health information systems such as the one described here.

Quality control results for malaria blood slide readings showed that there was wide variability in sensitivity and specificity of health centre laboratory microscopy results in comparison with expert microscopy in the QA laboratory, and that levels of false positives (identified as parasite positive on slides in the health centre, but negative by expert microscopy – specificity) were higher than false negatives (sensitivity). There was generally only moderate agreement above that expected by chance alone between the results obtained in sentinel health centre laboratories and those from the QA laboratory, suggesting that health centre classification of parasitologically confirmed malaria cases was not reliable. This demonstrates that, although a few health centre laboratories performed well, good-quality blood slide microscopy at peripheral health facilities cannot be guaranteed even with considerable investment in training and supervision. The introduction of Rapid Diagnostic Tests should help to improve the availability of reliable parasitological pre-treatment malaria diagnosis, in line with new WHO guidelines (2010).

The use of PDAs to enhance HMIS data capture in this study occurred in the context of monitoring IPTi implementation and evaluating the effectiveness of this malaria control intervention. Electronic HMIS data capture was not introduced to health centre staff in peripheral facilities of southern Tanzania, and this study did not explicitly monitor interaction of external PDA operators with health centre staff, sustainability of this form of data capture in this setting, or transferability of training applied to PDA operators for use by health centre staff. Nonetheless, anecdotal reports from project managers (WM, MC and KS) suggests that interactions were positive: PDA operators assisted health staff in various duties including weighing children, plotting their growth on the child health cards; and some health centre staff learned to operate PDAs out of their own interest, with very informal training. Project managers are of the opinion that training would be easily transferrable to health centre staff, and that introducing PDA-based data capture may be easier for health workers as they are more likely to understand medical terminology used, such as diagnostic and drug information. Acceptance of PDAs has also been monitored and documented in the same geographical area in the context of large household surveys (Shirima et al. 2007).

Electronic capture of data by methods such as the use of PDAs may be efficient, provide data in a format that is flexible and easy to summarise, and may help to highlight incompleteness of data through regular monitoring. However, this study shows that supervision, and monitoring of data quality and completeness are essential. Therefore, although this method of HMIS data collection has the potential to minimise health staff data entry and manipulation workload, incorporation of supervision of PDA-based data capture by health centre staff within current routine supervision is unlikely to function efficiently. Therefore, although the cost of PDA units themselves is reasonable, at about US $100 per unit (2005 price) – and likely to be lower still in 2011, with improved processing power, personnel costs and costs of external supervision in particular, it is likely to make this system unaffordable and impractical on a large scale.

In addition, use of this technology alone will not resolve underlying issues of quality and completeness of data, which are crucial to produce information that can be used for monitoring and evaluation of health systems, policies and interventions (WHO Roll Back Malaria 2000, Rowe et al. 2009), and can in turn influence resource allocation and policy. Improving data quality and completeness requires evaluation of HMIS performance, and strengthening of processes. Toolkits to this end exist, such as performance of routine information system management (PRISM) developed by MEASURE Evaluate/USAID (Aqil et al. 2009). PRISM consists of a conceptual framework and tools that focus on behavioural, technical and organisational determinants and has been used in Uganda (Aqil 2004) and Cote d’Ivoire (Gnassou et al. 2008) – a similar approach may be useful in Tanzania.

Footnotes
  • 1

    Note more than one OPD attendance and malaria diagnosis may be included for each infant over the 12 months of this study.

Acknowledgements

  1. Top of page
  2. SummaryAméliorer le système d’information sanitaire de routine dans les zones rurales dans le sud de la Tanzanie: succès, défis et leçons apprisesMejorando los sistemas de información sanitaria rutinarios en zonas rurales del sur de Tanzania: éxitos, retos y lecciones aprendidas
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

We thank health staff in the sentinel health centres, PDA operators and Regional and District Health Management Teams of the participating regions and districts. We appreciate the cooperation offered by the Tanzania Ministry of Health and Social Welfare, and the administrative support given by the Ifakara Health Institute. This study was financially supported by the Bill and Melinda Gates Foundation through the IPTi consortium. JAS was supported by funding from Gates Malaria Partnership, and JAS and BW are members of the MRC Tropical Epidemiology Group at the London School of Hygiene & Tropical Medicine. Finally, our thanks are also due to the anonymous TMIH reviewers for their helpful comments.

References

  1. Top of page
  2. SummaryAméliorer le système d’information sanitaire de routine dans les zones rurales dans le sud de la Tanzanie: succès, défis et leçons apprisesMejorando los sistemas de información sanitaria rutinarios en zonas rurales del sur de Tanzania: éxitos, retos y lecciones aprendidas
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  • Alonso PL, Smith T, Schellenberg J et al. (1994) Randomized trial of efficacy of Spf66 vaccine against plasmodium-falciparum malaria in children in Southern Tanzania. Lancet 344, 11751181.
  • Armstrong Schellenberg JR, Mrisho M, Manzi F et al . (2008) Health and survival of young children in Southern Tanzania. BMC Public Health 8, 194.
  • Aqil A (2004) HMIS and EMIS Situation Analysis, Uganda. UPHOLD Project, John Snow Inc, Kampala.
  • Aqil A, Lippeveld T & Hozumi D (2009) PRISM framework: a paradigm shift for designing, strengthening and evaluating routine health information systems. Health Policy and Planning 24, 217228.
  • Armstrong Schellenberg JRM, Shirima K, Maokola W et al. (2010) Community effectiveness of Intermittent Preventive Treatment for infants (IPTi) in rural southern Tanzania. American Journal of Tropical Medicine and Hygiene 82, 772781.
  • Azubuike MC & Ehiri JE (1999) Health information systems in developing countries: benefits, problems, and prospects. The Journal of the Royal Society for the Promotion of Health 119, 180184.
  • Cohen J (1960) A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20, 213220.
  • Gnassou L, Aqil A, Moussa T, Kofi D & Paul J (2008) HMIS Evaluation Report. HIS Department, Ministry of Health, Cote d’Ivoire.
  • Gosling R, Walraven G, Manneh F, Bailey B & Lewis SM (2000) Training health workers to assess anaemia with the WHO haemoglobin colour scale. Tropical Medicine & International Health 5, 214221.
  • Landis J & Koch G (1977) The Measurement of observer agreement for categorical data. Biometrics 33, 159174.
  • National Malaria Control Programme (2006) National Guidelines for Diagnosis and Treatment of Malaria United Republic of Tanzania Ministry of Health and Social Welfare. Tanzania Ministry of Health, Dar es Salaam.
  • Ndira SP, Rosenberger KD & Wetter T (2008) Data quality of and staff satisfaction with an electronic health record system in a developing country (Uganda) – a qualitative and quantitative comparative study assessment. Methods of Information in Medicine 47, 489498.
  • O’meara WP, Mckenzie FE, Magill AJ et al. (2005) Sources of variability in determining malaria parasite density by microscopy. American Journal of Tropical Medicine and Hygiene 73, 593598.
  • O’meara WP, Remich S, Ogutu B et al. (2006) Systematic comparison of two methods to measure parasite density from malaria blood smears. Parasitology Research 99, 500504.
  • Rowe A, Kachur SP, Yoon S, Lynch M, Slutsker L & Steketee R (2009) Caution is required when using health facility-based data to evaluate the health impact of malaria control efforts in Africa. Malaria Journal 8, 209.
  • Schellenberg D, Cisse B & Menendez C (2006) The IPTi Consortium: research for policy and action. Trends in Parasitology 22, 296300.
  • Shirima K, Mukasa O, Schellenberg J et al. (2007) The use of personal digital assistants for data entry at the point of collection in a large household survey in southern Tanzania. Emerging Themes in Epidemiology 4, 512.
  • Warhurst DC & Williams JE (1996) Laboratory diagnosis of malaria. Journal of Clinical Pathology 49, 533538.
  • WHO Roll Back Malaria (2000) Framework for Monitoring Progress and Evaluation Outcomes and Impact. World Health Organization, Geneva.
  • WHO Roll Back Malaria Monitoring and Evaluation Reference Group (2005) Building Capacity in Monitoring and Evaluating Roll Back Malaria in Africa: A Conceptual Framework for the Roll Back Malaria Partnership. WHO Roll Back Malaria Monitoring and Evaluation Reference Group (MERG), Geneva, Switzerland.
  • World Health Organization (2008) Toolkit on Monitoring Health Systems Strengthening Health Information Systems. World Health Organization, Geneva.
  • World Health Organization (2010) Guidelines for the Treatment of Malaria, 2nd edn. World Health Organisation, Geneva.