To obtain an overview of the quality of care for children in Indonesia, by assessing hospitals with a view to proceed to a quality improvement mechanism for child care.
To obtain an overview of the quality of care for children in Indonesia, by assessing hospitals with a view to proceed to a quality improvement mechanism for child care.
Stratified two-stage random sampling in six regions identified 18 hospitals (provinces Jambi, East Java, Central Kalimantan, South–East Sulawesi, East Nusa Tenggara, North Maluku). Three randomly selected hospitals in each province were visited by trained assessors who scored each assessed service (expressed as a percentage of achievement) and grouped into good (≥ 80%), requiring improvement (60–79%) and urgently requiring improvement (< 60%).
The overall median result score across all areas was 43% (IQR 28%–53%). Case management for common childhood illnesses had a median score of 37% (IQR18–43%), neonatal care 46% (IQR 26–57%) and patient monitoring 40% (IQR 30–50%), all indicating an urgent need for improvement. Qualitative data showed as main problems inadequate use of standard treatment guidelines, irrational prescribing of antibiotics, poor progress monitoring and poor supportive care.
We found serious shortcomings in the quality of hospital care for children. Finding and documenting those is the first step in a quality improvement process. Work is needed to start an improvement cycle for hospital care.
Obtenir un aperçu de la qualité des soins pour les enfants en Indonésie, en évaluant les hôpitaux en vue de procéder à un mécanisme d'amélioration de la qualité des soins aux enfants.
Un échantillonnage aléatoire stratifié à deux degrés dans six régions a identifié 18 hôpitaux (provinces de Jambi, est Java, Kalimantan Central, sud-est Sulawesi, est Nusa Tenggara, nord Maluku). Trois hôpitaux sélectionnés aléatoirement dans chaque province ont été visités par des évaluateurs qualifiés qui ont noté chaque service évalué (note exprimée en pourcentage de réalisation) et ont été regroupés en catégories telles que: bons (si > 80%), amélioration nécessaire (si 60–79%) et amélioration nécessaire d'urgence (si < 60%)
Le score médian global des résultats dans tous les domaines était de 43% (IQR: 28–53). La prise en charge des cas pour les maladies courantes de l'enfance avait un score moyen de 37% (IQR: 18–43), pour les soins néonataux 46% (IQR: 26–57) et pour le suivi des patients 40% (IQR: 30–50), tous indiquant un besoin urgent d'amélioration. Les données qualitatives ont révélé comme principaux problèmes, l'utilisation inadéquate des directives de traitement standard, la prescription irrationnelle d'antibiotiques, la mauvaise surveillance des progrès et de mauvais soins de soutien.
Nous avons constaté de sérieuses lacunes dans la qualité des soins hospitaliers pour les enfants. Rechercher et documenter celles-ci est la première étape dans un processus d'amélioration de la qualité. Du travail est nécessaire pour démarrer un cycle d'amélioration pour les soins hospitaliers.
Obtener una visión general de la calidad de los cuidados sanitarios en Indonesia, realizando una evaluación de los hospitales, con el objetivo de proceder con la implementación de un mecanismo de mejora en la calidad de los cuidados infantiles.
En un muestreo aleatorizado, estratificado en dos etapas, se identificaron 18 hospitales en six regiones (provincias Jambi, Java del Este, Kalimantan Central, el sudoeste de Sulawesi del sudeste, el este de Nusa Tenggara y el norte de Maluku). En cada provincia, unos asesores entrenados visitaron tres hospitales seleccionados al azar y puntuaron cada servicio evaluado (expresado como un porcentaje de logro) y se agruparon en buenos (≥ 80%), necesitando mejoras (60–79%) y necesitando mejoras urgentes (< 60%)
La media del puntaje total en todas las áreas era del 43% (IQR 28–53%). El manejo de casos para enfermedades comunes entre niños tenía un puntaje medio del 37% (IQR18–43%), cuidados neonatales del 46% (IQR 26–57%), y monitorización de pacientes del 40% (IQR 30–50%), todos indicando una necesidad urgente de mejora. Los datos cualitativos mostraban que los principales problemas se encontraban en el uso de guías de tratamiento estándares, en una prescripción irracional de antibióticos, un progreso insuficiente en la monitorización y unos cuidados de apoyo deficientes.
Encontramos deficiencias graves en la calidad de los cuidados hospitalarios para niños. Encontrarlos y documentarlos son el primer paso en el proceso de mejora. Se requiere trabajar para comenzar el ciclo de mejora en los cuidados hospitalarios.
Reducing child mortality is one of the Millennium Development Goals for 2015 (United Nations 2011). To achieve this, child health services need to be improved from the community level to first-level health facilities and to hospital care. The Integrated Management of Childhood Illnesses (IMCI) approach introduced comprehensive management of sick children in first-level health facilities in Indonesia in 1997. Children with severe conditions, who constitute around 10% of sick children, should be referred to facilities such as health centres with beds or district hospitals. As these children are the ones with the highest risk of dying, the quality of care at the referral health facility is critical for improving child survival (Duke et al. 2006a). Unfortunately, there is evidence that hospital care is often deficient in many countries (Nolan et al. 2001; Duke et al. 2006a,b).
To improve hospital care for children globally, WHO developed the pocket book of ‘Hospital care for children’ (WHO 2005). This is complemented by ancillary materials such as assessment tools, training CDs and documentation of the evidence base for the guidelines. To adapt this pocket book, the Ministry of Health of the Republic of Indonesia convened an expert working group led by the Indonesian Paediatric Society into the Indonesia version ‘Hospital Care for Children: Guidelines for First Referral Level Hospitals in District or Municipality’, which was launched in 2009 (WHO 2009). In parallel, the assessment tool and other supportive materials were adapted. Following an orientation on a hospital improvement framework developed by the WHO regional office (WHO 2009), it was decided that a first step of the hospital care improvement approach should be a hospital assessment covering a wide geographical and representative range of hospitals, to provide a basis for further improvement efforts.
We divided the whole of Indonesia into six geographical regions and applied a stratified two-stage random sampling method to obtain geographically representative results (Figure 1). In each region, we randomly selected one province. The provinces were Jambi from the Sumatera region, East Java from the Java and Bali region, Central Kalimantan from the Kalimantan region, South–East Sulawesi from the Sulawesi region, East Nusa Tenggara from the Nusa Tenggara region and North Maluku from the Maluku-Papua region. In the second stage, in each province, three hospitals (1 class B hospital and 2 class C or D hospitals) were randomly selected, using a random number generator.
Class B hospitals have specialist services in at least four core specialist areas (internal medicine, paediatrics, surgery and obstetrics and gynaecology), four supporting specialist services, at least eight other specialists and at least two basic sub-specialists. Class C hospitals have at least four core specialist areas and four supporting specialist services, and class D hospitals have at least two core specialist areas services and therefore not necessarily a paediatrician.
The assessment tool was based on a global assessment tool which WHO and partner agencies have used over the past 10 years (WHO 2001, 2009; Campbell et al. 2008). The Indonesian version was adapted in line with the Indonesian version of the Pocket Book of Hospital Care for Children (WHO 2009; WHO, Ministry of Health Indonesia, & Indonesian Paediatric Association 2009), which provided the standards of care against which the performance was assessed and used a scoring system to fit into the process of the Indonesian hospital accreditation system (Indonesian Commission for Hospital Accreditation 2007). The hospital assessment tool used in this assessment was comprised of 10 hospital service areas as follows: (i) Hospital support functions including drugs, supplies and equipment; laboratory and radiology services; (ii) emergency care; (iii) children's ward; (iv) case management in the ward; (v) neonatal care; (vi) patient monitoring; (vii) mother and child friendly services; (viii) hospital administration support; (ix) discharge and follow-up; and (x) access to hospital. Each component has standards, which are divided into criteria and grouped into parameters. There were overall 34 standards, 102 criteria and 73 parameters. An operational definition (OD) was set up for each parameter according to the related standard and criteria to be assessed (see online-only appendix).
Information was collected during the hospital visit from hospital documents, case and facility observations, and interviews. The hospital documents included letters of referral, medical records, guidelines, documents of standard operating procedures for medical care or nursing care and regulations related to the evaluated service. The core component was the direct observation of admitted cases. If there were not enough cases for direct review, simulated cases were presented to staff to assess clinical case management (see appendix). Each parameter was scored from 0 (service not provided or no standards in place and case management seriously deviating from the guidelines in the pocket book) to 5 (good service is delivered according to complete written and up-to-date standards (see Figure 3 and web Appendix of assessment tool for details). Scores of each service were summed up and for comparability converted into a percentage, which was grouped into three categories: <60% (urgently needs to be improved), 60–79% (needs to be improved), ≥80% (good). Overall summary scores were calculated in the same way. After scoring, the assessors commented on the strengths and weaknesses of each parameter and created recommendations for improvement. These were then presented in an exit conference before leaving each health facility. Hospital staff and assessors were jointly developing an action plan to improve the quality of child care in the facility, with the hospital agreeing to follow-up action with a time line and responsible person.
There were six teams of assessors, one for each province, comprising of one paediatrician for assessing the case management, one member of the Indonesian Commission of Hospital Accreditation (ICAHO) for assessing the hospital management system, an official from the Ministry of Health for reviewing the hospital profile, one paediatric nurse for assessing the nursing care area and one assessor from WHO. Assessors were trained in the use of the assessment tool at a 3-day training workshop in a district hospital, which helped with standardising findings between teams. During this process, the same cases were reviewed independently by different assessors and their findings compared. If there were discrepancies, these were jointly reviewed, if necessary by referring back to the patient, and consensus achieved.
The assessments were done in February 2009. Each team visited three hospitals and one health centre with beds (not reported here) accompanied by a representative from the district or provincial health office. The assessors spent 2 days in each health facility.
Data were entered during the visit into a form which also served as the basis for the hospital feedback session. Scores were summed up as described above. Data were entered centrally into a data base programme and managed with SPSS version 11.5, with the necessary consistency checks. Scores are presented as medians and interquartile range (IQR), as several of the data are not normally distributed. Comparisons between hospital categories were made using the Kruskal–Wallis test, taking P = 0.05 as significant. For drug supplies and equipment, the proportion of hospitals with or without the item is presented. For each area, strengths and weaknesses were highlighted in the feedback session. These qualitative data were summarised across the hospitals semi-quantitatively, highlighting commonly found problems.
The Ministry of Health and the Provincial Health Offices gave permission for the study. They deemed ethical review unnecessary as it constituted a supervisory activity in line with the routine line management of the health services.
Of the 18 hospitals assessed, seven were class B (provincial), nine class C and two class D (district hospitals). Three class B hospitals were accredited. The remaining hospitals were either not yet accredited, or in two, the accreditation had expired. Nine hospitals were classified as fulfilling the requirements for Comprehensive Emergency Obstetric and Newborn Care (CEONC). The total number of beds varied from 30 in a class D hospital to 323 beds in a class B hospital. Selected characteristics of the hospitals are shown in Table 1. Three of the nine class C hospitals and neither of the two class D hospitals had a paediatrician, whereas one of the class B hospitals had six paediatricians. Admission statistics showed the main causes of admissions to be diarrhoea, fever, acute respiratory infections and birth asphyxia.
|Class/type of hospital||B||C||D|
|Number of hospital by class||7||9||2|
|Number of beds (total)||202–323||63–198||30–86|
|Number of paediatricians||1–6||1 (6 hospitals)|
|0 (3 hospitals)||None|
|Number of non-specialist medical doctors||14–22||3–15||1–3|
|Hospital with comprehensive emergency obstetric and neonatal care service||4||4||1|
The median result score across all 10 areas (summing up all 73 parameters) was 43% (IQR 28–53%) The summary findings in the 10 areas are presented by province in Figure 2 and details by clinical area with the summary scores by area in Table 2.
|Cough or difficult breathing Score 52% (IQR 41%–64%)|| |
● 88% of hospitals had nebulizers
● All hospitals had X-ray equipment
● Proper indications for doing CXR,
● Second line antibiotics were used without evidence of need.
● Only 6 hospitals had oximeters
Improper monitoring of O2 administration.
● 14 hospitals had O2 concentrators, but none was well functioning (lack of information on how to use and maintain)
● scoring system to diagnose child TB was not used, tuberculin test rarely done and incorrect use of anti-TB drugs
|Diarrhoea Score 38% (IQR 29%–47%)|| |
● Availability of antibiotics, and IV fluids
● Zinc available, though only used for some children
● Appropriate plan of rehydration was not established.
● All diarrhoea cases directly given IV fluid therapy. ORS not given.
● Administration of antibiotics and anti-diarrhoeal drugs for watery diarrhoea
|Fever Score 36% (IQR 34%–57%)||● Availability of essential laboratory tests|| |
● Inappropriate assessment and lack of considering differential diagnosis
● Incorrect diagnosis, classification and management of Dengue viral infection, severe malaria (with complications) and meningitis
|Malnutrition Score 30% (IQR 14%–35%)||● National Health Insurance Programme provides free care to all malnourished cases.|| |
● Nutritional status not routinely assessed for all inpatients
● Assessment of hypoglycaemia and hypothermia for severe malnutrition children not done
Micronutrients and Formula 75 and Formula 100 not available
|HIV/AIDS Score 5% (IQR 0%–20%)|| |
● HIV consultation room available in 7 HIV referral hospitals
● HIV laboratory test functioning in HIV referral hospitals
● Poor counselling
● Nursing care for health conditions related to HIV infected children incorrect.
● Most staff not trained in HIV management
|Routine neonatal care Score 53% (IQR 40%–73%)|| |
● Availability of trained midwives
● Clean delivery practice
● Rooming in
● Inadequate procedures and equipment for resuscitation
● Early initiation and exclusive breastfeeding not fully implemented
● Milk formula given in all hospitals.
● Kangaroo Mother care not well promoted.
● Immunization for newborns and vitamin K-1 not routinely given
|Case management of sick newborn. Score 40% (IQR 30%–59%)|| |
● Neonatal intensive care available in 4 hospitals (3 class B and 1 class C l)
● Trained physicians
● Phototherapy mostly available.
● No breastfeeding promotion
● Incorrect diagnosis and management of neonatal sepsis
● Feeding for sick young infants and low birth weight not clearly defined in medical record
● Second-line and third line antibiotics mostly used
● No evaluation of bilirubin level. Jaundice diagnosis mostly based on clinical signs
For emergency services, the median summary score was 57% (IQR range 55–70%). All hospitals had emergency services which were open 24 h 7 days a week and attended by a doctor and nurses, but they were not necessarily present all the time. All were separated from other services, with easy access. However, none of the hospitals had a triage system or complete standard operating procedures (SOP) in emergency care for children. None had recently trained staff for childhood emergencies and none had a referral policy. Nine hospitals lacked essential laboratory services which were available around the clock.
Case management for five common illnesses (cough and difficult breathing, diarrhoea, fever, malnutrition and HIV/AIDS) was assessed. The overall median score was 37% (IQR18–43%). Details of the qualitative findings and scores by major disease categories are given in Table 2.
Assessment in neonatal care focused in routine care, facilities of the neonatal room and sick newborn care. The median score was 46% (IQR 26–57%). Nine of 18 hospitals had been trained for CEONC. However, of these seven CEONC hospitals, only two had a score of at least 60% (60% and 70%). Performance scores in routine and sick newborn care did not differ significantly between CEONC and non-CEONC hospitals.
None of the hospitals had standard operating procedures for accident prevention, hygienic facilities nor a policy of closest attention for the most seriously ill newborns. Three hospitals had no separate neonatal room, with poor monitoring, insufficient hand-washing facilities and no standard operating procedures for injury prevention. Details of qualitative findings and scores for routine and sick newborn care are included in Table 2.
The mean score for patient monitoring was 40% (IQR 30–50%). At all hospitals, the monitoring procedure for the most seriously ill children was incomplete, and no standard operating procedures were in place for a monitoring plan. Routine reassessment of patients by nursing staff was not well defined, and the re-assessments done by doctors were not comprehensive and generally not well recorded. The availability of medical records services and nursing staff in the children's ward were strengths observed in the assessment.
The score for children's ward facilities was in median 43% (IQR 33–51%). A doctor in-charge was available in all hospitals. None of the hospitals had a paediatric intensive care unit or area, and only two of 18 hospitals had a specific space near the nurse's station for the most seriously ill children for intermediate care; the rest did not have different rooms to separate children by diseases. Their hygiene facilities were joined with other wards. Patient safety procedures were largely inadequate: 16 hospitals had beds for children without railings, with dangerous sharp disposal practices and uncovered electrical sockets. No standard operating procedures were in place for cleaning schedules and services. There were no standards for patient safety nor for intensive monitoring for severely ill children in any hospital.
Mother and children friendly services mean that the mother or other caregiver is allowed to stay with the sick child, which they did in all cases. But they did not receive sufficient health education and support during the hospital stay. The median score was 50% (IQR 25–65%).
The median score for hospital support functions was 68% (IQR 54–75%), covering availability of drugs, supplies and equipment, laboratory and radiology services. On the other hand, problems found in drugs, supplies and medical devices were lack of life-saving drugs (e.g. rectal diazepam) and essential drugs (zinc, antimalarials, anti TB drugs). There were maintenance programmes for medical and consumables supplies but no standard operating procedures for drug management (such as ‘First In, First Out’ management). Problems found in the laboratory services were as follows: no SOPs for samples taken from children and neonates (e.g. for bilirubin test), large amounts of blood needed for samples and SOPs for chest X-ray including response time and delivery time were not always available.
The median achievement of hospital support administration (consisting of availability of guidelines, referral transport and procedures and internal medical audit) was 33% (IQR 20–53%). There were only two hospitals (both in one province) which had been implementing internal medical audit. There were some treatment guidelines available in all of the hospitals, but they were incomplete, not up-to-date and rarely used.
The median score for discharge and follow-up service was 20% (IQR 0–40%).There were generally no plans for follow-up. No notes were given to the patients' guardian explaining the child's condition for the healthcare worker who would follow-up on the treatment or who had referred. The child's caregiver received some oral explanation and advice when to follow-up at the hospitals or primary health centre.
For access to hospitals, the median score was 60% (IQR 45–80%). The national health insurance programme made essential services financially accessible for the poor in all health facilities. Care seeking behaviour at the community level was adequate but facing economic, geographical and transport barriers. However, there were no standard operating procedures for referral, and the pre-referral treatment was not in line with IMCI standards.
Comparing class B hospitals with class C or D hospitals (district hospitals), performance was generally better in provincial hospitals, but only in the case of facilities for newborns and sick newborn care significantly so (69% vs. 46%, P = 0.03).
The involvement of all related stakeholders from the Ministry of Health, the Commission of Hospital Accreditation and the Indonesian Paediatric Society was a major strength of the survey that made it possible and to be conducted and that will help to take the Quality Improvement initiative forward. Most of the paediatricians involved were already familiar with the Pocket Book of Hospital Care for Children (WHO 2009; WHO, Ministry of Health Indonesia, & Indonesian Paediatric Association 2009). All of the assessors joined the assessment tool training and field test. Moreover, hospital staff and district and provincial health officers were very cooperative during the assessment, which bodes well for future improvements.
However, there were also some weaknesses. The assessment tool covered most of the important areas, but some items overlapped and some were missing, such as evaluation of diagnostic systems for tuberculosis and anti-retroviral drugs. Despite the efforts to standardise, there was still some subjectivity in the assessment, especially when it came to summarising the strengths and weakness in each area or service of assessment (as described in Table 2). Relying on expert assessors, however, is a well-accepted component of hospital accreditation systems, such as that by the Joint Commission International, where assessors compare performance against standards (Joint Commission International 2010). We scored the case management mostly based on medical records, case observation and interviews with care givers, nurses and doctors. Incomplete medical records and improper management of medical records made it difficult to track cases that had been discharged. Another obstacle during the assessment was the lack of specific cases at the assessed hospital, particularly severe conditions such as meningitis, severe malnutrition, cerebral malaria and HIV. We tried to overcome this through prepared case simulations and interviews about clinical practice and how conditions would be handled.
Previous surveys in the country, such an unpublished survey in 2003 in East Java, found similar problems. Globally, other studies found similar results. In Bangladesh, several priority areas for improvement were identified: provision of essential equipment and drugs; staffing; clinical competency in case management and supportive care; and documentation of patient assessment and monitoring (Hoque, personal communication). A study of 21 hospitals across seven countries in Asia and Africa showed that more than half of children were undertreated or inappropriately treated with antibiotics, fluids, feeding or oxygen (Nolan et al. 2001). Lack of triage and inadequate assessment, late treatment, inadequate drug supplies, poor knowledge of treatment guidelines and insufficient monitoring of sick children were key adverse factors observed. These are recurrent problems, and it appears that hospital care in developing and better developed countries such as countries of the former Soviet Union share similar problems (English et al. 2004; Duke et al. 2006b).
Cases evaluated in this assessment are the common paediatric problems in daily practice. If not treated properly, they lead to the high mortality observed in Indonesia and globally. Those cases are taught at all medical and nursing and midwifery schools and in the training of health workers. Various training courses and guidelines related to those common cases are available nationwide. Yet, most practitioners are neither aware of nor do they follow international guidelines on best practice. We did not assess leadership, working environment, teamwork, motivation and commitment directly. Lack of these may result in the unwillingness to follow good medical practice and maybe are the underlying causes of the low quality of care provided. The importance of leadership and good administrative practices is shown in case studies from Tehran, Tanzania, Guinea-Bissau and South Africa, which achieved better performance (Biai et al. 2007; Mohammadi et al. 2007; Chandler et al. 2008; Puoane et al. 2008).
One of the most important findings in this assessment was the lack of adequate standard operating procedures (SOPs) and the lack of mechanism to monitor the performance of care, including no internal medical audit. Referral was not working well, as we could not find a proper referral system from primary health centres to hospitals and back again. The assessed hospitals had basic capacity in terms of equipment and human resources except for the availability of paediatricians. However, good quality care needs a consistent strategy to better organise all available resources. The lack of a medical audit system likely contributed to the low quality of care at the hospitals.
The 18 assessed hospitals were randomly selected without knowing the accreditation level of either the hospital or the inclusion into special initiatives such as CEONC. It is interesting to note that CEONC hospitals performed similarly badly as other hospitals in neonatal care. We did not assess maternal health aspects, which are part of an upcoming assessment following similar procedures. Among 18 assessed hospitals, there were only three accredited hospitals, yet they, too, had low standard of care. This indicates that the current compulsory accreditation process for hospitals is not functioning.
Based on these findings, a hospital quality improvement process should be a priority to accelerate the reduction of child mortality. Local governments in provinces and districts need to include hospital care in their health development plan. The extension of the existing network of the Ministry of Health, the Indonesian Paediatric Society, WHO and the Indonesian Committee of Hospital Accreditation to other stakeholders such as the Indonesian hospital association, local government and donor agencies is a way forward. However, the assessed hospitals themselves clearly need to initiate action for their improvement of care. Good practices could be driven by both financial and non-financial strategies such as a motivating working environment and close supervision, as reported from Tanzania (Chandler et al. 2008). Training for doctors or other health workers needs to be accompanied by support and reinforcement, such as parallel interventions for senior managers using external personnel, as in the Eastern Cape, South Africa and in Guinea-Bissau (Biai et al. 2007; Puoane et al. 2008). The SEARO Framework for hospital improvement for children can serve as a guiding document (WHO 2009).
In conclusion, there are serious shortcomings in the quality of hospital care for children in Indonesia. Assessing and documenting is the first step in a quality improvement process. Changes in the assessed hospitals have already been implemented, and we are working on involving all hospitals in the country which are providing care for children. Ultimately, if this process is seen through, it should lead to a significant reduction in child mortality.
Members of Hospital Accreditation Commission: Sidik NA, Wirastini D, Karniasih P, Joewana S, Sudasri, W Anthon. Members of Indonesian Paediatric Society: Mulyani NS, Sitoresmi MN, Pudjiadi AH, Damayanti W, Malisie RF, Yangtjik K, Naning R, Pawitro UE, Nurhamzah W, Padmosiwi WI, Abdiwati NMY, Hegar B, Agung FH, Sekartini R. Paediatric Nurses: Yuliani R, Triwaluyanti F, Lukitowati, Djuwitsari U, Rahayu DR, Haryanti F. Staff of Ministry of Health: Karniasih P, Pritasari K, Mulati E, Joewana S, Sudasri, Anthon, Nova Y, Melati I, Marini, Rachman M, Sardono N, Gisank T, Jaya C,Mujadid, Daisy L. Staff of District Health Offices:, Endang, Damayanti, Minah, Melda, Sinar, Husein, Reza AT. WHO: Roespandi H, Weber MW, Setiawan T.