Magnitude and pattern of improvement in processes of care for hospitalised children with diarrhoea and dehydration in Kenyan hospitals participating in a clinical network

Abstract Objective WHO recommends optimisation of available interventions to reduce deaths of under‐five children with diarrhoea and dehydration (DD). Clinical networks may help improve practice across many hospitals but experience with such networks is scarce. We describe magnitude and patterns of changes in processes of care for children with DD over the first 3 years of a clinical network. Methods Observational study involving children aged 2–59 months with DD admitted to 13 hospitals participating in the clinical network. Processes of individual patient care including agreement of assessment, diagnosis and treatment according to WHO guidelines were combined using the composite Paediatric Admission Quality of Care (PAQC) score (range 0–6). Results Data from 7657 children were analysed and improvements in PAQC scores were observed. Predicted mean PAQC score for all the hospitals at enrolment was 59.8% (95% CI: 54.7, 64.9) but showed a wide variation (variance 10.7%, 95% CI: 5.8, 19.6). Overall mean PAQC score increased by 13.8% (95% CI: 8.7–18.9, SD between hospitals: ±8.2) in the first 12 months, with an average 0.9% (95% CI: 0.3–1.5, SD ± 1.0) increase per month and plateaued thereafter, and changes were similar in two groups of hospitals joining the network at different times. Conclusion Adherence to guidelines for children admitted with DD can be improved through participation in a clinical network but improvement is limited, not uniform for all aspects of care and contexts and occurs early. Future research should address these issues.


Introduction
Diarrhoea and dehydration (DD) is still a major cause of death in children under 5 years, causing approximately 0.5 million deaths annually, mostly in sub-Saharan Africa and Asia, and optimisation of available interventions has been identified as key to reducing case fatality over the next decade [1][2][3]. WHO guidelines incorporate the best available evidence and have been adopted by Kenya's Ministry of Health as protocols for in-hospital care [4,5]. However, guideline adoption at the national level does not always result in local implementation [6] that together amount to scaling up interventions. Thus, widespread adoption requires dissemination to all hospitals linked to active local efforts to promote guideline use. Clinical Networks are one means to promote more standardised approaches to care with examples of their value in some high-income settings [7,8]. Employing some of the principles of existing clinical networks, 13 first-referral level (district) hospitals in Kenya have been involved in a collaborative effort referred to as the Clinical Information Network (CIN). The Network has been aiming to improve the uptake of standard guidance for admission care for common diseases since October 2013 [9]. Wider benefits include the ability to better characterise paediatric admissions at scale and the use of shared data to implement and evaluate strategies that promote guideline adoption.
Major activities within the CIN include promoting the use of standardised patient forms (admission, treatment, and discharge) [10], regular audit and feedback on performance for chosen processes of care for common conditions, and network meetings [9,11]. The set of network activities employed drew on prior studies in Kenya [12,13], an understanding of context and relevant theory.
Studies report inconsistent effects, and therefore uncertain benefits, of such efforts to promote guideline adherence; for example, reviews report only modest improvements when audit and feedback (A&F) are used with median increases of 4.3% and 1.3% for indicators measured as dichotomous or continuous variables respectively [14].
A challenge for studies of whole networks is that typically there is no counterfactual enabling comparison of activities within a network to those outside it. Acknowledging this limitation, we use data collected by the CIN to describe changes in care provided to children with DD over the period hospitals have been engaged in this network. We used a composite score, the Paediatric Admission Quality of Care (PAQC) score, to describe the changes. The PAQC score summaries three domains of process of admission care (assessment, diagnosis and treatment of illnesses) into a single quality score for DD, malaria or pneumonia as discrete illnesses as well as when children have two or more of these illnesses [15].
We took advantage of the fact that hospitals joined CIN at different times, eight in late 2013 and five in early 2014, to consider 'early' and 'late' groups of hospitals (respectively), to consider the data from different before/after studies allowing us to compare the patterns of change in the two groups. We hypothesized that observing similar patterns of change would suggest that the effects of network activities were replicated across the two studies.

Study setting
We used data collected between October 2013 and November 2016 from 13 hospitals currently involved in CIN. CIN is a collaboration between Kenya's Ministry of Health (MOH), the Kenya Medical Research Institute/ Welcome Trust Programme (KWTRP) and the Kenya Paediatric Association (KPA). Initial hospitals joined the network in October 2013 (eight hospitals, referred to as 'early' hospitals) while a second cohort joined in March 2014 (five hospitals, referred to as 'late' hospitals). The different entry periods were the result of logistical (largely geographical) considerations and hospitals were not randomly allocated. Within CIN, hospitals themselves are responsible for adopting and supplying recommended standard medical forms and care is provided entirely by the hospital personnel. The hospitals typically have a paediatrician but in some sites a general medical officer (a degree trained physician) to supervise other junior clinicians including medical officer interns, clinical officers (non-degree trained physicians) and clinical officer interns in the absence of a paediatrician. Admission assessments and initial treatments are prescribed mostly by these junior clinicians with the team leader reviewing the management often 24-48 h later. New groups of intern clinicians rotate onto the paediatric wards every 3 months in all the hospitals with medical officer and clinical officer interns rotating at different timepoints. Treatment and other nursing care is provided by nurses and the hospitals often have shortages of nursing staff as is common in many resource poor settings.
We have previously described CIN processes in detail [9,11]. Briefly, information on admission, clinical information, immediate treatment and discharge information is abstracted into an online database by a specially trained clerk as soon as a child is discharged from hospital. This clerk is the only person at the hospital engaged by the research team. Data are then synchronised into a central server managed by KWTRP [16,17]. Reports on various processes of care are produced every 2-3 months and sent to the hospitals together with a combined report that summarises performance of all hospitals and allows them to compare their performance. Reports are delivered to each hospital both in print and soft copies for the paediatrician to disseminate, discuss with their clinical teams, and jointly identify action plans. The paediatricians received mentorship on how to give effective local feedback and how to develop a joint action plan with their teams. A senior paediatrician, based at KWTRP and coordinates CIN, calls the hospital paediatrician after each report and highlights key areas of improvement or deterioration. CIN meetings to share experience are held twice yearly and attended by the hospitals key personnel (paediatrician or designated clinician, nurse in charge of paediatric areas, and health records officer in charge). Studies using anonymised routine data from the CIN have been approved by the KEMRI scientific and ethics review unit.

Defining diarrhoea with dehydration
We identified children in the database who: were admitted with a diagnosis of both diarrhoea and dehydration (DD); who were aged 2-59 months; had comprehensive information on assessment, diagnosis and inpatient management. We excluded children with bloody diarrhoea or with severe acute malnutrition (SAM), for whom there are different guidelines on managing dehydration, and children with a minimal dataset. The minimal dataset [9] consists of age, sex, diagnoses and outcome, which are required for routine health information system reporting. Such data are also collected for randomly selected records in two hospitals with high patient volumes, in all 13 hospitals for those admitted for surgery or burns and for all records when the clerk is on leave (total 4 weeks in a year).

Definition of quality of care
The recently developed composite PAQC score measures multiple processes of care in children admitted with DD, malaria, or pneumonia separately or when these conditions co-exist in the same individuals [15]. The PAQC score measures three domains of the admission process of care: assessment, diagnosis and treatment of illnesses.
The assessment domain is made up of three items: recording of primary assessment signs required to diagnose the disease of interest, recording of any secondary signs necessary for disease severity classifications, and a third item representing complete documentation of all primary and secondary signs. The diagnosis domain is a single item representing whether a guideline recommended severity classification is indicated in the medical record; while the treatment domain is made up of two items, correct prescription and correct use of the treatment (dose, route, frequency and duration). The treatment domain for DD examines only correctness of fluid administration in accordance with WHO guidance for various severities of dehydration (shock, severe, some, none) and does not include use of zinc.
Each item has a binary score of 0/1 therefore the PAQC score ranges between 0 and 6. In children with DD plus pneumonia or malaria comorbidities, separate scores are estimated for each disease and then combined while preserving an overall patient level point score of 0-6 (this method is described in full elsewhere [15]), facilitating reporting of quality in the presence of these common comorbidities. For these analyses individual PAQC scores were divided by 6 and multiplied by 100 to obtain a percentage score for each child and mean percentage PAQC scores were then calculated for all DD admissions to each hospital for each month.

Analysis
We calculated the PAQC score for each patient and then created a binary measure to represent 'good care', defined as a score of five or six out of six. Using this categorical metric, we then calculated the proportion of patients with 'good care' for each month since joining the network. Month 1 is the month when a hospital joined, October 2013 or March 2014 for 'early' and 'late' groups respectively. We examined the changes in proportion with 'good care' from the baseline (month of joining the network). We also compared proportions with score of 3 or less, 'good care', or maximum score of six between the first and last 3 months in the network for each hospital.
Although CIN data are observational, we used the two phases of joining the network ('early' and 'late') to explore whether the pattern of response from time of joining the network (measured in months) was replicated across the two groups of hospitals. We hypothesised that observing such replication would provide some support for a generalisable effect of network participation on clinical practice.
Separate plots for the two groups of hospitals, drawn using mixed effects fractional polynomial regression, which allows for different baseline PAQC scores and changes over time across hospitals, showed that changes in PAQC score are bi-phasic with a cut-off at 12 months between the phases. Therefore, we fitted split-line regression models to enable estimation of slopes (change in PAQC score over time) and intercepts (baseline PAQC scores) in the two phases. This was achieved by including a binary variable, coded as 0 for the initial phase and 1 for the second phase, in the model and included an interaction term for time in the network. PAQC scores for the two hospital groups were compared in this model by inclusion of a binary variable coding hospital type (early/ late) and interaction term for time.
Similar analyses were undertaken to examine patterns of changes in specific process of care domains (assessment, diagnosis and treatment) to establish if different domains followed distinct trajectories with time. This latter analysis used combined data without categorisation into 'early' and 'late' hospitals. Analyses were done using Stata 13.1 [18]. A description of theory of change informing CIN [13], implementation strategies used [11] and how these might have resulted in changes seen have been published and were not the focus of this analysis.

Effect of network participation on processes of care for diarrhoea and dehydration
Lowess plots (not shown) consisting of individual PAQC scores created for each hospital showed a variety of patterns: (i) non-linear change with a rapid initial increase and later plateau phase (7/13), (ii) sustained improvement throughout the duration of network participation (3/13), (iii) decline and improvement (2/13) and (iv) minimal change in performance over the entire period (1/13). The predicted trajectory showed biphasic improvement in PAQC scores with a rapid initial increase (first 12 months, Phase 1) and a plateau (Phase 2) afterwards in the network (Figure 2). The predicted mean percentage PAQC score for all hospitals at enrolment (first month) was 59.8% (95% CI: 54.7, 64.9), and was similar between early and late hospital groups ( There was no significant difference between early and late hospitals with respect to baseline PAQC scores and change in scores in the first or second phases ( Table 1).

Discussion
Our analysis of data from 13 hospitals involved in collaborative efforts aimed at promoting guideline adherence across a range of common childhood illnesses in Kenyan hospitals shows that documentation for various processes of care improved [9,11,16,17]. Using a recently developed composite score for processes of admission care, the PAQC score [15], we show that the process of DD admission care as a whole improved over time. Most improvement was seen in the first year; subsequently changes in the PAQC score measured across hospitals slow down or plateau. Overall improvement occurs in the three domains of process of care (assessment, treatment, and diagnosis) but is greatest for the assessment domain. Hospitals have different performance at baseline and show different patterns of improvement in response to the network activities. This 'hospital effect' is perhaps not surprising given the complexity of hospitals as organisations and the reliance on local paediatric team leaders' efforts to promote improved  practice. It is also important to note that in each of these hospitals it is the junior clinicians who admit patients and whose practice is assessed with the data available in the clinical information network. These junior clinicians rotate regularly; hence in each hospital the junior staff changed on at least 8 occasions (3 monthly) during the study period.
In several hospitals, the paediatric team leaders also changed but general improvements observed appear to be robust to such staff turnover. Recent reviews demonstrate only modest improvement of compliance with practice recommendations when audit and feedback are used, namely a median increase of 4.3% for dichotomous and 1.3% for continuous outcomes [10]. Our analysis suggests a mean improvement in PAQC score for children with DD of nearly 14% in the first 12 months and minimal change thereafter. We have previously shown some improvement in processes of in-patient care through training of health workers, facilitation, evidence dissemination, supervision and face-to-face feedback derived from intermittent surveys in Kenya [6,19,20]. In the current approach we sought to build on this platform and begin to explore the potential of clinical networks [21] linked to improved health information systems as a means to foster practice change. It has been suggested that networks may help overcome some of the 'wicked problems' inherent in achieving changes at scale, particularly if they can effectively engage sets of practice leaders who together embrace the change efforts [22][23][24].
This study raises questions on the nature of interventions that may be required as any change process evolves. Feedback in the network was given using methods associated with its effectiveness including provision by respected senior colleagues, frequent delivery, timeliness (2-3 monthly reports were based on data as recently as the previous week) and having both verbal and written components highlighting areas requiring improvement [14]. After good improvements in the first year of activities, however, performance tended to stagnate. It is possible therefore that new strategies are required to motivate and facilitate continuous improvement.
Improvements in the diagnosis and treatment domains appeared harder to change, although for the diagnosis domain relatively high baseline performance may have led to a ceiling effect. It is possible therefore that feedback might be best suited for changing specific types of behaviour and that it may be less effective for more complex cognitive tasks that could also require elements of instruction or coaching at the individual level to foster improvement [14,25,26]. In the current form of the network any coaching of individual junior clinicians is left to the paediatric team leader's discretion. Information systems that identify individual clinicians' areas of weakness could offer a means to tailor the form of feedback to needs or help inform team leaders which aspects of the care process require attention.
Findings from the CIN network are limited as it only collects information on documentation of processes of care but we have shown that correct fluid prescription is associated with better survival in children with diarrhoea and dehydration [27]. Our results therefore best represent clinicians' intention to treat in a context where constraints in health personnel and supplies may exist. It is also not possible to disaggregate what activities of the network are effective limiting our ability to make recommendations for adoption of any single activity. Our study design does not allow us to make causal claims that CIN activities promote guideline adoption. The efforts of networks are hard to examine using experimental designs as identifying a counterfactual population is challenging and networks themselves are typically a n = 1 activity (while not necessarily being an entirely homogenous intervention within all participating sites). Time series evaluations (including statistical process control methods) might have provided more credible evidence of effects than the before/after analyses we conducted. However, there are difficulties conducting such studies where the information available prior to intervention may be poor (and indeed one aim of the network was to improve such information). Additional qualitative research does support the proposition, the CIN was valuable in promoting changes in practice and guideline adoption through pastoral practices [28]; however questions remain over whether this strategy would work in all contexts.
In this analysis we only present quantitative outcome of processes of care for one condition but we recognise that CIN intervention was implemented in complex adaptive system and a realistic evaluation examining the context, mechanisms and outcomes would be most appropriate [29]. However, we have previously described theory of change informing CIN intervention [13], implementation strategies adopted [11] using typology described by Powell et al [30.], including which components of implementation strategies fall in various domains of realistic evaluation [11].

Conclusions
Care of children admitted with DD can be improved by a set of activities encompassed in a clinical network. Future research should explore how audit, feedback and linked interventions can best support changes for different contexts, for different tasks, at different stages of the performance cycle, and define appropriate implementation strategies for achieving these.