Quality of surgical care in hospitals providing internship training in Kenya: a cross sectional survey.

Abstract Objective To evaluate services in hospitals providing internship training to graduate doctors in Kenya. Methods A survey of 22 internship training hospitals was conducted. Availability of key resources spanning infrastructure, personnel, equipment and drugs was assessed by observation. Outcomes and process of care for pre‐specified priority conditions (head injury, chest injury, fractures, burns and acute abdomen) were evaluated by auditing case records. Results Each hospital had at least one consultant surgeon. Scheduled surgical outpatient clinics, major ward rounds and elective (half day) theatre lists were provided once per week in 91%, 55% and 9%, respectively. In all other hospitals, these were conducted twice weekly. Basic drugs were not always available (e.g. gentamicin, morphine and pethidine in 50%, injectable antistaphylococcal penicillins in 5% hospitals). Fewer than half of hospitals had all resources needed to provide oxygen. One hundred and forty‐five of 956 cases evaluated underwent operations under general or spinal anaesthesia. We found operation notes for 99% and anaesthetic records for 72%. Pre‐operatively measured vital signs were recorded in 80% of cases, and evidence of consent to operation was found in 78%. Blood loss was documented in only one case and sponge and instrument counts in 7%. Conclusions Evaluation of surgical services would be improved by development and dissemination of clear standards of care. This survey suggests that internship hospitals may be poorly equipped and documented care suggests inadequacies in quality and training.


Introduction
In many countries, health systems are organised on the basis of a district or equivalent administrative unit. These typically contain a district hospital linked to a network of primary care and community-based health services. From a systems perspective, and as recognised since Alma Ata (WHO 1987), it is critical that district hospitals deliver high-quality care effectively and efficiently. Hospitals should allow this through delivery of more complex forms of health care in a setting that concentrates skills and resources (English et al. 2006). In the case of surgery, there is some evidence that provision of basic surgical services can be highly cost-effective in low-and middle-income settings (McCord & Chowdhury 2003) and such services are likely to gain in importance in settings such as Kenya, as life expectancy increases and trauma and non-communicable diseases grow in importance (Lozano et al. 2012).
However, reports spanning maternal and paediatric care suggest that hospitals often fail to deliver good services in low-income settings, but there are few reports on surgical services (Nolan et al. 2000;English et al. 2004;Kingham et al. 2009;Kruk et al. 2010). Donabedian (1978) described the most widely used simple framework for examining healthcare delivery, dividing it into three major aspects: structure (the nature of the inputs available to provide care), process (the actions taken to con-vert inputs into the care provided) and outputs or outcomes (the consequences of care). The latter are not limited to health status but may encompass satisfaction, costs or other valued outputs depending on one's perspective. Unfortunately, routine health information systems (HIS) in low-income countries (LIC) are often unable to provide reliable, disease-specific outcome data, and routine data based on process indicators are also lacking (Kihuba et al. 2014). In the absence of routine data, a survey approach was designed to assess the quality of routine surgical services in Kenyan hospitals in collaboration with the Ministry of Health.

Selection of indicators
There were no nationally approved quality indicators for surgical services provided in hospitals in Kenya prior to this survey. National guidelines spanning common surgical presentations had, however, been published in 2009 by the Ministry of Health. Performance indicators were therefore developed with the Ministry of Health using these and WHO resources as reference standards (Ministry of Health Republic of Kenya 2009; WHO 2013). Initially, this team prioritised assessment of five specific admission conditions: head injury, chest injury, long-bone fracture, burns and acute abdomen. Indicators spanned availability of essential equipment and resources (structure) and provision of care in three domainsassessment of key elements of the admission history and examination, general management (including investigation and adjunctive medical or supportive therapy) and operative carewere undertaken. Individual indicators were constructed to allow an 'achieved' (1) or 'not achieved' (0) assessment based on direct observation for facility-level structure indicators or examination of case records for process of care indicators. The latter has the limitation of evaluation based only on what is documented about the care provided but the advantage of making examination of multiple patient events possible within reasonable timescales. Draft tools were pilot-tested in a hospital not included in the survey with revisions made for clarity as required, and a detailed standard operating procedure was developed to guide its use.

Selection of hospitals and sample size
Kenya has 235 public hospitals that should be able to offer first referral-level surgical services provided by general medical officers (e.g. non-operative fracture management, minor surgery) as a minimum standard of care.
Of these, 40 are 'internship training centres' with (i) consultants in each of the four major inpatient disciplines (surgery, internal medicine, obstetrics and paediatrics) and (ii) basic laboratory, pharmacy, radiology services and at least one working theatre. Such centres provide supervised training to graduate clinical staff during a 1-year compulsory internship period prior to full registration. These provide both routine and emergency operative care for major cases with the support of physician or non-physician assistant anaesthesiologists. Young clinicians leaving these internship hospitals then often lead service delivery in smaller hospitals.
Study hospitals (n = 22) were purposely selected by the Ministry to provide a logistically feasible and geographically representative sample of the 40 internship hospitals. With 22 hospitals to be assessed, we explored how precisely we might report estimates of compliance with quality indicators. Taking hospitals as the units of clustering, assuming a design effect of 1.5 based on a previous study in district hospitals ) and anticipating reporting 50% or 10% correct performance, we estimated that a minimum of 12 and four cases per hospital featuring the included diagnoses listed above would be required to report estimates with a precision of AE7.5% (95% CI). The case records included in the survey were identified from ward registers by working backwards from 31 May 2012 (just prior to survey visit) until 15 cases of a specific diagnosis most proximate to the survey were retrieved or the time available for survey work was exhausted.

Survey conduct
Data collection was performed over 5 weeks in June and July 2012. Survey staff comprised 22 Ministry employees (nurses, records officers or clinical officers) with one drawn from each hospital selected for survey. All staff underwent 1 week of training together that included a pilot survey in a non-study hospital. Staff were subsequently divided into five teams (4-5 per team) that each visited 4-5 hospitals for 3-4 days during which they also undertook other survey activities. Each team was led by a researcher who was also an experienced health worker. Resource availability (structure) was assessed by team leaders, who conducted a facility walk-through using a standard checklist. Process of care and outcome data were entered directly into laptops from the case records using REDCap â (Research Electronic Data Capture) (Harris et al. 2009). Additional source documents included prescription records, monitoring and discharge charts and operative notes where applicable.
Real-time range and consistency checks were run as the data were entered. At the end of each data collection day, data entries were examined for errors using procedures executed in STATA â v10 (Stata Corps, TX, USA). Corrections were ultimately made by referring back to the source document under the supervision of the team leader. The clean data files from all study hospitals were then uploaded into a central REDCap server daily for secure backup of study data.

Analysis
Availability of essential resources was scored using dichotomous indicators (0: absent vs. 1: present) for each structure item for each hospital. The simple proportion of the 22 hospitals complying with each indicator could then be reported. For process indicators, with multiple cases assessed in each hospital, we calculated the proportions of cases complying with the quality indicator together with 95% confidence intervals adjusted for clustering within hospitals. In a number of areas, such as documentation of elements of the primary and secondary survey, dichotomous indicators were summed to derive a score with a denominator equal to the number of tasks. Such scores have the limitation of focusing on what is documented as an indication of what is done that is common to most clinical audit. For these scores, we began by calculating the median score for all cases within a hospital and then report the median of the 22 hospitals' scores together with the interquartile range.

Ethical approval
Scientific and ethical approval was obtained from the KE-MRI Ethical Review Committee. The Ministry of Health provided administrative permission, and assent was obtained from the hospital management prior to surveys.

Results
Surveys were successfully conducted in the 22 hospitals approached with all data collected in a period of 5 weeks in June and July 2012.
On the day of survey, consistent running water was available on wards in 19/22 (86%) hospitals and soap in 16/22 (73%). Only 7/22 (32%) had alcohol hand rubs. There were discarded sharps overflowing from containers in 6/22 (27%) hospitals. National clinical guidelines for common surgical problems were available in 3/22 (14%) hospitals and wall charts for management of surgical emergencies in 5/22 (23%). Clinical audits and regular staff meetings were reported in 12/22 (55%) hospitals, although only 8/22 (36%) had minutes for these meetings available for review at the time of the survey.
Crystalline penicillin was available in all hospitals. Gentamicin, amoxicillin, oral antistaphylococcal penicillins and ciprofloxacin were available in half of study hospitals (11/22, 50%), but injectable antistaphylococcal penicillins were available in only one hospital (5%). Ceftriaxone and chloramphenicol were available in 9/22 (41%) hospitals. Analgesics had better availability. Diclofenac and paracetamol were available in almost all facilities. Morphine and pethidine were present in half of the hospitals (11/22, 50%). Heparin and cimetidine were available in fewer than half of the hospitals (7/22, 32%). With regard to anaesthetic agents, bupivacaine for spinal blocks was available in 5/22 (23%) hospitals only, but all hospitals had inhalational agents. All hospitals had nasogastric tubes. Intravenous fluid administration sets, gloves (both clean and sterile) and urethral catheters were available in at least three quarters of hospitals. Fewer than half of hospitals had both a source of oxygen (typically cylinders) and a working flowmeter on the surgical ward.

Process
Overall 956 case records were reviewed. Among these selected records, the diagnoses were distributed as follows: head injury n = 178 (19%), chest injury n = 136 (14%), acute abdomen n = 220 (23%), long-bone fractures n = 237 (25%) and burns n = 185 (19%). A summary of patient characteristics is provided in Table 1. The sampling approach does not allow reliable estimation of true prevalence. Identification of chest injury diagnoses was most difficult as this event was rare. Irrespective of diagnosis, more than 80% of cases presented directly to the hospital from home or the site of trauma rather than 242 being referred from another facility. Key demographic data were missing in from 5% to 17% cases (detailed in Table 1). Where available, included case records indicated a preponderance of male cases for trauma-related admissions [specifically head injury, chest injury and fractures (81%, 86% and 71% respectively, detailed in Table 1)]. Additionally, the study population was young with the median age for cases reviewed <40 years for all conditions, and for those with burns, the median age of cases was only 7 years. Overall, mortality across all cases examined was low (1.9%) with the highest fatality in the group of cases of acute abdomen (4.3%, detailed in Table 1). Referrals to a higher level of care were most frequent (25/178, 14%) for head-injured patients. Time of admission was recorded in the case records on fewer than 30% of reviewed records (detailed in Table 1), regardless of the diagnosis. Similarly temperature, respiratory rate, pulse and blood pressure were recorded in the clinical records of, respectively, fewer than 40%, 31%, 36% and 30% of admissions, regardless of the diagnosis (detailed in Table 1).

Selected diagnoses: trauma
Road traffic accidents were responsible for 53%, 40% and 42% admissions with head injury, chest injury and fracture, respectively (see Table 2 for detail). Table 2 also provides values for the number of multiple-injury patients. We examined records for documentation of a primary survey evaluating airway, breathing and circulation at admission for each of our included trauma diagnoses (Table 2). A secondary survey comprises a fuller assessment of seven key areas (for detail, see Table 2). The median number of secondary survey assessment tasks documented in trauma patients' case records was very low, at 2/7, 1/7 and 1/7 for head injury, chest injury and 0-3, 0 (0-1) 0-3, 0 (0-2) 0-3, 0 (0-0) Primary examination score as percentagesrange, median (IQR) 0-100, 0 (0-33) 0-100, 0 (0-67) 0-100, 0 (0-0) Secondary examination scorerange, median (IQR) 0-7, 2 (1-2) 0-7, 1 (0-2) 0-6, 1 (0-1) Results presented include % in parentheses and 95% confidence intervals adjusted for clustering at hospital level in square brackets unless otherwise specified. fracture diagnoses, respectively. Rather than documenting a full secondary survey, these findings suggest that clinicians focused, at least for documentation, on areas anticipated to be closely related to the primary cause of admission. For example, examination of the head and face (65%) and a coma scale assessment or pupillary responses (42%) were documented in head-injured patients, and chest examination (65%) and extremity examination (65%) were documented in chest injury and fracture patients, respectively (see Table 2 for detail). When further detailed information on the injuries was sought from records, this was rarely found. For example, assessments of peripheral pulses or for compartment syndrome were documented in fewer than 19/237 (8%) long-bone fracture cases, while auscultation for bilateral breath sounds and palpation for chest wall crepitus were documented in fewer than 34/136 (25%) chest injury cases.

Burns
Documentation of the total body surface area (%TBSA) was missing in 25/185 (14%) of burn cases but when recorded was mostly <20% (

Surgical procedures
A total of 171/956 (18%) admissions were documented as taken to theatre for surgical procedures. This was uncommon for head-injured patients (6/178, 3%) and most frequent for those admitted with acute abdomen (84/220, 38%) ( Table 1). The form of anaesthesia used was not documented in 12/171 (7%) cases. Where documented, general anaesthesia was employed in 128/159 (81%), spinal anaesthesia in 17/159 (11%) and other forms of sedation or local anaesthesia in the remainder. Assuming that cases in which general or spinal anaesthesia was used to represent more major cases (n = 145), we examined how well the surgical procedures were documented, focusing on elements that might be considered universal standards of care for professionals or linked to patient safety (Table 3). Of these 145 major cases, we found operation notes in the record for 143 (99%). There was a clear indication of who the surgeon was for 130/ 145 (90%) cases but far fewer contained a formally signed record of surgery (94/145, 65%). Anaesthetic records were less commonly found (104/145, 72%), and the identity of the anaesthetist could be determined in half of cases with spinal or general anaesthesia (72/145, 50%). Pre-operatively measured vital signs of pulse, blood pressure and temperature were noted in approximately 116/145 (80%) cases (Table 3), but time of last meal (56/145, 39%) and any pre-medication (33/145, 23%) were less commonly recorded. Evidence of consent to operation was found in 113/145 (78%) cases. Records of the operation reported findings in 120/145 (83%) cases and the nature of the procedure in 137/145 (95%) and documented the patient's condition at the end of the surgery (either in surgical or in anaesthetic notes), the drugs used for induction or maintenance of anaesthesia and vital signs during the operative procedure in twothirds (Table 3). Blood loss was documented in only one case and sponge and instrument counts in 7% (10/145).

Discussion
Quality and safety of surgical care have been of concern in developed countries for many years driven by desires to improve patient outcomes, to reduce health system costs attributable to poor care and, perhaps most powerfully in some countries, to avert litigation (Maggard-Gibbons 2014). Considerable efforts aimed at improvement span highly structured ways of working, often guided by checklists (Soar et al. 2009), and professional and public 246 scrutiny of performance (Maggard-Gibbons 2014). In lowincome settings, we know little about the challenges facing surgeons in non-tertiary hospital settings and little about the quality of services or patient outcomes. Published research suggests major deficiencies (Kingham et al. 2009;Notrica et al. 2011;Groen et al. 2012). In this study, we evaluated the quality of surgical services in hospitals providing experiential (internship) training to young doctors. These hospitals had a qualified surgeon who had undergone 3 years of postgraduate specialist training and might be expected to offer a higher technical level of care than the majority of Kenya's public hospitals. It is therefore of concern that basic resources (including running water, soap and first-line drugs) were not uniformly available and that surgical patients had to share beds. Assessing quality of the process of care typically requires that standards or criteria are defined and disseminated. Explicit standards for basic surgical services have not been developed in Kenya. National guidance in narrative form in books aimed at practitioners was developed by the Ministry of Health 3 years prior to this study (Ministry of Health Republic of Kenya 2009), but these or other practice guidelines were found in very few hospitals. The default position is therefore that surgeons within the hospitals set standards through example and supervision. Formal opportunities for this form of oversight typically seemed limited to scheduled outpatient clinics, ward rounds and half-day sessions allocated to conduct of elective operations; all typically conducted once or at most twice per week. The contribution surgeons make to emergency care or other forms of training and practice was not evaluated in this study, but anecdotal reports suggest that these are often limited.
To address the gap in explicit standards, we worked with Ministry of Health staff to identify basic indicators for this survey drawing on local knowledge, professional norms and existing WHO tools (WHO 2013). Because policy makers requested that specific causes of surgical admission be prioritised for study, we cannot provide information on the general pattern of surgical admissions to hospitals. Amongst the 956 cases studied, three quarters (Table 1) were likely the result of trauma or Results presented include % in parentheses and 95% confidence intervals adjusted for clustering at hospital level in square brackets unless otherwise specified.
accidents (head injury, chest injury, fractures and burns) and most (80%) presented directly to the hospital. We noted major deficiencies in documentation of the findings of primary and secondary surveys in these patients. We acknowledge that documentation is an imperfect measure of whether the examination was actually performed, but Kenyan policy makers deemed such documentation good practice, a sentiment shared by most working in emergency departments globally (WHO 2003). Documentation of key symptoms and signs that aid diagnosis of acute abdomen was also poor, and investigations were sparsely used, including microbiological investigations. A relatively small proportion of admissions ultimately received surgery, while the majority of long-bone fractures and burns were treated conservatively. In the limited number of cases that ultimately required an operation, the rationale was not always well documented. Pre-operative assessment of vital signs was inconsistently recorded, signed consent was on occasions not found in the record, and records of pre-medication used, estimated blood loss or instrument/sponge counts were rare. Anaesthesiologist identification, which anaesthetic agents were used, and the patient condition at the end of surgery were available in about half to two-thirds of cases. Such findings suggest that there is room for significant improvement in documentation of care provided by all members of the operative team. While tools such as checklists are available to support improved care (Soar et al. 2009), these were not found in the hospitals surveyed although work on essential obstetric (Hussein et al. 2012;Dumont et al. 2013) and inpatient paediatric care English et al. 2011) suggests that improvement of services is possible with appropriately designed interventions.
Although they cannot be directly linked to patient outcomes, our findings on the structure and process of care suggest that the Kenyan Ministry of Health should devote increased attention to the quality of surgical services and the quality of experiential training being received by young doctors. In other work, we have found that efforts to monitor outcomes of hospital care through the routine health information system are undermined by poor data quality (Kihuba et al. 2014). A need exists for clearly defined standards for surgical care, improved access to guidelines, and monitoring and evaluation of core services against agreed standards as part of increasingly system-wide efforts to improve care. This would be facilitated by improved routine information systems that are now a global priority (Kim 2012). While awaiting this, we have demonstrated that surveys can be useful for facilitating national and global debate on the urgent need to improve the quality and safety of surgery.