Implementing an intervention to improve decision making around referral and admission to intensive care: Results of feasibility testing in three NHS hospitals

Abstract Rationale, aims, and objectives Decisions about whether to refer or admit a patient to an intensive care unit (ICU) are clinically, organizationally, and ethically challenging. Many explicit and implicit factors influence these decisions, and there is substantial variability in how they are made, leading to concerns about access to appropriate treatment for critically ill patients. There is currently no guidance to support doctors making these decisions. We developed an intervention with the aim of supporting doctors to make more transparent, consistent, patient‐centred, and ethically justified decisions. This paper reports on the implementation of the intervention at three NHS hospitals in England and evaluates its feasibility in terms of usage, acceptability, and perceived impact on decision making. Methods A mixed method study including quantitative assessment of usage and qualitative interviews. Results There was moderate uptake of the framework (28.2% of referrals to ICU across all sites during the 3‐month study period). Organizational structure and culture affected implementation. Concerns about increased workload in the context of limited resources were obstacles to its use. Doctors who used it reported a positive impact on decision making, with better articulation and communication of reasons for decisions, and greater attention to patient wishes. The intervention made explicit the uncertainty inherent in these decisions, and this was sometimes challenging. The patient and family information leaflets were not used. Conclusions While it is feasible to implement an intervention to improve decision making around referral and admission to ICU, embedding the intervention into existing organizational culture and practice would likely increase adoption. The doctor‐facing elements of the intervention were generally acceptable and were perceived as making ICU decision making more transparent and patient‐centred. While there remained difficulties in articulating the clinical reasoning behind some decisions, the intervention offers an important step towards establishing a more clinically and ethically sound approach to ICU admission.


| INTRODUCTION
Treatment on an intensive care unit (ICU) improves the survival rates for patients with life-threatening illnesses, 1 and timely admission to an ICU is associated with better patient outcomes. 2 However, ICU treatments such as ventilatory and cardiovascular support and renal replacement therapy place a considerable burden on patients: patients may be left with significant physical and psychological morbidity which sometimes persist for many years. 3 In some cases, the resultant burdens of intensive care treatments may outweigh any potential benefit, and patients may receive greatest benefit by having life-supporting therapy limited to less invasive treatments that can be provided safely on a ward or from palliative care. Decisions Despite this complexity, there is currently no specific guidance or framework to assist clinicians with this decision-making process.
The limited available guidance tends to focus on process issues 4 or high-level principles such as the need to balance burdens and benefits of treatment. 5 Empirical studies have found that a wide range of clinical and nonclinical factors influence these decisions, with factors such as the patient's functional status (eg, ability to perform certain activities) and quality of life being assessed from the perspective of the doctor rather than the patient. [6][7][8][9][10][11][12][13][14][15][16][17][18] There is very little published research on the involvement of patients and those close to them in this decision-making process, despite this being a principle of good clinical practice. 19 There is, therefore, a need to support referring teams and ICU doctors who make these decisions in order to consistently achieve more transparent, patient-centred, and ethically justifiable decisions.
To address this, we developed a decision-support intervention (DSI) to improve the decision-making process regarding ICU referrals and admissions and tested its feasibility in three hospitals in England. In this paper, we present an evaluation of the intervention in terms of how well it was implemented, its acceptability, and its impact on decision making as perceived by staff at the implementation sites.

| The intervention
The development and implementation of the intervention was part of a larger project 20  The decision support intervention drew on data from all elements of the larger project. 21 It was underpinned by a conceptual "decision support" framework of a patient-centred, evidence-based, and ethically justifiable decision process that drew on Accountability for Reasonableness 22 as its theoretical model. It included the following elements: 1. a referral form to be used by the referring doctor when making a decision whether to refer a patient; 2. a decision form to be used by the ICU doctor when making a decision about whether to admit a patient; 3. credit card-sized outlines of the framework as prompts for clinicians ( Figure 1) framework is on what is likely to be the optimal treatment for this particular patient rather than the binary question of whether or not the patient should be admitted to ICU. Clinicians are asked to document their recommendation for the patient and to explicitly document who has contributed to the decision (ie, referring and ICU team members, patient's family, and/or the patient).

| Setting and implementation plan
Having developed the intervention, we drew on the normalization process theory (NPT) literature and ran a workshop with ICU doctors and critical care outreach (CCOR) nurses to inform the implementation strategy. 23,24 Three NHS hospitals purposively sampled for ICU size (less than 20 ICU beds; 20-30 ICU beds; more than 30 ICU beds) were recruited as feasibility sites. Each site was asked to identify two members of staff to act as implementation champions, and one chose to have three champions. Champions were ICU consultants (Hospitals A and C), ICU registrars (Hospital B), and CCOR nurses (Hospitals A and C).
The implementation period prior to commencing data collection was set at 8 weeks. Decisions about the specific implementation method were the responsibility of the champions at each site as they were familiar with the structure and processes within their own institution. We held a "train the trainers" event for the champions at each site, providing information about the study, detailed education on the conceptual framework and use of the referral and decision forms. Champions were asked to set up a log for all patients referred to ICU during the study period; one site had an existing log, two introduced it for the study.

| Evaluation
A 6-week data collection period followed the 8 weeks of implementation.
A mixed methods approach was used. A single researcher (SR: postdoctoral research fellow with a PhD in health and social studies) collected both quantitative and qualitative data. Quantitative data was collected by examining the records of patients referred to ICU during the data collection period. Qualitative methods included interviews with champions, observation of staff training, and interviews with referring and ICU clinicians during the data collection period. The project was approved by the

| Quantitative data collection
Use of the intervention was recorded by examining the records of all patients identified as having been referred to the three ICUs during the data collection period. For patients who had been referred, we extracted the following data from their clinical records: date, time, and location of referral; doctors making/reviewing the referral; and whether a referral/decision form had been used. If a form was present in the records, the extent of completion was documented.

| Qualitative data collection
Interview topic guides were developed for champions and participating doctors. Interviews with implementation champions were conducted regularly throughout the 8-week implementation period, either face-to-face or by telephone. Two instances of the champions delivering the educational materials were observed at each site and field notes taken.
Data from patient records were used to invite referring and ICU doctors to face-to-face interviews to explore the intervention's acceptability and impact on their decision making. Participants were sampled according to whether they had used a referral/decision form during the data collection period and to gain a mix of specialties, grades, and admit/not admit decisions (see Table 2). Interviews lasted between 7 and 60 minutes and took place in the hospitals. Written consent was obtained prior to interview. In the interviews, doctors were asked to recall a specific case to avoid hypothetical discussions.
The referral and decision support forms were used to prompt their recall. Finally, we carried out "debrief meetings" with the champions at each site to gain further insight into the implementation. All interviews were audio-recorded, transcribed verbatim, and anonymized.

| Data analysis
Data extracted from the patient records were analysed as counts, percentages, and, where there was a continuous variable, means and standard deviation. We looked for associations between patient/doctor/organizational factors and form completion. Chi-squared tests were used to compare categorical variables, and Student's t test was used to compare continuous data. Analyses were carried out by the statistician (HP) using the software R. 26 All interview transcripts and field notes were entered into NVivo QSR 11 27 and analysed thematically. [28][29][30] Initial coding of data was undertaken by SR and checked for consistency by FG (sociologist and GP) using a sample of transcripts. During analysis meetings the team (SR, FG, AS, [ethicist and GP], and CB [ICU consultant]) reviewed codes and emerging themes and checked interpretation against the data, leading to further analysis and refinement of themes until consensus was reached. We used both predefined (from our topic guide) and emergent nodes.
Due to time constraints, the main analysis occurred following data collection, so to check that we had not missed any important perspectives in our interviews, we conducted audio-recorded debrief meetings with the implementation champions. In these meetings, we explored the emergent themes in our data to aid interpretation and check for missing perspectives. No new perspectives emerged suggesting data saturation had been reached in our interviews.
To integrate the quantitative and qualitative data, we examined both datasets and considered where they converged, complemented, or contradicted one another (triangulation). 31

| Form usage (compliance with intervention)
Data were extracted from 181 sets of patient records ( Table 1). The forms were used in 28.2% of all eligible referrals across the three sites (44.4% at Hospital A, n = 28/63; Hospital B 21.4%, n = 3/14; and Hospital C 19.2%, n = 20/104). The presence of a referral form in the record was associated with a higher likelihood of a decision form being present (16.6% vs 3.3% P < .001). We investigated a number of factors with respect to an association with form usage, including patient gender, time of day, and ICU bed availability, but patient age was the only factor that had a statistically significant association with use of the forms: forms were used more often in older patients (P < .001, t test). See Supporting Information for more detail.

| Implementation
The champions took different approaches to implementation.
Hospitals A and C intended to roll the intervention out across the whole site, whereas Hospital B only included referrals from three clinical departments: haematology/oncology, respiratory, and emergency department (ED). Champions at all three sites used the presentations provided by the research team at grand round and departmental meetings, modifying them to be hospital specific. The CCOR champions did not use these resources, preferring to explain the study verbally at team meetings, individually at shift changeovers, and/or utilizing a group message service to remind colleagues about the study. The champions placed boxes containing forms in prominent positions on hospital wards and the ICU.
The champions described several challenges in achieving implementation.
Reaching target groups to raise awareness and to deliver training on the intervention within the time period (8 weeks) was challenging, particularly at the two larger hospitals. The champions at Hospital B decided to limit the use of referral forms to three clinical areas to overcome this. In practice, however, this meant that ICU doctors at this site forgot about or disregarded the forms in between referrals from these three areas.
The status and credibility of champions within a hospital influ-  At Hospital A, the champions used a "league table" to harness the competitive spirit between the doctors, challenging them to achieve the highest percentage of form use in the referrals and decisions they made.
The champions at Hospital C felt that the intervention fitted well with their already-existing activities in relation to recognizing and improving advance decision making for deteriorating adults. For example, their hospital had recently embedded daily "safety huddles" on every ward, which included discussion of patients who were not for resuscitation and who were or were at risk of deteriorating. They were hopeful that this would improve the chances of embedding the intervention given the significant crossover.   The forms were designed to provide a structured account to be held in the record in place of more generic documentation in the notes. However, doctors appeared uncomfortable if they were not also writing in the formal notes because the form itself was not "Hospital approved documentation". It was also frequently suggested that incorporating the forms into the hospital's electronic system would facilitate their use and reduce duplication of work and records.

| Acceptability of the intervention
Concerns about workload were particularly noticeable in the ED where doctors described having to make decisions quickly based on little information: decision making was fluid and responsive to new information.
We are too busy to complete forms, simple as that.

| Improving communication and documentation of decision
Several doctors stated the forms helped them to clearly set out a rationale for their decisions, and to communicate their reasoning to colleagues, leading to a shared understanding of the situation. Failing to provide accurate information on a referral form might provoke more irritation than simply failing to mention it in a telephone call or an entry in the medical record. Champions said they were unable to implement the PILs/FILs because they were focusing their efforts on embedding use of the clinician forms, which was a challenging enough task. Champions also expressed concern that the leaflets might cause distress to patients and families if provided without appropriate explanation and support.

| DISCUSSION
This study has demonstrated the feasibility of implementing an intervention to facilitate a transparent and ethically justifiable decisionmaking process around ICU admission in three contrasting hospital settings. However, uptake of the intervention across the sites was variable, from just under 20% at one site to 44% at another, and champions struggled to implement the PILs/FILs. Clinicians who used the referral and decision forms perceived them as acceptable and impacting positively on practice. Observation and interview data identified several challenges to and facilitators of the intervention uptake.
A key message was the need for organizational endorsement and adoption of the forms as official hospital documentation if the intervention was to become embedded in day-to-day clinical practice.
Implementation champions recommended an electronic format for the forms to facilitate and prompt its integration into decision-making and hospital documentation systems. Despite the challenges encountered, two of the three sites expressed an interest in taking the intervention or a variation of it forward as a service development.

| Implementation
May and colleagues have argued that when implementing an intervention, contextual factors should be seen as the "normal conditions" of practice into which the intervention needs to be integrated, rather than as confounders to be eliminated. 32 All our sites shared the context of a busy NHS hospital, which created challenges for the implementation champions both in finding time to promote the intervention and engaging their referring and ICU colleagues in both learning about and implementing the intervention. However, the size of the organization made a difference. For example, the highest uptake was at our smallest site, which was probably related to the ability of the champions to reach key individuals more easily and to maintain continuing awareness of the intervention among a smaller pool of colleagues. The presence of a CCOR team also facilitated intervention uptake. The CCOR are involved in the early identification of patients who may subsequently be referred to ICU and often act as a link between referring teams and ICU teams. The intervention therefore fitted directly into much of their daily practice, so they were well placed to remind both referring and ICU doctors to use the forms.
Timely reminders to referring doctors also encouraged appropriate use of the referral forms rather than a post hoc request to document the reasons for a referral that had already been made.
May also suggests that the plasticity of an intervention contributes to the ease with which it can be successfully integrated into different contexts. 24,33 The educational component of the intervention had some plasticity in that its format and the presentation of intervention materials could be modified to fit with established educational opportunities within each hospital. There was also some plasticity in the mode of delivery of the forms to referring teams. Being able to upload the referral form and include it in one specialty team's electronic referral system substantially improved knowledge of and uptake of the referral form. However, the content of the referral and decision support forms could not be amended, and this rigidity meant champions could not respond to criticisms or suggestions for change from their colleagues during implementation. Future implementation work will need to consider the balance between need for flexibility and maintenance of the core components of the framework.
A key concern of health care professionals regarding new interventions at organizational level relates to increased workload and duplication. 24 These concerns were expressed by doctors in our study, but interview data from our champions suggested possible ways to address this. Incorporating the forms into the hospital electronic record system would identify them as permanent formats for recording patient information removing the need for backup paper notes.
This was supported by the evidence from the single specialty team

| Impact of the intervention on decision making
As

| Strengths and weaknesses
The use of quantitative and qualitative methods meant we were able to triangulate the data. 46 We achieved diversity in our sample in terms of clinician specialty and grade, and sites reflected a range of NHS ICU settings. Specific cases were used during the interviews to avoid hypothetical scenarios or generalizations as much as possible. The position of the interviewer as a nonclinician meant that assumptions about patient care and decision making were probed for clarity and understanding without preconceptions about what "normal practice" might be. Doctors might have found it easier to discuss cases with a fellow clinician, but conversely, they may have felt less comfortable discussing or admitting to uncertainty with a clinical colleague.
Eight weeks may not have been a long enough period to embed the intervention in clinical practice, a point that was raised by our champions. However, a longer period may not have addressed the intervention being perceived as a time-limited research project, rather than an organization-driven quality improvement. We found the retrieval of clinical records within the study time period challenging.
Many doctors did not write their names clearly in the patient record, so we were unable to identify them to request an interview. Poor documentation in patient records also resulted in missing data, so our quantitative results should be interpreted with caution. We were nevertheless able to identify differences in form use, which were supported by our qualitative findings. We did not observe informal education/training, nor did we observe referrals as they happened. If the intervention is implemented more widely, future research should explore the process from the perspective of patients and families, whose voices are currently missing from the data.

| CONCLUSION
We have demonstrated that it is feasible to implement an intervention to support decision making around referral and admission to intensive care. When used, the decision support framework appeared to improve practice in terms of encouraging more transparent (better documented and communicated) and patient-centred decision making (greater attention to patient wishes and values), and improved communication between staff, but clinicians still found it difficult to articulate the process of balancing burdens and benefits of treatment. However, to achieve full implementation in practice, the decision support framework for referral and admission decisions needs to be embedded within normal referral pathways so that they become a routine aspect of practice for decision makers. We suggest that future implementation of the intervention should take place over a longer period with concurrent evaluation, ideally as a part of an action research 47,48 or a quality improvement 49 approach. This would require high-level organization support and allow an iterative process within each setting, allowing for response and adaptation to obstacles to embedding.