Building a Simulation-based Crisis Resource Management Course for Emergency Medicine, Phase 1: Results from an Interdisciplinary Needs Assessment Survey

Abstract

Introduction:  Emergency department (ED) resuscitation requires the coordinated efforts of an interdisciplinary team. Human errors are common and have a negative impact on patient safety. Although crisis resource management (CRM) skills are utilized in other clinical domains, most emergency medicine (EM) caregivers currently receive no formal CRM training.

Objectives:  The objectives were to compile and compare attitudes toward CRM training among EM staff physicians, nurses, and residents at two Canadian academic teaching hospitals.

Methods:  Emergency physicians (EPs), residents, and nurses were asked to complete a Web survey that included Likert scales and short answer questions. Focus groups and pilot testing were used to inform survey development. Thematic content analysis was performed on the qualitative data set and compared to quantitative results.

Results:  The response rate was 75.7% (N = 84). There was strong consensus regarding the importance of core CRM principles (i.e., effective communication, team leadership, resource utilization, problem-solving, situational awareness) in ED resuscitation. Problems with coordinating team actions (58.8%), communication (69.6%), and establishing priorities (41.3%) were among factors implicated in adverse events. Interdisciplinary collaboration (95.1%), efficiency of patient care (83.9%), and decreased medical error (82.6%) were proposed benefits of CRM training. Communication between disciplines is a barrier to effective ED resuscitation for 94.4% of nurses and 59.7% of EPs (p = 0.008). Residents reported a lack of exposure to (64.3%), yet had interest in (96.4%) formal CRM education using human patient simulation.

Conclusions:  Nurses rate communication as a barrier to teamwork more frequently than physicians. EM residents are keen to learn CRM skills. An opportunity exists to create a novel interdisciplinary CRM curriculum to improve EM team performance and mitigate human error.

Emergency medicine (EM) shares many common features with aviation; the demand for rapid, high-stakes decisions in stressful, complex, and rapidly evolving situations can compromise effective team performance and poses a threat to safety.^1,2 Research by the National Aeronautics and Space Administration has concluded that the majority of commercial aviation accidents result not from technical or mechanical error, but from breakdowns in communication, leadership, and teamwork among flight crews.^1,3,4 To address this, the aviation industry turned to the study of high-reliability teams in combat aviation to develop crew resource management (CRM), a flight simulator–based team training program for pilots,⁵ designed as a series of countermeasures to detect, avoid, and mitigate human error during critical in-flight events.⁶

Like aviation, hospital-based health care is classified as a high-hazard industry.⁷ While effective medical team performance requires individual task work, this alone is not sufficient to resolving dynamic crisis situations—teamwork skills are also required.^8–10 Anesthesia has been an early adopter of human factors theory in resuscitation training,¹¹ incorporating elements of aviation team training in the development of anesthesia crisis resource management (ACRM).^12–14 This team-based approach uses high-fidelity patient simulation to teach the nontechnical skills required to respond to operating room (OR) crises (Table 1). ACRM training emphasizes clear leadership and communication, situational awareness, problem-solving skills, and shared mental models of resuscitation processes to facilitate an individual team member’s contribution toward a common team goal.²

Table 1.   Key Behaviors in CRM

CRM = crisis resource management.

Leadership skills

• Stays calm and in control during crises

• Prompt and firm decision-making

• Maintains global perspective (“big picture”)

Situational awareness

• Avoids fixation errors

• Reassesses and reevaluates situation constantly

• Anticipates likely events

Communication skills

• Communicates clearly and effectively

• Uses directed verbal/nonverbal communication

• Listens to team input

Problem-solving

• Organized and efficient problem solving approach

• Quick in implementation (concurrent management)

• Considers alternatives during crisis

Resource utilization

• Calls for help appropriately

• Utilizes resources at hand appropriately

• Prioritizes tasks appropriately

Preliminary work has been undertaken to examine the feasibility and benefits of introducing CRM training for EM resuscitation teams.^15–18 While many elements of ACRM may be applicable to EM, the extent to which there is overlap or disparity between traditional ACRM paradigms and the needs of ad hoc interdisciplinary emergency department (ED) teams has not been formally explored. Multiple patients, chaotic and distracting ambient conditions, and new or constantly changing team composition are examples of ways in which EM practice differs from the controlled environment of the OR.

Military and aviation research has produced the recommendation that the development of team training programs be guided by the results of an a priori team task analysis to establish the knowledge, skills, and attitudes (KSA) needed for effective team performance in a given domain.^19,20 One component of team task analysis involves performing a needs assessment survey to define specific training objectives that can be linked to the KSAs required for effective team performance.²⁰ The goal of the present study was to assess and compare knowledge and attitudes regarding CRM training among staff emergency physicians (EPs), nurses, and postgraduate medical trainees (residents), at two Canadian academic teaching hospitals.

Methods

Study Design and Population

This was a Web-based survey of EPs, administrators, nurse managers, and nurse educators from two academic teaching hospitals in Toronto. Both hospitals are designated core teaching sites for specialist training in EM and have regional trauma center designations. Institutional research ethics board approval was obtained before survey distribution and the requirement for informed consent was waived, because consent was implicit in survey completion.

Survey Content and Administration

The survey instrument was developed in three stages. In Stage 1, we held two focus groups involving EPs, administrators, nurse managers, and nurse educators from the two hospitals. Focus groups explored EM-specific themes related to resuscitation team training using a facilitated semistructured format. Participants were asked to reflect on their own practice to conceptualize effective team function, as well as challenges to effective team performance, with particular attention to human and system factors. Thematic analysis of focus group transcripts was then used to derive the content of the survey tool. In the second stage, the on-line instrument was developed, which included a blend of 5-point Likert scale and open-ended listing questions. Inquiry focused on three major themes: resuscitation experience, challenges to effective ED resuscitation, and resident education. Open-ended questions were included to assess whether respondents identified important team management skills that are not encapsulated by traditional ACRM paradigms. The third stage involved piloting the survey to a small cohort of EM experts to refine usability as well as face and content validity. Reviewers had experience and expertise in academic EM practice, medical education, survey methodology, and CRM. Based on recommendations from the reviewers, minor changes were made to the body of the survey.

The final survey included 22 content and 3 demographic questions (see Data Supplement S1, available online at http://www.blackwell-synergy.com/doi/suppl/10.1111/j.1553-2712.2008.00185.x/suppl_file/acem_185_sm_DataSupplementS1.pdf). The survey was distributed using commercially available on-line software²¹ to all current EM staff, residents, and a select cohort of experienced EM nurses (independently identified by nurse managers as those with substantial EM experience) at the two sites. Distribution was carried out using a modification of the Dillman total design method for surveys:^22,23 after e-mail notification, participants were sent a Web link to the on-line survey and then two reminder notices spaced 1 week apart. Respondent anonymity was maintained throughout collection and analysis. The survey was open for responses from late November until mid-December 2007.

A single data abstractor (CMH) entered survey results into a computerized database. Regular meetings were held between one of the study investigators (CJD) and the data abstractor to clarify any issues arising from data collection.

Data Outcomes and Analysis

Descriptive statistics were calculated to summarize the quantitative data using Excel software (Microsoft Corp., Redmond, WA). A two-tailed Fisher’s exact test was used to examine the significance of the association between type of training (MD vs. RN) and responses in a 2 × 2 contingency table with small independent samples. For the purpose of analysis, residents were identified as either junior (Postgraduate Years 1 and 2) or senior (Postgraduate Years 3–5) by year of training. Thematic content analysis²⁴ was performed on text generated by the following open-ended question: “Regarding team performance, describe three things that you feel could be improved during patient resuscitation in the ED.” To facilitate comparison between data sets, qualitative data were converted into a quantitative format by counting the frequency with which responses were thematically linked with one of the five CRM domains (identified a priori to be communication, leadership, problem-solving, situational awareness, and resource management).²⁵ For example, “better recognition of who is in charge” would be coded under “leadership” according to the framework adopted for analysis. In addition, qualitative data were analyzed for nascent or emerging themes that lie outside traditional CRM paradigms, but may be relevant to EM resuscitation. Issues related to data coding were discussed and resolved by consensus in a meeting between study investigators before the final analysis.

Results

Surveys were emailed to 111 participants. Responses were received from 72.3% (35/48) of staff physicians, 61.3% (19/31) of nurses, and 93.8% (30/32) of residents, yielding an overall response rate of 75.5% and a survey completion rate of 94.0%. Of the staff physicians, 25.7% were female, compared to 84.2% of nurses and 66.7% of residents. The majority of respondents (51.2%) reported being involved in 2 to 5 resuscitations per month, although a significant number (31.7%) participated in 6 to 10 or more (Table 2).

Table 2.   Demographics and Resuscitation Experience

Characteristic	n (%)
PGY = postgraduate year.
Staff physician	35 (41.6)
Resident
PGY 1–2	12 (14.3)
PGY 3–5	18 (21.4)
Registered nurse	19 (22.6)
Gender
Female	45 (53.5)
Male	39 (46.4)
Resuscitations per month
0–1	12 (14.3)
2–5	42 (50.0)
6–10	17 (20.2)
>10	9 (10.7)
Unable to quantify	2 (2.4)

Team Resuscitation Experience

There was strong consensus between staff physicians, nurses, and residents regarding the importance of ACRM behaviors in team resuscitation. Effective team leadership (100%), communication (100%), resource utilization (86.4%), problem-solving (98.8%), and situational awareness (98.8%) were rated as important or very important determinants of resuscitation outcomes by the vast majority of respondents (Table 3). A total of 84.1% reported having been directly involved in resuscitations where they felt improved CRM behaviors would have positively influenced the outcome.

Table 3.   Relevance of CRM Skills to ED Resuscitation

	Somewhat Unimportant/ Unimportant, %	Neutral, %	Very Important/ Important, %	n
CRM = crisis resource management; ED = emergency department.
Which of the following team management skills do you feel contributes to the outcome of ED resuscitation?
Effective communication	0.0	0.0	100	81
Leadership	0.0	0.0	100	81
Resource utilization	2.5	11.1	86.4	81
Problem-solving	0.0	1.2	98.8	81
Situational awareness (re-assessment, re-evaluation)	0.0	1.2	98.8	81
When adverse events occur during resuscitation, which of the following challenges do you believe are important contributing factors?
Need for rapid decisions	3.8	2.5	93.8	80
Complex and rapidly evolving situations	1.3	5.1	93.4	79
Stress	3.8	15.0	81.2	80
Challenges in coordinating team actions	1.3	5.0	93.8	80
Poor communication	0.0	3.8	96.2	79
Authority gradients (reluctance to question those with seniority)	8.8	10.0	81.2	80
Failure to establish clear goals and priorities	7.5	16.3	76.2	80
Conflicting occupational cultures	10.0	28.8	61.2	80
Lack of medical knowledge or training	7.5	28.8	63.8	80
Ineffective procedural skills	8.8	22.5	68.8	80

Challenges to Effective ED Resuscitation

Important factors contributing to adverse events were judged to be: the need for rapid decisions, poor communication, challenges in coordinating team actions, stress, and conflicting occupational cultures (Table 3). Multiple patients (61.3%), frequent interruptions or distractions (55%), and the demand to maintain pace and efficiency (45%) were cited as important barriers to effective team performance in the ED. Nurses placed significantly greater importance on communication between disciplines (94.4%) compared to physicians (59.7%; p = 0.008; Figure 1). The majority of respondents agreed that CRM training could have a positive impact on job stress (82.5%), incidence of medical error (97.5%), efficiency of patient care (89.9%), and interdisciplinary collaboration (95%).

Figure 1.  Perceived barriers to effective team performance in the emergency department (ED).

Thematic content analysis of the open-ended question examining ways to improve ED team performance yielded a total of 206 responses from 76 survey participants. Themes derived from ACRM were central to critiques of team performance by all three groups under study, with 41.4% of staff physician, 57.1% of nursing, and 70% of resident responses clustered around one of the five core domains. Emerging themes included administrative issues (documentation, use of protocols, available hospital resources) and a lack of ongoing resuscitation training (team debriefings, resuscitation drills). Seventy-two percent of staff physicians implicated administrative issues as a barrier, compared to 15.4% of residents and 22.2% of nurses. A lack of medical knowledge (e.g., recognition of sepsis) or procedural skills (e.g., intubation) were identified as problematic less frequently than human factors in the thematic analysis (Table 4).

Table 4.   Categories of Responses to the Open-ended Question “Regarding Team Performance, Describe Three Things That You Feel Could Be Improved during Patient Resuscitation in the ED.”

	Staff Physician, n = 32 (%)	Resident, n = 26 (%)	Nurse, n = 18 (%)
CRM = crisis resource management; ED = emergency department.
CRM themes
Communication	13 (40.6)	20 (76.9)	9 (50)
Leadership	14 (43.8)	13 (50)	14 (77.8)
Problem-solving	2 (6.2)	4 (15.4)	1 (5.6)
Situational awareness	4 (12.5)	7 (26.9)	2 (11.1)
Resource management	8 (25)	7 (26.9)	6 (33.3)
Administration	23 (71.9)	4 (15.4)	4 (22.2)
Resuscitation training	7 (21.9)	4 (15.4)	3 (16.7)
Medical knowledge	3 (9.4)	0 (0)	3 (16.7)
Procedures	1 (3.1)	1 (3.8)	0 (0)
Other	10 (31.2)	6 (23.1)	6 (33.3)

Resident Education

When asked if they felt equipped with the necessary team management skills to lead an interdisciplinary team during ED resuscitation, the majority of senior residents agreed (56.3%) with this statement, while the majority of junior residents disagreed (58.3%; Table 5). Most residents (60.7%) did not have easy access to materials to help them learn how to manage or work within a resuscitation team, and only 25% reported having formal exposure to CRM at any point in their training. The vast majority of residents (96.4%) reported interest in participating in an educational session designed to teach team management skills, with a hands-on format using high-fidelity human patient simulation emerging as the overwhelming first choice (96.4%) for this purpose.

Table 5.   Resident Team Training Attitudes and Experience

	Strongly Disagree/Disagree, %	Neutral, %	Strongly Agree/Agree, %	n
CRM = crisis resource management; EM = emergency medicine. *Junior resident = postgraduate years 1 and 2; Senior resident = postgraduate years 3–5.
I feel well-equipped with the necessary team management skills to lead a team during complex resuscitations*
Junior resident	58.3	41.7	0.0	16
Senior resident	0.0	31.3	56.3	12
I have easy access to materials to help me learn how to work within or manage a team during times of medical crisis	60.7	17.9	21.4	28
I have had formal exposure to education in CRM	64.3	10.7	25	28
I would be interested in participating in a half-day session designed to teach team management and resuscitation team training skills	0.0	3.6	96.4	28
I believe high-fidelity human patient simulation is a useful tool for teaching EM residents	0.0	7.1	92.9	28

Discussion

The Institute of Medicine’s 1999 report “To Err Is Human: Building a Safer Health System” emphasized the role of human and system factors in medical error and advocated for the application of aviation team training theory and CRM to health care as a means to improve patient safety.²⁶ CRM training is not uniform throughout the aviation industry, as different programs have been developed to suit the needs of diverse commercial and military applications.² A similar approach is warranted in health care, where a team task analysis can be used to guide the development of CRM programs that address local and domain-specific team training requirements.²⁰

This needs assessment survey demonstrates an endorsement of core ACRM principles for resuscitation team training among an interdisciplinary cohort of EM workers. The majority of participants believe that human factors play a more important role in determining ED patient outcomes than do medical knowledge or procedural skills. This conclusion is further supported by the convergence of qualitative and quantitative data sets: themes derived from open-ended responses emphasized the importance of human and system factors in improving EM team performance, in keeping with the quantitative results. Furthermore, formal team training education and structured performance feedback were identified as effective ways to improve team dynamics. While the importance of content knowledge and procedural skill for determining resuscitation outcomes is not in dispute, these data suggest that current training programs for EM physicians and nurses are effective at instilling relevant knowledge and skills, but are deficient in team-based instruction that includes human factors.

In 2003, Reznek et al.¹⁵ described participant perceptions of a novel simulation-based CRM course for EM residents. The project expanded on traditional ACRM paradigms by adding triage/prioritization, efficient management of multiple patients, and effective coping with multiple disruptions/distractions to the list of core competencies. While these concepts have intuitive relevance to EM team training, it is not clear how they were derived. Our study appears to validate their inclusion in EM CRM education, as the majority of respondents across disciplines identified these issues as important barriers to effective teamwork in the ED. Our results elaborate on these themes by comparing responses across disciplines—communication between disciplines and coping with administrative concerns were highly relevant issues for nurses and staff physicians, respectively. We hypothesize that this finding highlights a unique element of EM team training: perhaps more than anywhere else, EM physicians and nurses interface with multiple disciplines and levels of administration when coordinating the care of critically ill patients. Team training strategies that address these issues in an interdisciplinary format may therefore be relevant to developing EM-specific CRM programs.

EM residents reported both a lack of exposure to and a keen interest in learning CRM skills using human patient simulation. Senior residents had a higher degree of confidence in managing interdisciplinary resuscitation teams compared to their junior resident counterparts, suggesting that CRM initiatives should be introduced early in postgraduate training to maximize their impact. This finding is congruent with a 2007 survey by Hayes et al.²⁷ of Canadian internal medicine residents: an increased level of training and receiving performance feedback was associated with residents’ sense of adequacy in leading cardiac arrest teams, according to a multivariate regression model. In our study, residents at all levels of training were less likely than staff physicians to identify administrative issues as problematic. One possible explanation for this is the degree to which EM residents are sheltered from systems-level decision-making during resuscitation by the presence of a staff physician, who assumes the role of coordinating care at the hospital level. Although further work is needed to explore this finding in detail, CRM programs designed for EM residents may wish to explore strategies to enhance interdisciplinary and interdepartmental coordination and resource utilization as a component of resuscitation training.

Over the past decade, heavy emphasis has been placed on competency-based assessment for postgraduate medical trainees. The Royal College of Physicians and Surgeons of Canada (RCPSC) has defined seven key competencies (the CanMEDS roles), while the American Council for Graduate Medical Education (ACGME) endorses a six-domain model for competency-based medical education.^28,29 CRM training using human patient simulation provides an opportunity for focused instruction and assessment in the CanMEDS roles communicator, collaborator and manager, while encompassing ACGME-endorsed practice-based learning, interpersonal and communication skills, and systems-based practice competencies. Furthermore, pairing realistic resuscitation scenarios with focused performance feedback via group debriefing sessions led by a skilled instructor ensures that the educational benefit of simulation-based training is maximized.³⁰

Results from our needs-assessment survey suggest several ways in which established ACRM approaches can be tailored for an EM audience. The simulation “script” may focus on interdisciplinary communication, working with consultants, or coordinating multiple administrative issues, in addition to the ACRM principles of leadership, communication, situational awareness, problem-solving, and resource management. Coping with disruptions and distractions, managing multiple patients, and triage and prioritization may also be included in EM-focused CRM scenarios.¹² For example, in managing a critically ill patient who has ingested a large amount of an unknown medication, an EM-focused CRM scenario may call upon a resuscitation team to enlist the help of a poison control center and critical care (consultation and administration), manage a situation in which a team member disagrees with the treatment priorities (interdisciplinary communication, leadership), or be called away to assess a patient who has collapsed at the front desk (triage/prioritization). Discussing the extent to which these unique elements of EM practice pose a threat to effective team coordination, and strategies targeted at mitigating their effects on patient care, becomes the focus of facilitated group debriefing sessions. Furthermore, we believe that adopting an interdisciplinary format for EM team training that includes physicians, nurses, paramedics, and technicians will improve communication and collaboration between disciplines and assist team members in developing shared mental models of resuscitation processes. The latter point is particularly important, as the extent to which team and task-based mental models are “shared” by team members predicts subsequent enhanced team performance in empirical studies.³¹

CRM is not a “single-dose” intervention, but one that requires repeated practice, feedback, and evaluation to achieve expertise.¹¹ High-fidelity human patient simulation allows for reproducible, longitudinal exposure to team training principles in a controlled environment that poses no threat to patients. Simulation-augmented CRM training that includes formal feedback and performance debriefings has advantages over didactic instruction by increasing engagement in the learning process,³² improving knowledge acquisition and knowledge transfer,^30,33 and engendering positive attitudinal changes that reflect improved teamwork and team performance.^11,34 Results from the MedTeams Project have demonstrated that aviation-based team training can have a positive impact on team behaviors and error rates in the ED, with the important caveat that aviation principles need to be adapted to suit the specific needs of the health care industry.³⁵ Finally, improved individual and team behaviors must be accompanied by changes in hospital organizational culture that reflect the inevitability of human error and the adoption of systemic approaches to error management.¹

Limitations

Survey distribution was limited to two sites from the same academic environment, thus leading to a limited overall number of survey participants. Results may therefore not be generalizable to other academic or community EM environments. A single data abstractor was involved in interpreting and categorizing the open-ended responses, introducing the possibility of bias in the thematic analysis. Future work should address these issues by performing interinstitutional or nationwide surveys and utilizing multiple data abstractors to facilitate comparisons between interpretations. In addition, the optimal timing for CRM education (undergraduate, postgraduate, and/or beyond), the effect of high- versus low-fidelity simulation on team training and performance, and optimal methods for evaluating changes in individual and team behavior arising as a result of CRM training should be the focus of future investigation.

Conclusions

To our knowledge, this study represents the first attempt to quantify the attitudes and needs of EM workers regarding CRM training. Our needs assessment survey of staff EPs, residents, and nurses at two academic teaching hospitals has identified that although aviation-style CRM education is perceived to be highly relevant to EM resuscitation training, team factors are incompletely addressed in current instructional approaches. Our results suggest several ways in which existing simulation-based CRM programs can be modified to include EM-relevant crisis management themes. Improving communication between disciplines, addressing conflicting occupational cultures, and defining strategies for managing multiple administrative issues during resuscitation are important topics to include in EM-focused CRM initiatives. Residents demonstrated a keen interest in learning CRM skills using high-fidelity human patient simulation.

Acknowledgments

The authors thank Vicki LeBlanc, PhD, for her editorial assistance, and the residents, staff physicians, nurses, and nurse administrators of Sunnybrook Health Sciences Centre and St. Michael’s Hospital for their assistance with and participation in this project.