SEARCH

SEARCH BY CITATION

Keywords:

  • residency;
  • simulation;
  • curriculum;
  • emergency medicine

Abstract

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. Acknowledgments
  9. References

Objectives:  Utilization of simulation-based training has become increasingly prevalent in residency training. The authors compared emergency medicine (EM) resident feedback for simulation sessions to traditional lectures from an EM residency didactic program.

Methods:  The authors performed a retrospective review of all written EM conference evaluations over a 29-month period. Evaluation questions were scored on a 1–9 Likert scale.

Results:  Lectures and simulation accounted for 77.6 and 22.4% of the conferences, respectively. Scored means (±standard deviations [SDs]) were as follows: overall, lecture 7.97 ± 0.74 versus simulation 8.373 ± 0.44 (p < 0.01); Question 1, lecture 7.97 ± 0.74 versus simulation 8.40 ± 0.43 (p < 0.005); Question 2, lecture 7.92 ± 0.74 versus simulation 8.34 ± 0.48 (p < 0.01); Question 3, lecture 8.01 ± 0.77 versus simulation 8.26 ± 0.51 (p < 0.15); and Question 4, lecture 8.00 ± 0.75 versus simulation 8.42 ± 0.46 (p < 0.01). There was no longitudinal decay of scores.

Conclusions:  Emergency medicine residents scored simulation-based sessions higher than traditional lectures. The scores over time suggest that this preference for simulation can be sustainable long term. Residents perceive simulation as more desirable teaching method compared to the traditional lecture format.

“Simulation is a technique—not a technology—to replace or amplify real patient experiences with guided experiences, artificially contrived, that evoke or replicate substantial aspects of the real world in a fully interactive manner.”1 The pioneering work in health care simulation was conducted by Dr. David Gaba and colleagues in the anesthesia discipline. They created a simulated operating room environment incorporating high-fidelity manikins to provide a venue for residents to practice and learn without risk to patients.

Emergency medicine (EM) has broadly adopted this modality, and since 2003 when a survey of EM training programs was completed,2 the use of simulation in EM training has steadily proliferated. The use of simulation continues to gain acceptance in multidisciplinary areas of medicine, surgery, nursing, allied health, and other domains in health care. Adult learning depends primarily on engaging the learner in the learning activity and stimulating an experiential learning cycle for the student for the subject matter.3,4 Many features of simulation lend themselves to accomplishing this requirement. High-fidelity simulation (HFS) can maximize the educational environment to immerse the learner in the educational process. This step is crucial to developing motivation and perceived relevance within the learner to promote the self-perceived need to learn.3 Thoughtful debriefing sessions can further augment the experiential learning cycle for the adult learner. Procedural task-trainers permit deliberate practice of procedures in a repetitive fashion. This allows the learner to make mistakes without risk and to gain confidence in procedural skills.

The simulation literature is expanding in EM as well as other fields in medicine. Research in the effectiveness of simulation in education has demonstrated significant improvement in advanced cardiac life support and procedural training for internal medicine residents.5–8 Medical education experts have outlined the cardinal features of effective simulation education, feedback, repetitive practice, and integration into overall curriculum.9 Cognitive psychologists have provided insight into the concept of deliberate practice and how it relates to medical expertise.10 The surgical literature has outlined how procedural competency may be assessed and achieved.11–13

In another recent development, the Residency Review Committee for EM (RRC-EM) is encouraging programs to develop innovative methods for conference participation with a portion devoted to asynchronous education: “In response to 1 program’s request for an extensive simulation curriculum (approved by the RRC), RRC discussion was enthusiastic about encouraging educational innovation (active learning formats) within programs.”14

Although the utilization of simulation as an instructional method in EM residency training is increasing, it is not an integral part of the educational repertoire in a significant number of training programs. McLaughlin et al.2 reported in 2006 that 29% of EM programs utilize HFS in their resident educational program. Of those, EM residents are using simulation for education or assessment purposes every 1–2 weeks at 3 (8%) programs, every 1–4 months at 16 (42%) programs, and yearly at 9 (24%) programs. Simulation is not regularly used at 10 (26%). The simulation curriculum is described as “no formal curriculum” or “initial development” in 60% of programs.2

The Center for Simulation Technology and Academic Research (CSTAR) at Evanston (IL) Hospital has been providing HFS training for medical professionals since 2002. The hospital is one of the secondary training sites for the EM residency at McGaw Medical Center of Northwestern University. Conferences are scheduled at the site every other week during the academic year. Simulation was integrated into the educational curriculum beginning in 2004.

We examined resident feedback for simulation sessions and traditional lectures from the EM didactic curriculum in a longitudinal fashion. We evaluated the residents’ perceptions of the overall quality and desirability of simulation-based sessions versus the traditional format. The null hypothesis was that residents did not perceive a difference in quality or desirability between simulation and the traditional lecture formats.

Methods

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. Acknowledgments
  9. References

At Evanston Northwestern Healthcare, the study was exempt from informed consent requirements based on our institutional review board criteria. A retrospective review of all EM conference evaluations from the EM residency didactic sessions from the period of July 2004 to November 2006 was performed. We included all submitted resident conference evaluations from that period.

Conferences were conducted twice a month in 3-hour sessions. EM residents were divided into two groups: Postgraduate Years (PGYs) 1 and 2 and PGY 3 and 4. The PGY 1/2 group participated in simulation sessions for the first half of the conference, while the PGY 3/4 group were engaged in another educational activity. The groups would then switch midway through the 3-hour conference session.

The traditional lecture format consisted of lectures, given by EM attending faculty, guest non-EM faculty from other specialties who lectured on topics related to their expertise, and EM residents. Lecture and simulation education were focused on a predetermined rotating topic-based curriculum, derived from the Model of Clinical Practice of EM.15

Simulation sessions were conducted by a core group of EM attending faculty at CSTAR. These sessions were a mixture of case-based simulation scenarios utilizing high-fidelity simulators (conducted by one to two attending faculty attending each session) and procedural training sessions utilizing task trainers (conducted by four to six attending faculty each session). Like lecture topics, these sessions pertained to the curricular topic for the month.

A convenience sample of surveys representing the opinions of all of the residents present for the required conference sessions was obtained. (Survey completion was not mandatory, and the sample depended on the residents taking the time to respond after each conference offering.) Four questions were asked for each of the conferences: 1) lecturer/simulation session effectively addressed goals and objectives; 2) lecturer/simulation session increased my knowledge and understanding of the topic; 3) lecturer/simulation session was clear and easy to follow; and 4) lecturer slides and handouts/simulation sessions were clear and easy to follow. The data were collected as numerical responses on a 1–9 Likert scale. The evaluation forms were anonymous. Total scores and means were calculated for each conference, and average scores were compared between the two types of sessions using two-sample t-tests.

Results

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. Acknowledgments
  9. References

There were a total of 125 conference hours during the study period. The lectures were given by EM attending faculty (49%), guest non-EM faculty (24%), and EM residents (27%).

Lectures and simulation accounted for 77.6 and 22.4%, respectively. The results of survey are summarized in Table 1. There appeared to be a statistically significant difference in residents’ preferences for simulation-based conferences when comparing the Likert-based statements, except for the one specifically addressing whether the lecture/simulation conference was “clear and easy to follow.” When plotted longitudinally, there was an initial rise in scoring and no decay of scores over time (Figure 1).

Table 1.   Survey Question Results
QuestionTypenMean (±SD)p-Value
  1. SD = standard deviation.

1. The lecturer effectively addressed the goals and objectives.Lecture977.97 (0.74)0.0041
Simulation288.40 (0.43)
Diff (1–2) −0.43 (0.68)
2. The lecture increased my knowledge and understanding of the topic.Lecture977.92 (0.74)0.0052
Simulation288.34 (0.48)
Diff (1–2) −0.42 (0.69)
3. The lecturer was clear and easy to follow.Lecture978.01 (0.77)0.1290
Simulation248.26 (0.51)
Diff (1–2) −0.26 (0.73)
4. The slides/handouts were clear and easy to follow/the time allotted was sufficient for demonstration and practice of the procedure.Lecture978.00 (0.75)0.0057
Simulation288.42 (0.46)
Diff (1–2) −0.42 (0.70)
TotalLecture977.97 (0.74)0.0074
Simulation288.37 (0.44)
Diff (1–2) −0.40 (0.68)
image

Figure 1.  Longitudinal rating of traditional lecture format versus simulation.

Download figure to PowerPoint

Discussion

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. Acknowledgments
  9. References

Adult learning theory advocates active engagement of the learner as a more effective methods of knowledge acquisition. Active engagement increases the likelihood of a positive commitment to the instructional exercise and of enjoyment of the experience. Our findings reject the null hypothesis and support the assumption that simulation was a more desirable method of learning EM curriculums at this institution.

We noted that the inaugural simulation session was rated more favorably, but then dropped below the ratings given to lecture format from August 2004 to 2005. This initial trend may represent the classic “honeymoon period” where implementing a new novel technology such as simulation-based training receives high marks initially and decays over time as the newness of the modality diminishes. However, with continued experience over time, simulation scores improved to a higher overall rating than lecture format longitudinally, and the scoring differential was statistically significant for the entire period of review.

We interpret the trajectory of the graph to represent the “learning curve” for our simulation center as we worked through our own “experiential learning cycle” of how to conduct simulation sessions. As we deliberately practiced presenting our simulation curriculum to the resident complement, we reflected on and incorporated the feedback from the resident evaluations into subsequent sessions. We troubleshot technical difficulties and content presentation. In short, we learned how to make our simulations better by answering the following questions from the planning phase of the experiential learning cycle”What have I learned?” and “How will I approach such a [teaching session] next time?” This was an important step in helping us to generate a sustainable long-term positive evaluation of our simulation-based educational sessions.

Limitations

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. Acknowledgments
  9. References

The analysis was retrospective and the data were self-reported. This study employed a convenience sample rather than a random sample and does not provide data regarding the proportion of respondents versus nonrespondents. As such, we do not know if the nonrespondents differed significantly from respondents in their general evaluation of the simulation sessions. It is possible that nonrespondents were less likely to fill out the evaluation forms because they did not favorably rate either the simulation sessions and/or the general didactic curriculum as a whole. Moreover, there was variability in the number of responses for each of the individual sessions, due to the fact that only residents who were in attendance at the specific conference sessions were eligible to fill out a conference evaluation form. These factors could have significantly influenced the scores.

Another point to consider is that the results obtained are not necessarily generalizable across all institutions performing simulation. There is variability in the time invested and degree of integration of simulation into the didactic curriculum across programs. As we discovered, we had to conquer our own learning curve before the simulation sessions had improved to the level at which we wanted them conducted. This required repetitive practice, reflection, and continual revisions.

Finally, although the survey demonstrated a preference for simulation, this does not necessarily translate into improved attainment of educational objectives. Nor does this study imply that simulation is a better method of knowledge acquisition or knowledge retention. It is also not possible to directly link this preference to improved physician performance. These are areas that demand further exploration.

Conclusions

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. Acknowledgments
  9. References

Simulation is one of many educational techniques to facilitate resident education. Simulation was rated by EM residents at our institution to be better than traditional lectures at increasing knowledge and understanding of a topic and presenting material directly applicable to clinical practice. When integrated into the curricular structure in a systematic fashion, this preference for simulation-based education can be sustained long-term.

Acknowledgments

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. Acknowledgments
  9. References

We would like to thank Laura Seal for her tremendous assistance.

References

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Limitations
  7. Conclusions
  8. Acknowledgments
  9. References