ACADEMIC EMERGENCY MEDICINE 2011; 18:559–562 © 2011 by the Society for Academic Emergency Medicine
Objectives: Oral examination is a method used to evaluate emergency medicine (EM) residents and is a requirement for board certification of emergency physicians. Second Life (SL) is a virtual three-dimensional (3-D) immersive learning environment that has been used for medical education. In this study we explore the use of SL virtual simulation technology to administer mock oral examinations to EM residents.
Methods: This was a prospective observational study of EM residents who had previously completed mock oral examinations, participating in a similar mock oral examination case scenario conducted via SL. EM residents in this training program completed mock oral examinations in a traditional format, conducted face to face with a faculty examiner. All current residents were invited to participate in a similar case scenario conducted via SL for this study. The examinee managed the case while acting as the physician avatar and communicated via headset and microphone from a remote computer with a faculty examiner who acted as the patient avatar. Participants were surveyed regarding their experience with the traditional and virtual formats using a Likert scale.
Results: Twenty-seven EM residents participated in the virtual oral examination. None of the examinees had used SL previously. SL proved easy for examinees to log into (92.6%) and navigate (96.3%). All felt comfortable communicating with the examiner via remote computer. Most examinees thought the SL encounter was realistic (92.6%), and many found it more realistic than the traditional format (70.3%). All examinees felt that the virtual examination was fair, objective, and conducted efficiently. A majority preferred to take oral examinations via SL over the traditional format and expressed interest in using SL for other educational experiences (66.6 and 92.6%, respectively).
Conclusions: Application of SL virtual simulation technology is a potential alternative to traditional mock oral examinations for EM residents.
Simulation-based education and assessment strategies have been increasingly applied in medical education.1 Advances in computer processing and internet connectivity have created opportunities for remote simulated virtual experiences. Second Life (SL) is an Internet-based, virtual three-dimensional (3-D) world (Web 2.0/Web 3-D dynamic computer-based application produced by Linden Labs, Inc., Atlanta, GA) in which individuals from anywhere in the real world can interact.2,3
This immersive learning environment has been used successfully for medical and public health education.3 SL has been used to model doctor–patient relationships, teach clinical diagnosis, train for disasters, and even virtually tour the human anatomy.4–6 However, educational research on 3-D virtual worlds and their effect on outcomes is limited.2
In SL, users are represented in a virtual world by their avatars. An avatar is an online, self-created, animated characterization of the user that can act in any role (doctor, patient, nurse, or teacher) and perform programmed tasks (Figure 1).
An oral board examination offered semiannually by the American Board of Emergency Medicine is the second step in the board certification process for residency-trained emergency physicians. In the current format, examiners and candidates must travel to Chicago, and patient cases are administered face to face to assess candidates’ competencies in the practice of emergency medicine (EM). Residents frequently prepare during residency by practicing “mock” oral examinations. Residents at our academic EM residency program participate in annual mock oral examinations conducted in the traditional format (face-to-face interaction with proctor) in preparation for the oral certification examination. In this study we explore the use of SL virtual simulation technology to administer mock oral examinations to EM residents.
This study was a prospective, observational study that was granted institutional review board exemption from informed consent requirements.
Study Setting and Population
The study setting was a university-based, academic EM residency training program. All current EM residents (n = 36) were invited to participate in an additional oral examination case conducted via SL virtual interface.
The case scenario was a patient with ectopic pregnancy (identical to a case that was administered in the traditional format within 1 month prior to the SL examination). The examinee managed the patient case while acting as the physician avatar in SL. The faculty proctor acted as the patient avatar. The examinee and proctor were in remote physical locations and communicated via headset and computer within SL.
The SL program was downloaded to the computers designated for use in this study free of charge.7 Two EM faculty proctors administered all SL examinations after limited training (2 hours of self-directed use and 1 hour of case rehearsal). Faculty proctors used a generic login for the patient avatar. The examinee was provided instructions with generic login information (screen name and password for the physician avatar) and instructions for navigation and communication in the SL hospital and examination room.
Examinees (physician avatar) verbally interviewed the proctor (patient avatar) for historical details and physical examination findings. A collection of pertinent diagnostic data was created in PowerPoint with individual slides subsequently loaded onto an image viewer in SL (Figure 1). The image viewer was controlled by the faculty proctor, allowing diagnostic data (initial and repeat vital signs, laboratory reports, and diagnostic imaging including ultrasound) to be displayed in the virtual examination room when requested by the examinee or at appropriate times during the case.
Participants were surveyed regarding their experience with the traditional and virtual oral examinations using a five-point Likert scale (Table 1). Additional questions following the survey addressed the students’ perceptions of the utility of SL in medical education and elicited subjective comments from participants.
|% Agree or Strongly Agree||% Neutral||% Disagree or Strongly Disagree|
|It was easy to login to SL using the instructions provided.||92.6||7.4||0|
|It was easy to use my avatar to navigate in SL.||96.3||3.6||0|
|It was easy to locate the virtual examination room and patient.||96.3||3.7||0|
|I felt comfortable using the computer to communicate during the SL mock examination.||100.0||0||0|
|I felt comfortable using the microphone to communicate during the SL mock examination.||92.6||7.4||0|
|SL provided a realistic patient encounter for a mock examination.||92.6||7.4||0|
|SL provided a more realistic patient encounter than traditional mock oral examinations.||70.4||25.9||3.7|
|SL mock examination was intimidating.||11.1||7.4||81.5|
|SL provided a less intimidating experience than traditional mock oral examinations.||63.0||37.0||0|
|SL mock examination was fair and objective.||100.0||0||0|
|SL mock examination provides a more fair and objective evaluation of my patient management than traditional oral examinations.||40.7||59.3||0|
|SL examination is an efficient way to complete mock examinations.||100.0||0||0|
|I would prefer to complete more of my oral examination requirements using a virtual patient and interface than a traditional format.||66.7||33.3||0|
|SL should be used for other medical education experiences.||92.6||7.4||0|
Descriptive data are reported as percentages.
Twenty-seven EM residents participated in the virtual examination (results in Table 1). None of the examinees had used SL previously. Examinees found SL to be user-friendly and easy to log into. Examinees were able to navigate within SL using the physician avatar and locate the examination room with the instructions provided. All examinees felt comfortable communicating with the faculty proctor via remote computer. Nearly all examinees felt comfortable communicating with the faculty proctor via microphone and headset.
Most examinees found the SL encounter to be realistic, and many felt that the virtual format was more realistic than the traditional format. Participants commented that they “were more able to use visual clues (as in real life),”“can see what has been done (i.e., there are my fluids, there’s the monitor),”and “like seeing the labs, images, and data on the screen.”
Very few examinees found the virtual format intimidating, and a majority found it less intimidating than the traditional format. Specific comments were “less intimidating when the proctor is not so close, able to gather your thoughts easier and stay organized” and “not face-to-face with stranger you don’t know and awkward.” All examinees expressed that the virtual examination was fair and objective and was conducted efficiently. A majority of participants expressed a preference for practicing oral examinations via SL, and residents were interested in using SL for other educational experiences, such as mock codes and case conferences.
When queried regarding potential disadvantages, some participants suggested drawbacks to a lack of face-to-face interaction. Expressed concerns were that they might receive “no cues from the examiner (nonverbal as feedback)” or it is “perhaps harder to assess physician’s interpersonal skills.”
Concerns about the process of traditional oral examination have been cited by EM residents and recent graduates preparing for this examination.8 Attempts are made to replicate a patient encounter in the ED; however, the traditional oral examination process remains somewhat artificial. The examinee is unable to visualize the patient and is only able to view paper copies of physical examination findings and diagnostic imaging. Another concern is related to the intimidation or anxiety that may occur when examinees are queried face to face by examiners with whom they are not familiar. Further, despite formalized scoring systems, there is an element of subjectivity when an examiner assesses examinees in person.9 Literature suggests a possible bias for candidates with good interpersonal skills, good communication skills, and those who are physically attractive.10 Other criticisms are of a logistical nature, such as the expense of preparation and practice or travel to the testing site in Chicago.9
If virtual examinations are found to be comparable to traditional oral examinations, there may be many potential benefits; such advantages may be economic savings, time conservation, and possible limitation of subjectivity in EM residency and medical student oral examinations. Although this study does not evaluate whether SL is equivalent to the traditional oral board examination in assessing candidates, one can envision the application of advanced virtual simulation technology as a way to alleviate some of the barriers encountered in the current process. In addition to reported ease of use and perception by many that this was a more realistic experience, the SL virtual examination format is adaptable. Using SL, a mock oral examination could be administered from any remote location with computer access and at any time of day. Thus, oral examinations could be completed while on away rotations, while travelling, at home rather than in an office setting, or at a remote testing site.
Many aspects of testing via virtual simulation require exploration before such technology can be appropriately implemented for general use in oral board examination. Faculty perceptions and experiences need to be evaluated. Additionally, this format must be assessed for limitations regarding reliability, interobserver agreement, and general outcomes in formative and evaluative settings.
Resident feedback regarding further use of SL in EM education was overwhelmingly positive. Many examinees desired the capability of more advanced interaction with the virtual patient and within the examination room. With currently available animation and programming capabilities, items in the room could be more interactive in future examinations. The examinee might click on the patient avatar’s body to perform physical examination skills or the IV pole to order IV fluids and instruct a nurse avatar to perform programmed tasks. Transition to an automated SL scenario, without a real-life proctor, could be achieved through application of artificial intelligence.4 Further, this could be easily expanded to a multicase format, requiring concurrent care of multiple patients and the physician avatar to transition between patient rooms.
This study design measures feasibility of administering a virtual examination; however, it does not evaluate the effectiveness of a virtual examination in the assessment of examinees relative to a standard oral examination.
This study was conducted at a single academic training site. Examinees experienced the same simulated patient case in the traditional format used previously, and case familiarity may have contributed to a more positive experience during the subsequent virtual examination.
Three examinees reported feedback or echoing in the headset, and one experienced a shutdown of the computer system requiring a restart of the examination. Such impediments can likely be eliminated completely with upgraded quality of microphone and headset and system capabilities.
Application of Second Life virtual technology is a potential alternative to traditional mock oral examinations for EM residents. Residents found Second Life to be a user-friendly, realistic venue for simulated patient encounters. Research is necessary to characterize this immersive learning environment for use in resident education and assessment.