ACADEMIC EMERGENCY MEDICINE 2011; 18:420–427 © 2011 by the Society for Academic Emergency Medicine
Objectives: Telesimulation is a novel concept coupling the principles of simulation with remote Internet access to teach procedural skills. This study’s objective was to determine if telesimulation could be used by pediatricians in Toronto, Ontario, Canada, to teach a relatively new intraosseous (IO) insertion technique to physicians in Africa.
Methods: One simulator was located in Toronto and the other in Gaborone, Botswana. Instructors and trainees could see one another, see inside each other’s simulators, and communicate in real time. Learner’s opinions and skills were evaluated. Before and after the curriculum, physicians completed a self-assessment questionnaire, a multiple-choice test, and during session 3, a demonstration of competence using an IO infusion system was timed and scored locally and via the Internet.
Results: Twenty-two physicians participated. The scores on the pretest ranged from 1 to 12 out of 15. The range of scores on the posttest was 10 to 15 out of 15. The mean (±SD) score on pre- and post–multiple choice testing increased by +5 (±2.75; 95% confidence interval [CI] for mean difference = 3.92 to 6.35). Based on McNemar’s chi-square test, physicians reported a significant improvement in their comfort and knowledge inserting IO needles (p < 0.01), familiarity with the EZ-IO infusion system (p < 0.01), and knowledge handling the IO equipment (p < 0.01). Postintervention, all physicians reported that telesimulation teaching was a worthwhile experience, and 95% felt more prepared to manage pediatric resuscitation. There was no evidence of a difference in scoring or timing of IO insertion tasks whether measured locally or remotely (mean ± SD score difference = −0.11 ± 1.22 [95% CI = −0.66 to 0.43]; mean ± sd time difference = 0.01 ± 0.15 seconds [95% CI = −0.06 to 0.08 seconds]).
Conclusions: Telesimulation is a novel method for teaching procedural skills. The session improved physicians’ knowledge, self-reported confidence, and comfort level in inserting the IO needle. Accurate scoring is possible via the Internet. This modality offers potential for teaching other procedural skills over distances.
Telesimulation was first developed by a small group of surgeons at the University of Toronto. Their objective was to teach the fundamentals of laparoscopic surgery to surgeons in Botswana. Following this first fundamentals course in which Canadian surgeons traveled to Botswana, it was clear that more mentorship was required to help surgeons in Botswana develop their laparoscopic skills.1 With travel to Botswana taking upwards of 24 hours, more frequent trips to this country were not feasible. From this challenge emerged the concept of telesimulation.2
Telesimulation uses the Internet to link simulators between an instructor and trainee in different locations. Using two simulators, multiple computers, a series of webcams, and basic videoconferencing software, the instructor and trainee can see within each other’s simulators as well as see and speak to each other. Telesimulation differs from telementoring or teleconferencing because it actually connects two simulators in different physical locations, allowing teacher and student to see, but not control, what the other is doing in real time.2 This surgical group successfully demonstrated that using remote telesimulation for teaching laparoscopic skills resulted in skills improvement in comparison to self-practice.2
Telesimulation Applied to Pediatric Resuscitation
Resuscitation of acutely ill and poorly perfused infants and children is a challenging process requiring knowledge and skill. The success or failure of resuscitation often depends on the ease and timeliness with which vascular access is established.3–8
Large-bore peripheral intravenous (IV) catheters have traditionally been the first choice for obtaining venous access. However, successfully placing peripheral IVs can be time-consuming or even impossible, resulting in delayed patient care.3–8 Intraosseous (IO) needles provide fast, safe, and reliable vascular access that can be established within seconds, permitting administration of fluids and medications via the bone marrow.
Primary indications for IO access are altered levels of consciousness, respiratory compromise, or hemodynamic instability. Example situations include shock, cardiac arrest, respiratory arrest, trauma, hypovolemia, seizures, or any other condition that would require life-saving fluids or medications.9 Insertion into the medial aspect of the tibia (below the tibial tuberosity, on the flat anterior aspect of the bone) can be implemented in both the conscious and unconscious patient.9
Intraosseous blood flow is rapid and continues even during shock. Therefore, drugs and fluids infused via this route reach the central circulation as quickly as those administered through standard IV access.7,8,10 Central lines can have complications in and of themselves, but this is accentuated in emergent situations where they are placed in unsterile conditions.11–15 IO needles confer no risk of pneumothorax, and the insertion point is out of the way of possible efforts for airway management or chest compression.9 Recent technological advancements have improved the once manual task of IO insertion.
The EZ-IO (Vidacare Corp., San Antonio, TX) is a hand-held, battery-powered medical drill that is an acknowledged alternative to emergency IV access.7,8,16,17 This technology is not readily available in low- and middle-income countries due to a variety of factors, including cost of the needle driver and needles, as well as a lack of local training opportunities.9 However, as the economic situation in some of these countries improves, interest in learning novel techniques in vascular access is emerging. Some argue that a rapid, safe, and easy alternative to traditional vascular access is even more relevant in these environments where there are no alternatives to peripheral vascular access.3,4,6–8 Physicians in Gaborone, Botswana, expressed an interest in learning about mechanically inserted IO needles after one of their group suffered a needle stick injury with a manually inserted IO needle on an HIV-positive infant.
Appropriate training in new procedural technology remains a challenge in developing countries. Educational limitations imposed by factors such as cost, distance, politics (visa requirements), time constraints, and inconvenience are often insurmountable. Some health care practitioners in high-income countries have responded by traveling to low- and middle-income countries to provide clinical care and to teach local health care providers.
In Botswana there is a high incidence of dysentery, cholera, and HIV/AIDS. Using EZ-IO for vascular access is a logical and practical alternative to IV access, as it may protect care provider and patient from complications and exposure to infectious disease.9
In this study, the techniques used by Okrainec et al.1 for teaching the fundamentals of laparoscopic surgery have been applied for the first time to the pediatric resuscitation skill of IO access. The purpose of this research was to determine if telesimulation is effective for teaching a multidisciplinary group of physicians in Botswana mechanical IO insertion skills.
Twenty-two participants in Botswana from a variety of medical subspecialties (pediatrics, emergency medicine, surgery, and anesthesia) participated in the telesimulation training. Participants volunteered based on their clinical exposure to acutely ill children and their desire to learn about this mechanically assisted IO insertion technique. Each participant gave informed consent under a protocol approved by the University of Toronto Institutional Review Board. Demographic information and baseline experience was collected from each participant. The official language in Botswana is English, which eliminated any potential language barrier between learners and instructors. None of the physicians had previous experience in using the EZ-IO device. Vidacare Corp. donated the EZ-IO equipment used in this project to the Princess Marina Hospital in Gaborone.
Telesimulation Setup in Botswana
A telesimulation room was established at the Princess Marina Hospital, Gaborone. A similar room was set up in the Toronto Western Hospital surgical simulation lab. Each telesimulation room contained one simulation trainer box with gooseneck camera. The simulation camera connected to a standard television using an S-video cable, allowing the physician to see the inside of the trainer box. The simulator camera was connected to a laptop computer using a USB cable, allowing this camera to be used as a webcam. The image from the simulator camera in Gaborone was transmitted using free Skype videoconferencing software (Skype Limited, Luxembourg) to a screen in Toronto, and vice versa, allowing instructor and trainee to see the contents of each other’s simulator in real time. A second webcam, connected to a second computer and LCD projector, displayed an external image of each person using the simulator. This image was transmitted simultaneously with the image from within the simulator to a second screen, allowing the instructor and trainee to see each other. Instructor and trainee could communicate using standard computer speakers and microphone (Figures 1–3).
Two of the authors (AM, AK) were in Toronto to administer sessions 1, 2, and 3. One author (GA) was on site in Botswana and managed all local administrative tasks; he did not provide any formal instruction to physicians and deferred all major questions to the instructors in Toronto. The odd minor question may have been answered, but the premise was that all instruction was to be done from Toronto. GA ensured that the scheduling, organizational, and technical issues on the Botswana side were dealt with and was also responsible for administering the pre- and postintervention self-assessment questionnaires and the pre- and postintervention multiple-choice tests.
Each physician completed pre- and postintervention self-assessment questionnaires that measured changes in attitudes as a result of the telesimulation. The preintervention questionnaire consisted of six questions focused on comfort, confidence, and familiarity with pediatric resuscitation and EZ-IO use. Responses were scaled from 1 to 5 (with 1 representing “strongly disagree” and 5 representing “strongly agree”). Statements addressed two main areas of knowledge, comfort, and skill: pediatric resuscitation in general and IO insertion.
The written test consisted of 15 multiple choice questions. The questions focused on pediatric resuscitation in general and EZ-IO insertion. The written test was pilot-tested on pediatric residents at the instructor’s base hospital during their resuscitation and IO insertion skills training to ensure that the questions were relevant and clearly worded. The same 15 questions were readministered as the postintervention test.
Resuscitation and EZ-IO Teaching: Session 1
The content for this didactic session was developed by four of the authors (SS, AM, AK, and GA) and consisted of a 30-minute PowerPoint (Microsoft Corp., Redmond, WA) presentation, with vascular access placed in the context of patient hydration assessment and management. The key areas of the presentation were: 1) hydration status assessment and fluid resuscitation, 2) the EZ-IO system, 3) indications and contraindications, 4) insertion locations, 5) insertion technique, 6) removal technique, and 7) complications.
At all times during the presentation, students could see both the PowerPoint slides and the presenter. After the slide presentation, presenters from Toronto demonstrated the correct insertion technique and procedure on a mannequin tibia using the telesimulation setup. The final 15 minutes of the 1-hour session were for interactive discussion and questions between students and presenters. All sessions were held at the end of the clinical day in Botswana, which was morning in Toronto.
Telesimulation Training: Session 2
One week after the 1-hour didactic session, all physicians in Botswana participated in a telesimulation technical practice session. The timing of this session was chosen for the practical reasons of physician availability for regularly scheduled local academic time. During this telesimulation session, each participant obtained 15 minutes of dedicated mentoring from an instructor in Toronto. Participants would demonstrate the entire EZ-IO technique, from landmarking and insertion to removal and disposal of sharps, on the infant and adult tibial models developed by Vidacare. An instructor in Toronto would provide specific feedback, demonstrate proper technique, and focus on areas requiring improvement.
Skills Testing: Session 3
One week later, at the third and final session, during the regularly scheduled academic time, each participant was presented with a clinical scenario and instructions. The scenario was a 2-year-old child with protracted vomiting and diarrhea suffering moderate to severe dehydration. On a Laerdal (Laerdal Medical, Stavanger, Norway) infant IO insertion model, with anatomical landmarks at the tibial tuberosity, the participant was expected to recognize the child’s hydration status and verbally describe the resuscitative plan and landmarks, while physically demonstrating the landmarks followed by complete insertion, fluid administration, and removal of the EZ-IO on the task trainer.
Each participant was scored and timed by two independent observers. One observer (GA) was present in Botswana’s teaching room. The other observer (either AM or AK) scored and timed each participant across the Internet connection from Toronto. Participants were scored on patient management and IO insertion with an evaluation tool designed for the course. Scoring was modified from that described on the Vidacare website and determined based on three factors: 1) correct sequence and performance of assessment and management (airway, breathing, circulation), 2) recognition of physical examination and physiologic abnormalities, and 3) appropriateness of therapeutic interventions. A higher score indicates better performance, with a perfect score being 18. The goal was to show that telesimulation scoring and timing was equivalent to scoring and timing each participant in person (Figure 4).
The written posttest was administered at the end of the instructional program. It consisted of the same 15 questions as the pretest. Physicians were aware of their total scores from the pretest but did not know how they scored on each individual question. After finishing the multiple choice test, physicians completed an eight-item posttelesimulation questionnaire with items similar to the pretelesimulation questionnaire. Two new questions were added to the posttelesimulation survey, asking participants to rank their final opinion as to the benefit of the telesimulation session, and their agreement with the statement, “I now feel more prepared to manage pediatric resuscitation than I did previously.”
Descriptive statistics were calculated to summarize the participants’ subspecialty profiles. The difference in each physician’s score on the pre- and postintervention multiple choice test was calculated and the 95% confidence interval (CI) for the mean of the difference was determined. For each survey statement pre- and posttelesimulation, responses were grouped into low (1 = strongly disagree, 2 = disagree, 3 = neutral) and high (4 = agree, 5 = strongly agree) categories. A 2 × 2 table was created for each pre- and posttelesimulation survey question to compare pre low/high responses with post low/high responses. Participants’ pre- and posttelesimulation survey responses were compared using McNemar’s chi-square test with continuity correction. The differences between EZ-IO skill demonstration scoring and timing measured locally versus remotely were determined, and the 95% CI for the mean of the difference was calculated. All reported p-values are two-sided; a significance level of 0.05 was used on tests.
Twenty-two physicians participated in the three-part telesimulation session on pediatric resuscitation using the EZ-IO. The group’s subspecialty information is summarized in Table 1. Twenty participants completed the preintervention questionnaire, and all 22 completed the postintervention questionnaire. All participants completed the pre- and postintervention multiple-choice test, as well as the timed and scored final scenario evaluation.
|Medical Specialty||n (%)|
|Emergency medicine||1 (4.5)|
|General Surgery||2 (9.1)|
|Obstetrics and gynecology||1 (4.5)|
The scores on the pretest ranged from 1 to 12 out of 15. The range of scores on the posttest was 10 to 15 out of 15. Physician scores on the posttelesimulation written test improved by a mean of +5 (95% CI for the mean difference = 3.92 to 6.34) compared to their scores on the pretelesimulation test.
Questionnaire Responses (Table 2)
|McNemar chi-square test|
|1. At this time I feel comfortable and knowledgeable inserting IO needles.||0||17||0||3||15.0588|
p < 0.01
|2. I understand my role within a pediatric resuscitation.||0||4||0||16||2.25|
p = 0.13
|3. I am familiar with the EZ-IO IO infusion system.||0||19||0||1||17.0526|
p < 0.01
|4. I know how to handle the EZ-IO equipment.||0||20||0||0||18.05|
p < 0.01
|5. I feel comfortable with the vascular access skills that may be required of me during a pediatric resuscitation.||1||12||0||7||10.0833|
p < 0.01
|6. I am aware of the assessment and management priorities during a pediatric resuscitation.||0||6||0||14||4.1667|
p = 0.04
Compared with the pretelesimulation responses, on the posttelesimulation questionnaire physicians expressed improved comfort and knowledge inserting IO needles (McNemar’s chi-square = 15.06, p < 0.01), an increased familiarity with the EZ-IO infusion system (McNemar’s chi-square = 17.05, p < 0.01), and an enhanced awareness of how to handle equipment (McNemar’s chi-square = 18.05, p < 0.01).
Posttelesimulation, physicians also expressed an increased comfort with vascular access skills (McNemar’s chi-square = 10.08, p < 0.01) and an enhanced awareness of assessment and management priorities during a pediatric resuscitation (McNemar’s chi-square = 4.17, p = 0.04). Their pre- and posttelesimulation rank of understanding their role within a pediatric resuscitation did not change significantly (McNemar’s chi-square = 2.25, p = 0.13). After telesimulation, all 22 physicians rated the telesimulation as a worthwhile experience, and 95% (21 of 22) felt more prepared to manage a pediatric resuscitation than they did previously.
There was no evidence of a difference in the amount of time measured to complete the scenario tasks and demonstration either in person or over the Internet (mean ± SD time difference = 0.01 ± 0.15 seconds [95% CI = −0.06 to 0.08]). The mean score of the practical 18-point EZ-IO demonstration measured locally in Botswana was 15.89 (SD ± 1.69). The mean score of the practical demonstration measured over the Internet was 15.77 (SD ± 1.04). There was no evidence of a difference in the skills test scores measured either in person or over the Internet (mean ± sd score difference = −0.11 ± 1.22 [95% CI = −0.66 to 0.43]).
This study demonstrates that the telesimulation educational initiative on mechanical IO insertion increased physicians’ comfort managing pediatric resuscitation, comfort with IO procedural skills, familiarity with the EZ-IO infusion system, and awareness of pediatric resuscitation management priorities. Physicians’ knowledge, as measured by scores on the multiple-choice test, improved significantly after the telesimulation session. It was possible to accurately score and time all EZ-IO insertion tasks from Toronto.
For this discussion, distance learning is defined by the following criteria from “A Teacher’s Guide to Distance Learning” by the Florida Center for Instructional Technology: 1) the teacher and students are separated by distance; 2) instruction is delivered via print, voice, video, or computer technologies; and 3) communication is interactive. The teacher receives feedback from the student, which can be immediate or delayed.18
There are a number of reasons why distance learning occurs. These include educational opportunities close to home, students receiving exposure to telecommunication technologies, and in our case, students making contact with expert physicians from an academic quaternary-care hospital that they may otherwise not have been able to access.
Distance learning technologies offer convenient locations for students to learn and for instructors to teach. Telesimulation uses the Internet to allow the instructors and students to be in separate locations but communicate and interact in real time.18
Distance learning often provides students with an option to participate whenever they wish, on an individualized basis.18 This was not the case in Botswana where students were required to participate at a set time and location. What the experience did allow for was direct contact and interaction with pediatric emergency medicine experts.
Telesimulation is both convenient and effective. Research results show that telesimulation can be equally or more effective than traditional didactic instruction with demonstration of procedures.2 When teaching methods and available technologies are used appropriately to the instructional tasks, when there is student-to-student interaction, and when there is timely teacher-to-student feedback, distance learning is effective.18 In Botswana, students were able to interact with one another, received immediate feedback, and had physician support on site at their teaching location. Therefore, not only were they interacting via telesimulation with instructors in Toronto, they also had access to an investigator (GA) on site for administrative, organizational, and technical support. This made their learning effective, as was evident through the significant improvement in their written test scores.
The primary strength of this study is the good correlation in remote versus onsite scoring of skill testing. In contrast, a study by Curran et al.,19 comparing face-to-face versus remote assessment of neonatal resuscitation skills, showed inconsistency in certain skill assessments between on-site and remote observers. Although our physical setup of equipment and technologies was relatively similar to that of Curran et al., the difference in correlation of remote versus onsite scoring of skill testing can most likely be explained by the differences in the performance checklists used. Curran et al. used a 131-point checklist of multiple skills and tasks. It required scoring on a three-point, subjective scale (0 = not performed; 1 = performed, but incorrectly, out of sequence, or with prompting; 2 = performed correctly).19 The skill assessment we used was focused to one, more specific task and scored on an 18-point scale with only two possible assessments (0 = not performed; 1 = performed).
A threshold score of 15 out of 18 on the skills test, without a critical error in landmarking or insertion technique, was considered to be a reflection of competence in EZ-IO insertion. This study did not provide insight into the number of practice sessions needed to achieve lasting competence using distance learning, nor was it designed to define the score needed to become instructors of the procedure in the participants’ own country.
In the context of procedural skills education programs in developing countries, telesimulation offers several advantages over other commonly used training models. Telesimulation learning in the Botswana case involved little or no cost to the learners. Telesimulation means not having to pay for instructors to travel to Botswana with all of their supplies and teaching material. As well, it saves on accommodation and instructor time, which is also deemed precious. Learners did not have to travel far to be taught a new procedure. They were instructed to go to a predetermined location with Internet and videocamera access that was close to their practice facility. It is relatively easy for students to watch telesimulation via real-time Internet access. They did not have to pay for their learning or for long-distance travel.20
Currently, many courses provided by international faculty are offered over short, intensive time periods. Cronin et al.21 showed that modern communications technology can be used to assess neonatal resuscitation skills over large distances and at significant cost savings compared to traditional, in-person assessments. Telesimulation is an ideal method for delivering a distributed curriculum on a weekly basis. In addition, offering more frequent onsite courses in these countries would require faculty to travel long distances at significant cost. Telesimulation eliminates this need by allowing instructors to teach from their own institution at regular intervals.2
One of the main advantages of our described setup for telesimulation is the ease of availability and relatively low cost of the equipment and software required. This is obviously of crucial importance when considering educational programs in resource-restricted countries. Although two computers are required to connect both the simulation camera and the external webcam, these need not be powerful systems, as only access to the Internet and use of the free Skype software is required. The need for a liquid crystal display projector, which adds to the realism of the interaction, and a separate computer monitor can be avoided by purchasing two basic laptops, which are readily available in most countries where sufficient bandwidth is not the limiting factor.2
The variety of teaching materials and methods used during distance learning results in more students having their learning preferences met. For example, some students learn better through visual stimuli, such as video, while others experience optimal learning through listening, interacting or by doing.18 In Botswana, students had the opportunity to learn via video, voice, and hands-on experience, enabling them to use the various senses required for optimal learning.
Previous research suggests that distance learning courses enable increased student–teacher interactions.18 In Botswana, students and educators interacted in a way that created a favorable learning environment. Although separated by physical distance, a number of interaction modes were provided, including video, voice, and on-site professional presence. Participant survey responses and test results indicate the efficacy of telesimulation. Students were able to ask questions freely to acquire knowledge. It is well established that increased teacher–student interaction results in an increased number of student needs being met.20
Educational inequity is a major issue in many countries, including Botswana. Rural schools often have less access to qualified educators. The educators they do have often lack access to current trends and technology, which would allow them to better transfer knowledge to students. Distance learning offers great potential for alleviating these issues. This study used the “train the trainer” approach, whereby instructors from Toronto trained and certified the first group of physicians in Botswana. These physicians will subsequently travel to rural and remote locations within Botswana to further train more local physicians who do not have access to Internet resources.
Without a measurement of skills before and immediately after the practice session, we were unable to prove that this procedural skill was learned initially. Because the physicians were familiar with the manual IO insertion technique, but had no previous experience with this mechanical assistance device, we can only indirectly infer that the training intervention was successful. Without a skills posttest immediately after training, we were unable to determine if the skill deteriorated between the second and third sessions.
One disadvantage of distance learning via telesimulation is the possibility of unforeseen technical problems, such as communication logistics with Skype or functioning cameras with sound. These malfunctions can disrupt teaching or inhibit the teaching process altogether. The Skype connection was dropped during one physician’s final skills test, which made his timing assessment invalid. Also, during the teaching and training sessions, the connection was disrupted and needed to be reestablished, which created an inconvenient interruption to the sessions. Other disadvantages are equipment start-up costs (although relatively small and potentially available for multiple applications) and need for sufficient bandwidth for Internet service. The discussion on telesimulation as a distance learning technique is based on an assumption that, despite their different backgrounds, all physician learners should be similar in their educational needs.
Telesimulation is a novel, practical, inexpensive, effective, and well-received method for teaching appropriate procedural skills such as intraosseous insertion. It can be applied between any two centers with Internet connections of sufficient bandwidth. This new educational tool has great potential for teaching other procedural skills over long distances.
The authors thank Brent Hagel, PhD, and Alberto Nettel-Aguirre, PhD, PStat, of the Alberta Children’s Hospital Research Methods Team for consultative guidance on the statistical analysis and Agnes Bellegris for manuscript preparation.