SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Box 1: Definitions of terms
  6. Results
  7. Discussion
  8. Acknowledgments
  9. References

Medical Education 2010: 44: 765–774

Objectives  Educators often speak of web-based learning (WBL) as a single entity or a cluster of similar activities with homogeneous effects. Yet a recent systematic review demonstrated large heterogeneity among results from individual studies. Our purpose is to describe the variation in configurations, instructional methods and presentation formats in WBL.

Methods  We systematically searched MEDLINE, EMBASE, ERIC, CINAHL and other databases (last search November 2008) for studies comparing a WBL intervention with no intervention or another educational activity. From eligible studies we abstracted information on course participants, topic, configuration and instructional methods. We summarised this information and then purposively selected and described several WBL interventions that illustrate specific technologies and design features.

Results  We identified 266 eligible studies. Nearly all courses (89%) used written text and most (55%) used multimedia. A total of 32% used online communication via e-mail, threaded discussion, chat or videoconferencing, and 9% implemented synchronous components. Overall, 24% blended web-based and non-computer-based instruction. Most web-based courses (77%) employed specific instructional methods, other than text alone, to enhance the learning process. The most common instructional methods (each used in nearly 50% of courses) were patient cases, self-assessment questions and feedback. We describe several studies to illustrate the range of instructional designs.

Conclusions  Educators and researchers cannot treat WBL as a single entity. Many different configurations and instructional methods are available for WBL instructors. Researchers should study when to use specific WBL designs and how to use them effectively.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Box 1: Definitions of terms
  6. Results
  7. Discussion
  8. Acknowledgments
  9. References

The role of web-based learning (WBL) in health professions education has grown rapidly1 and nearly all medical schools in the USA and Canada currently employ online course materials.2 With this growth comes a pressing need to understand how to effectively design and implement WBL. Yet WBL is multifaceted. The outcomes of web-based educational activities will vary depending on the WBL intervention (and the interaction of this intervention with specific topics, learners and learning contexts). Cook provided a conceptual framework for the diversity in computer-assisted instruction (CAI), identifying variations in the configuration (‘“big picture” differences within a given media format’, such as the tutorial, online discussion group or virtual patient context), instructional methods (‘techniques that support learning processes … [which] include questions, cases, simulations, interactive models, analogies, group discussion and activities, and feedback’) and presentation (‘elements … that enhance a given intervention’, such as font, multimedia and simulation fidelity).3 Our recent systematic review and meta-analysis of WBL demonstrated large heterogeneity in effect sizes of individual studies, and subgroup analyses (according to study design, participants, instructional designs and outcomes) did not readily explain these differences.4 These data corroborate the conceptual understanding that WBL is not a single entity and that the variations in WBL studies to date are complex and not easily captured in univariate subgroup analyses.

However, it is our experience that educators and researchers frequently view WBL as a single entity or a cluster of similar activities with relatively homogeneous effects. The studies in our meta-analysis4 attempted to evaluate the effectiveness of WBL in comparison with no intervention or other instructional methods. In most instances WBL was (at least implicitly) viewed as a monolithic construct and in some instances the WBL intervention was scarcely described. Our systematic review did not fully describe the variations in instructional designs and configurations, and we are not aware of any such summaries.

The purpose of the present study (a planned extension of the published systematic review) is to describe in detail the variations in configurations, instructional methods and presentation elements that we encountered. Such a description could enable researchers to more clearly define their instructional interventions, which, in turn, will facilitate future evidence syntheses. A detailed summary of previous instructional designs might also provide developers with ideas for future implementations. Our intent is to illustrate the many options available, rather than evaluate the effectiveness of any individual configuration or method (which has been previously reported).4,5

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Box 1: Definitions of terms
  6. Results
  7. Discussion
  8. Acknowledgments
  9. References

Study eligibility and selection

We included all studies from a recent review of Internet-based instruction.4,5 Our search strategy and article selection have been previously described in detail.4 To summarise briefly, with the assistance of an experienced research librarian we searched indexing databases including MEDLINE, EMBASE, ERIC, Scopus and CINAHL for articles describing studies comparing Internet-based instruction for health professionals with no intervention, non-CAI interventions (media-comparative studies) or CAI interventions. We conducted the last full search on 17 January 2008 and updated the search for comparisons with CAI on 18 November 2008. We also hand-searched the reference lists of all included studies to identify additional citations. Reviewing in duplicate the titles, abstracts and, when necessary, the full text, we identified 266 relevant articles, including six in languages other than English.

Data extraction and synthesis

We abstracted information on the geographic location of the web-based course (in most cases the server location, except when the majority of learners were in another location), learner location, educational setting, course length and course instructional design. We distinguished configurations (‘“big picture” differences within a given media format’3) and instructional methods (‘techniques that support learning processes’3) (see Box 1 for more detailed definitions of terms used). We also coded computer and non-computer configurations separately.

We developed, iteratively tested and refined a data abstraction form until high consensus was achieved. Most information was abstracted in duplicate with high inter-rater agreement, as previously reported.4,5 Author DAC abstracted the information on some instructional configurations and methods alone, although spot checks with other reviewers revealed few disagreements. We tabulated data using sas 9.1 (SAS Institute, Inc., Cary, NC, USA).

We also purposively selected and briefly described several reports that illustrate specific web-based configurations and methods.

Box 1: Definitions of terms

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Box 1: Definitions of terms
  6. Results
  7. Discussion
  8. Acknowledgments
  9. References

Web-based instructional configurations

  • • 
    E-mail discussion: communication among students conducted via e-mail.
  • • 
    Learning management system: a software platform designed to facilitate the rapid and systematic development of a course. This typically offers tools for site security and user identification, posting course assignments and resources, web page creation, synchronous and asynchronous online discussion, and testing.
  • • 
    Multimedia: different content forms, such as text, sound, photographs and other still images, animations or video.
  • • 
    Online discussion other than by e-mail: communication among students conducted using, for example, threaded discussion, online chat or videoconferencing.
  • • 
    Online resources: calculators, patient resources, full-text journal articles and other useful web sites made available through hyperlinks.
  • • 
    PowerPoint™: a product by Microsoft® that creates slide presentations. Most web browsers can view PowerPoint™ slides, such that simply uploading a PowerPoint™ presentation to the web is a very quick (albeit limited) way to produce web-based instruction.
  • • 
    Simulation or virtual patient: an activity that emulates real life and requires learner interaction; virtual patients constitute one form of simulation.
  • • 
    Synchronous communication: simultaneous interaction between two or more course participants over the web, using methods such as online chat, instant messaging or two-way videoconferencing.4
  • • 
    Tutorial: the online equivalent of a lecture; typically a stand-alone, web-based course or module within a course that learners complete alone; the amount of interactivity and multimedia varies.
  • • 
    Videoconferencing: simultaneous transmission of both audio and video information. Video information might comprise an image of the instructor, other video media, or a shared projection of the computer screen (i.e. whiteboard).4

Instructional methods

  • • 
    Adaptation: tailoring of course content or navigation in response to learner needs or characteristics.
  • • 
    Cognitive interactivity: enhancing the level of cognitive engagement through activities such as practice exercises, essays and group collaborative projects.
  • • 
    Discussion: instructor–student or student–student interaction (online or face-to-face) intended to facilitate the achievement of learning objectives.
  • • 
    Feedback: information provided to the learner on his or her performance.
  • • 
    Homework assignments: tasks to be completed offline. These are often required to be submitted online at a future date.
  • • 
    Patient cases: scenarios describing the initial presentation or clinical course of a real or hypothetical patient, often used as a prelude to a self-assessment question or as part of a virtual patient.
  • • 
    Practice exercises: cases, self-assessment questions and other activities that require learners to apply information they have learned.4
  • • 
    Self-assessment: an opportunity, usually a question but sometimes a simulation or other task, allowing learners to ascertain their own mastery of the material.
  • • 
    Spacing of learning: intentional spacing involves repeating the same course material at least once during the course (a course spread out over a prolonged period, but in which new material is presented at each session, does not count as intentional spacing); unlimited access requires that learners be able to use most or all of the course content at any point in the course.

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Box 1: Definitions of terms
  6. Results
  7. Discussion
  8. Acknowledgments
  9. References

What has been done: a summary

Course participants and context

We identified 266 articles comparing WBL with no intervention or with another educational intervention. Of these, 78 described randomised trials. The number of reports increased steadily over the study period (Fig. 1). Most of these reports described comparisons with no intervention; media-comparative studies were the next most frequent type. Studies comparing one computer-based intervention with another comprised a small but growing minority.

image

Figure 1.  Temporal trends in the number of publications describing web-based learning in comparison with no intervention or another intervention group. Shading indicates the type of comparison reported (some articles reported two comparisons); NI = no intervention; M = an alternative media format (i.e. ‘traditional’ instruction); CAI = another computer-assisted instruction intervention; Other = confounded comparisons (e.g. comparison of lecture versus computer-based + small-group activities). Data for 2007 and 2008 are not reported because database indexing was not complete on the date of the last full search

Download figure to PowerPoint

The most frequent course participants were medical students and doctors in postgraduate training (Table 1). Over one-third of the courses involved learner participants from locations other than the main school or hospital campus. The time learners spent studying was unavailable for nearly half the interventions; when provided, the most common length per course unit (i.e. module or topic) was < 60 minutes (38% of reports).

Table 1.    Participants, location and duration of web-based courses
FeatureClassificationReports, n (%) (n = 266)
  1. * Some studies enrolled participants from several learner groups

  2. † Some studies included learners participating from multiple locations

  3. ‡ Percentage calculated after excluding ‘cannot tell’ codes

Participants*Pre-clinical medical students32 (12)
 Clinical medical students52 (20)
 Doctors in postgraduate training48 (18)
 Doctors in practice41 (15)
 Nurses in training31 (12)
 Nurses in practice34 (13)
 Dentists in training10 (4)
 Dentists in practice1 (0.4)
 Pharmacists in training17 (6)
 Pharmacists in practice9 (3)
 Other health professionals in training24 (9)
 Other health professionals in practice23 (9)
Geographic location of courseAfrica2 (1)
 Asia10 (4)
 Australia7 (3)
 Europe: UK22 (8)
 Europe: non-UK39 (15)
 Latin America8 (3)
 North America: Canada16 (6)
 North America: USA176 (66)
Location of teaching institution (school, hospital)On site, main campus197 (74)
 On site, satellite campus35 (13)
 Off site (distance learner)71 (27)
Setting for courseClassroom108 (42)
 Clinical practice154 (58)
 Cannot tell4 (1)
Course lengthSingle setting/1 day104 (44)
 2–7 days7 (3)
 1–4 weeks25 (11)
 > 4 weeks101 (43)
 Cannot tell30 (11)
Time spent studying online< 60 minutes54 (38)
 60–119 minutes25 (18)
 2–4 hours18 (13)
 > 4 hours44 (31)
 Cannot tell126 (47)
Web-based configurations

The vast majority of courses (89%) used at least some static written text; those that did not (and some that did) used online discussion (e-mail or discussion boards) or videoconferencing (Table 2). Most courses (55%) also used multimedia. Eighty-five courses (32%) used online communication via e-mail, threaded discussion, chat or videoconferencing. A total of 25 courses (9%) implemented live (synchronous) components. Sixty-five courses blended web-based and non-computer-based instruction, most often in face-to-face lectures or small groups (60%), assigned readings (32%) or clinical experiences (23%).

Table 2.    Frequency with which various course configurations appear in published reports of web-based learning
ConfigurationReports, n (%)
  1. * ‘Assigned readings’ refers to journal articles, book chapters or other written text assigned as part of a course; ‘paper tutorials’ refers to written documents intended for stand-alone self-instruction

  2. † Real or standardised patient interactions refers to student interaction with a real patient or a standardised patient (with or without an instructor present), whereas clinical patient experiences or teaching refers to instruction (given by a teaching doctor) in a clinical setting, such as bedside teaching on a hospital ward

Web- and computer-based configurations (n = 266)
 Static written text238 (89)
 Multimedia (audio, video, still images)146 (55)
 Links to online resources106 (40)
 Used a learning management system82 (31)
 Online discussion other than e-mail59 (22)
 E-mail discussion37 (14)
 PowerPoint™26 (10)
 Discussion (any form) was synchronous (live)25 (9)
 Videoconferencing14 (5)
 Simulation/virtual patient4 (1)
Non-computer configurations for blended courses (computer and non-computer used together) (n = 65)
 Face-to-face (lecture, small group)39 (60)
 Assigned readings*21 (32)
 Clinical patient experiences or teaching15 (23)
 Paper tutorials*5 (8)
 Laboratory experiences3 (5)
 Human patient simulator2 (3)
 Real or standardised patient interaction1 (2)
 Audio- or videotaped lecture (not web-based)1 (2)
Instructional methods

A total of 205 (77%) web-based courses employed specific instructional methods, other than text alone, to enhance the learning process. The most common methods – each used in nearly 50% of courses – were patient cases, self-assessment questions and feedback (Table 3). About one-third of courses had high levels of interactivity (cognitive engagement), but another third had low interactivity. Eighteen courses (7%) intentionally spaced instructional content to enhance learning.

Table 3.    Frequency with which various instructional methods appear in published reports of web-based learning
MethodClassificationReports, n (%) (n = 266)
Online lecture ortutorialPresent205 (77)
Online discussionNone or cannot tell182 (68)
 Small amount38 (14)
 Large amount47 (18)
Spacing of learningNone (single instance)120 (45)
 Unlimited usage129 (48)
 Intentional spacing18 (7)
InteractivityLow84 (32)
 Moderate94 (35)
 High81 (30)
Practice exercises (cases or questions)None or cannot tell99 (37)
 Few93 (35)
 Many75 (28)
Patient casesPresent128 (48)
Self-assessmentPresent120 (45)
FeedbackPresent119 (45)
Homework assignmentsPresent27 (10)
Simulation/virtual experiencesPresent13 (5)
AdaptationPresent4 (2)

What has been done: case studies

The 266 studies in our review describe a broad array of instructional configurations and methods. We will briefly review several studies, intentionally selected to illustrate the scope of what has been done and reported. For the sake of clarity and brevity, we focus on the instructional design rather than the research design or evaluation results (which we have reported previously).4,5

Configuration: web-based tutorials

The earliest web-based course in our review, published in 1996, presented a multimedia tutorial on eye histology for medical students.6 The first four modules included digitised colour photographs accompanied by both written text and narration. Clicking key words in the text would highlight a region of the photograph, retrieve additional information or link to a self-assessment question. Question formats included multiple-choice, true/false, short-answer fill-in-the-blank and ‘drag-and-drop’ extended matching formats. Incorrect answers prompted an audio-recorded explanation of the error. The final module comprised a 43-question quiz. The program operated over a local area network and was accessible via 13 computer terminals in the faculty library.

A report in 2002 described the use of PowerPoint™ slides to deliver a 1-hour web-based ‘lecture’ for medical students on preventive screening.7 Four cases with questions were presented, followed by the instructor’s approach to each case. Students had the option to view slides with or without accompanying audio (a ‘presentation’ variation).

A third web-based tutorial, published in 2007, was intended to train surgery and emergency medicine residents in central venous catheter insertion.8 This course consisted of 42 pages of text, still images and short (< 15-second) video clips illustrating optimal and suboptimal procedural performance. Residents were prompted to analyse each video clip for possible problems. The course also contained text information on catheter-related infections, two short quizzes and a list of references with links to full-text publications.

Configuration: web-based text discussion

Courses facilitated communication using various text-based discussion configurations. For example, a medical student course on ethics, described in 2001, blended weekly face-to-face lectures and online group discussion.9 Students worked in groups using an online threaded discussion board to analyse four ethically complex patient cases. In a threaded discussion all responses to a given message (‘post’) are organised hierarchically under that message, enabling participants to follow a conversation (‘thread’) according to the topic or theme discussed, rather than chronologically or by author.

A report published in 2002 described the use of e-mail to teach about dietary supplements.10 Each week, participants received two or three e-mail modules consisting of a patient case scenario, one question, the answer and an explanation, and links to additional information. In addition, participants joined an online e-mail discussion group (‘listserv’) in which they could post questions to the other participants, respond to others’ questions if they wished, or simply follow the conversation.

A nursing student course (reported in 2003) on electrocardiogram interpretation illustrates the use of discussion via online chat (synchronous text communication).11 This 6-week course comprised informational web pages, practice questions, a glossary, case studies and self-tests. Students completed the case studies and self-tests in online small groups using chat. Students also communicated with the instructor using chat, e-mail and threaded discussion.

Configuration: web-based conferencing

Educators implemented multi-site videoconferencing using several different arrangements. We describe three: a special-purpose videoconferencing studio; a simple webcam system, and a ‘virtual meeting’ tool. In 2003, an article described the use of live (synchronous) videoconferencing over high-speed telephone (ISDN) lines to enable doctors in three distant communities to discuss four teaching cases.12 The facilitator broadcast from a videoconferencing studio and other participants used personal computer-based videoconference equipment.

In 2006, an article described a course to train research study personnel to use the Hamilton Depression Scale.13 The course began with a web-based tutorial which included interactive video vignettes and a self-assessment. Trainees then interviewed two patients (one real patient, one standardised patient) at a central training site using a webcam and synchronous desktop conferencing software. A trainer in the patient room provided on-the-spot feedback. Trainees later participated in a group interview via teleconference to facilitate calibration.

Another 2006 publication described ‘webinars’ using the ‘virtual meeting environment’ tool Elluminate Live!® as part of a course for nursing students.14 In these webinars students viewed slides and heard the instructor’s voice. They communicated with the instructor by typing questions, responding to multiple-choice questions, or using built-in signals to indicate confusion or applause.

Configuration: virtual patients

A course on medical ethics, reported in 2000, illustrates the use of web-based virtual patients.15 In this course, medical students interviewed an on-screen ‘patient’ by typing in questions and the patient responded using text, audio and video. The virtual patient’s satisfaction and the completeness of information obtained depended on the type and sequence of questions. The student could also consult experts such as an attorney or medical ethicist and access external resources. When the student had committed to a course of action, the computer provided detailed feedback.

Configuration: learning management systems

Several courses used Internet-based learning management systems (see Definitions of terms). At least 10 different commercial learning management systems were used by the courses identified in our review and many investigators developed their own systems locally. Of the commercial systems, WebCT (reported at least 30 times) and Blackboard™ (reported at least 10 times) were used most often.

WebCT was used for a problem-based learning (PBL) course on acid-base physiology described in 2005.16 This course included 78 media-rich web pages, 26 self-assessments with feedback, online simulations and online PBL group activities using synchronous chat (text-only messages), a whiteboard (which permits participants to draw images and share graphics) and an asynchronous threaded discussion board. WebCT tools facilitated the authentication of users and the organisation of the web pages and resources, managed the chat, whiteboard and threaded discussions, administered the tests and provided immediate feedback.

Instructional method: tailored instruction

A report published in 2003 illustrates the use of a multi-modal approach to continuing doctor education that includes instruction tailored to individual participants.17 This course on chlamydia screening included patient vignettes to promote case-based learning and tailoring of instructional content based on responses to questions. In addition, information on practice-specific outcomes (doctors’ chlamydia screening rates) was obtained from a national health maintenance organisation and provided to course participants to help them implement the learning in their practice.

In 2008, a report described the adapting of instruction to participants’ prior knowledge of the topic.18 Each module began with a multiple-choice question. If residents responded correctly, they had the option of skipping the topic information related to that question and proceeding to the next question. If they responded incorrectly, they were required to review the related information before they could proceed.

Instructional methods: other

As noted above, most of the courses we identified implemented specific strategies to promote learning. Although we cannot illustrate all such strategies here, we have selected for discussion three instructional methods that appear to improve outcomes in WBL.5 Firstly, many courses attempted to enhance the cognitive engagement of learners using a variety of methods. One such approach, described in 2006, involved asking learners to explicitly contrast cases and write a case summary.19 Secondly, several courses made a point of providing learners with feedback in response to self-assessment questions or performance. This feedback was often personalised (e.g. adjusted according to how the learner had answered or performed) and usually provided an explanation for why the answer or performance was correct or suboptimal, as illustrated by a study published in 2008.20 Thirdly, several studies encouraged learners to study the same content multiple times (repetition). In one such course, described in 2008,21 medical students received daily e-mails, each of which offered one case-based question and detailed feedback. After 4 weeks the cycle was repeated with the same questions.

Presentation

As with instructional methods, many courses employed measures to enhance the presentation. These included using audio to replace text chat in online discussion groups,22 using animations rather than static images,23 and reducing the size of an e-mail message by delivering hyperlinks to information rather than embedding the information itself in the e-mail text.24

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Box 1: Definitions of terms
  6. Results
  7. Discussion
  8. Acknowledgments
  9. References

Using both quantitative summaries and detailed descriptions, this study portrays the broad range of instructional approaches encompassed by the term ‘web-based learning’. Using Cook’s framework3 as a guide, we found that courses varied widely in their configuration (tutorial, asynchronous discussion, live conferencing, etc.), instructional methods (e.g. practice exercises, cognitive interactivity) and presentation. Courses also varied in their integration (blending) with other instructional activities, with about one-fourth combining web-based and traditional instruction.

Limitations and strengths

This study has limitations. Several reports omitted key details on the instructional design and this impeded our coding of those interventions. We did not describe or compare study designs or study outcomes as these analyses have been performed previously. We purposively selected the studies described to reflect specific features, but other studies might have been equally illustrative. As all included studies had a comparison group, the quantitative summaries do not account for non-comparative studies.

This study also has strengths. We addressed an important focused question and employed a duplicate review system for study inclusion and most data abstraction. We identified studies using a rigorous search strategy led by an experienced librarian, which enabled us to present a complete summary of research in the field and then to select examples from this comprehensive list.

Comparison with previous reviews

Previous reviews in health professions education4,25 and other educational fields26 have demonstrated that CAI is superior to no intervention and, on average, similar in effectiveness to traditional instructional methods. Recent reviews have also demonstrated that different instructional methods (in particular, interactivity, practice exercises, feedback and repetition) can favourably influence learning outcomes.5,27 However, these reviews focused on quantitative outcomes and provided only brief descriptions (if any) of the courses included in the review. Other reviews have offered only limited summaries of the technologies and methods used.28,29 The present study complements and builds upon previous reviews by systematically summarising the frequency with which various configurations and methods have been used, and illustrating these features through detailed examples of web-based courses. Recent reviews of evolving technologies have highlighted WBL configurations not represented in the studies in our review, such as user-generated content platforms (so-called Web 2.0 tools like blogs and wikis)30,31 and online virtual worlds such as Second Life®.32

Our findings regarding the frequency of publications and the relative distribution of comparative study types parallel those of a review of CAI published 10 years ago;33 namely, they indicate a dramatic increase over time in reports and a paucity of studies comparing two computer-based interventions.

Implications

These findings have important implications for teaching and research. The diversity of approaches encompassed by WBL permits the creative application of this delivery medium for many objectives, contexts and learners. However, educators and researchers cannot treat WBL as a single entity or as a panacea. Educators contemplating the use of WBL should carefully consider the needs of their learners and the learning context in question as they decide whether to use this medium, other media or a combination. Those electing to use WBL should consider a broad range of WBL configurations and methods before choosing a specific approach. Those who study WBL must carefully describe what they do; in comparative studies they must also describe the comparison group with attention to between-group differences. We emphasise that the interventions described in this report are illustrative of WBL’s potential variation and do not represent a comprehensive inventory of technologies and methods. Although space prohibits the in-depth discussion of guidelines and theories for instructional design, readers may find useful the work by Cook and Dupras,34 Mayer,35 Jonassen,36 Spiro et al.37 and Laurillard,38 along with our empiric findings.5

Researchers should avoid attempting to define WBL as either good or bad. There is simply too much variation in the technology to permit generalisable statements. A more useful approach is to study when we should use WBL (and specific WBL configurations) and how we can use WBL effectively when we do. Unfortunately, only limited research informs our answers to these questions. We desperately need more and better research to clarify our use of WBL. ‘When’ studies might employ qualitative methods to explore how characteristics of the course, learners and context influenced outcomes in a specific course, or might quantitatively look for learner characteristics that predict success in WBL. ‘How’ studies might employ Cook’s framework for the systematic investigation of CAI3 to compare WBL courses employing different instructional methods or configurations, or might employ methods similar to ‘when’ studies. Theory-based and theory-building research will be particularly important. The instructional approaches, frameworks and research questions described here and in previous reviews,3,5,39,40 along with recent work in the field,41–45 suggest directions for future research. Such studies will improve medical education by aligning and optimising our use of this and other educational technologies for specific contexts, learning objectives and learners.

Contributors:  All authors were involved in the planning and execution of this study, in analyzing data, and in the drafting and revising of this manuscript. All approved the final manuscript.

Acknowledgments

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Box 1: Definitions of terms
  6. Results
  7. Discussion
  8. Acknowledgments
  9. References

Acknowledgements:  the authors thank Patricia J Erwin MLS for her role in the initial identification of studies.

Funding:  this work was supported by intramural funds and a Mayo Foundation Education Innovation award. AJL is supported in part through the John R Evans Chair in Health Sciences Education Research, McMaster University, Hamilton, Ontario. There was no external funding.

Conflicts of interest:  none.

Ethical approval:  not required.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Box 1: Definitions of terms
  6. Results
  7. Discussion
  8. Acknowledgments
  9. References
  • 1
    Johnson PT, Eng J, Rowell MR, Fishman EK. Evolving physician perception of worldwide web education: 2007 update and review of the literature. Acad Radiol 2007;14:1092101.
  • 2
    Kamin C, Souza KH, Heestand D, Moses A, O’Sullivan P. Educational technology infrastructure and services in North American medical schools. Acad Med 2006;81:6327.
  • 3
    Cook DA. The research we still are not doing: an agenda for the study of computer-based learning. Acad Med 2005;80:5418.
  • 4
    Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA 2008;300:118196.
  • 5
    Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Instructional design variations in internet-based learning for health professions education: a systematic review and meta-analysis. Acad Med 2010;85:90922.
  • 6
    Mars M, McLean M. Students’ perceptions of a multimedia computer-aided instruction resource in histology. S Afr Med J 1996;86:1098102.
  • 7
    Spickard A III, Alrajeh N, Cordray D, Gigante J. Learning about screening using an online or live lecture: does it matter? J Gen Intern Med 2002;17:5405.
  • 8
    Xiao Y, Seagull FJ, Bochicchio GV et al. Video-based training increases sterile-technique compliance during central venous catheter insertion. Crit Care Med 2007;35:13026.
  • 9
    Lipman AJ, Sade RM, Glotzbach AL, Lancaster CJ, Marshall MF. The incremental value of Internet-based instruction as an adjunct to classroom instruction: a prospective randomised study. Acad Med 2001;76:10604.
  • 10
    Kemper KJ, Amata-Kynvi A, Sanghavi D, Whelan JS, Dvorkin L, Woolf A, Samuels RC, Hibberd P. Randomised trial of an Internet curriculum on herbs and other dietary supplements for health care professionals. Acad Med 2002;77:8829.
  • 11
    Frith KH, Kee CC. The effect of communication on nursing student outcomes in a web-based course. J Nurs Educ 2003;42:3508.
  • 12
    Allen M, Sargeant J, Mann K, Fleming M, Premi J. Videoconferencing for practice-based small-group continuing medical education: feasibility, acceptability, effectiveness, and cost. J Contin Educ Health Prof 2003;23:3847.
  • 13
    Kobak KA, Engelhardt N, Lipsitz JD. Enriched rater training using Internet-based technologies: a comparison to traditional rater training in a multi-site depression trial. J Psychiatr Res 2006;40:1929.
  • 14
    Little BB, Passmore D, Schullo S. Using synchronous software in web-based nursing courses. Comput Inform Nurs 2006;24:31725; quiz 326–7.
  • 15
    Fleetwood J, Vaught W, Feldman D, Gracely E, Kassutto Z, Novack D. MedEthEx online: a computer-based learning program in medical ethics and communications skills. Teach Learn Med 2000;12:96104.
  • 16
    Taradi SK, Taradi M, Radic K, Pokrajac N. Blending problem-based learning with web technology positively impacts student learning outcomes in acid-base physiology. Adv Physiol Educ 2005;29:359.
  • 17
    Casebeer L, Strasser SM, Spettell CM, Wall TC, Weissman N, Ray MN, Allison JJ. Designing tailored web-based instruction to improve practising physicians’ preventive practices. J Med Internet Res 2003;5 (3):20.
  • 18
    Cook DA, Beckman TJ, Thomas KG, Thompson WG. Adapting web-based instruction to residents’ knowledge improves learning efficiency: a randomised controlled trial. J Gen Intern Med 2008;23:98590.
  • 19
    Thompson GA, Holyoak KJ, Morrison RG, Clark TK. Evaluation of an online analogical patient simulation program. Presented at the IEEE (Institute of Electrical and Electronics Engineers) Symposium on Computer-Based Medical Systems, 2223 June 2006, Salt Lake City, UT.
  • 20
    Kopp V, Stark R, Fischer MR. Fostering diagnostic knowledge through computer-supported, case-based worked examples: effects of erroneous examples and feedback. Med Educ 2008;42:8239.
  • 21
    Kerfoot BP. Interactive spaced education versus web-based modules for teaching urology to medical students: a randomised controlled trial. J Urol 2008;179:23516.
  • 22
    Jedlicka JS, Brown SW, Bunch AE, Jaffe LE. A comparison of distance education instructional methods in occupational therapy. J Allied Health 2002;31:24751.
  • 23
    Tunuguntla R, Rodriguez O, Ruiz JG, Qadri SS, Mintzer MJ, Van Zuilen MH, Roos BA. Computer-based animations and static graphics as medical student aids in learning home safety assessment: a randomised controlled trial. Med Teach 2008;30:8157.
  • 24
    Kemper KJ, Gardiner P, Gobble J, Mitra A, Woods C. Randomised controlled trial comparing four strategies for delivering e-curriculum to health care professionals. BMC Med Educ 2006;6:2.
  • 25
    Chumley-Jones HS, Dobbie A, Alford CL. Web-based learning: sound educational method or hype? A review of the evaluation literature. Acad Med 2002;77 (10 Suppl):8693.
  • 26
    Bernard RM, Abrami PC, Lou Y, Borokhovski E, Wade A, Wozney L, Wallet PA, Fiset M, Huang B. How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Rev Educ Res 2004;74:379439.
  • 27
    Bernard RM, Abrami PC, Borokhovski E, Wade CA, Tamim RM, Surkes MA, Bethel EC. A meta-analysis of three types of interaction treatments in distance education. Rev Educ Res 2009;79:124389.
  • 28
    Chaffin AJ, Maddux CD. Internet teaching methods for use in baccalaureate nursing education. Comput Inform Nurs 2004;22:13242.
  • 29
    Lau F, Bates J. A review of e-learning practices for undergraduate medical education. J Med Syst 2004;28 (1):7187.
  • 30
    Sandars J. Twelve tips for using blogs and wikis in medical education. Med Teach 2006;28:6802.
  • 31
    McGee JB, Begg M. What medical educators need to know about ‘Web 2.0’. Med Teach 2008;30:1649.
  • 32
    Beard L, Wilson K, Morra D, Keelan J. A survey of health-related activities on Second Life. J Med Internet Res 2009;11:17.
  • 33
    Adler MD, Johnson KB. Quantifying the literature of computer-aided instruction in medical education. Acad Med 2000;75:10258.
  • 34
    Cook DA, Dupras DM. A practical guide to developing effective web-based learning. J Gen Intern Med 2004;19:698707.
  • 35
    Mayer RE, ed. The Cambridge Handbook of Multimedia Learning. New York, NY: Cambridge University Press 2005.
  • 36
    Jonassen DH. Problem solving: the enterprise. In: SpectorJM, OhrazdaC, Van SchaackA, WileyDA, eds. Innovations in Instructional Technology: Essays in Honor of M David Merrill. Mahwah, NJ: Lawrence Erlbaum Associates 2005;91110.
  • 37
    Spiro RJ, Coulson RJ, Feltovich PJ, Anderson DK. Cognitive Flexibility Theory: Advanced Knowledge Acquisition in Ill-structured Domains. Champaign, IL: University of Illinois at Urbana-Champaign 1988.
  • 38
    Laurillard D. Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies, 2nd edn. New York, NY: RoutledgeFalmer 2001.
  • 39
    Ruiz JG, Cook DA, Levinson AJ. Computer animations in medical education: a critical literature review. Med Educ 2009;43:83846.
  • 40
    Cook DA, Triola MM. Virtual patients: a critical literature review and proposed next steps. Med Educ 2009;43:30311.
  • 41
    Cook DA, Beckman TJ, Thomas KG, Thompson WG. Measuring motivational characteristics of courses: applying Keller’s instructional materials motivation survey to a web-based course. Acad Med 2009;84:15059.
  • 42
    Youngblood P, Harter PM, Srivastava S, Moffett S, Heinrichs WL, Dev P. Design, development, and evaluation of an online virtual emergency department for training trauma teams. Simul Healthc 2008;3:14653.
  • 43
    Wiecha J, Heyden R, Sternthal E, Merialdi M. Learning in a virtual world: experience with using Second Life for medical education. J Med Internet Res 2010;12 (1):1.
  • 44
    Philip CT, Unruh KP, Lachman N, Pawlina W. An explorative learning approach to teaching clinical anatomy using student-generated content. Anat Sci Educ 2008;1 (3):10610.
  • 45
    Hull P, Chaudry A, Prasthofer A, Pattison G. Optimal sequencing of bedside teaching and computer-based learning: a randomised trial. Med Educ 2009;43:10812.