Cluster randomized controlled trial of the impact of a computer-assisted learning package on the learning of musculoskeletal examination skills by undergraduate medical students

Authors


Abstract

Objective

To identify whether there was measurable impact of a specific computer-assisted learning (CAL) package, “Virtual Rheumatology,” on the learning of musculoskeletal examination skills by medical students.

Methods

We conducted 2 parallel, cluster-randomized controlled trials using undergraduate curricula at 2 locations: Newcastle and London, UK. Medical students attending a musculoskeletal rotation were allocated to the intervention (Virtual Rheumatology CD) or the control arm of the study by placement group. A formative 14-item objective structured clinical examination (OSCE) assessment on the examination of shoulder and/or knee joints was the main outcome measure at Newcastle. At London, a 17-item knee station formed part of the summative OSCE. We also used a questionnaire including a 15-item confidence log (C-Log) for self assessment of musculoskeletal examination skills and knowledge. Analysis was by intention to teach.

Results

At Newcastle, there were 112 students in the CD allocated group and 129 in the non-CD group. The CD allocated group performed significantly better on the OSCE (P = 0.002) and C-Log (P = 0.005) than the non-CD group. At London, there were 48 students in the CD allocated group and 65 in the non-CD group. The CD allocated group performed better on the knee OSCE than the non-CD group (adjusted P = 0.040), but there was little difference in the change in C-Log scores from baseline to followup between the 2 groups (P = 0.582).

Conclusion

The Virtual Rheumatology CD has a positive impact on the acquisition of musculoskeletal examination skills in medical students. Further study is needed to see if similar advantages could be gained in other clinical specialities and how CAL resources could be effectively integrated into the medical curriculum.

INTRODUCTION

Musculoskeletal problems are a common reason for consultation in both primary (1) and secondary (2) care. There is, however, evidence of deficiencies in musculoskeletal examination skills among health professionals (2) and undergraduate medical students (3). Although the last decade has seen considerable changes in many medical schools, with more structured teaching of clinical skills, there remains considerable variation in teaching between and even within medical schools (4). Because of the pressure on the medical curriculum to cover all medical specialties, improvement in the learning of musculoskeletal examination skills is unlikely to result from an increase in allocated time, but rather from improved quality of teaching and/or an improvement in the curriculum.

Computer-assisted learning (CAL) programs offer a number of advantages. They allow self-paced autonomous learning and improve access to learning materials for students who are geographically dispersed. The learning content can be presented through multiple media. Within a musculoskeletal curriculum, this can help students learn when presenting conceptually difficult materials, such as 3-dimensional relationships in anatomy, by simplifying information with the use of animated visual images. Although students prefer and learn best with “real-life” examples (4, 5), issues such as the move to ambulatory care and concern over patient confidentiality have made patient-based teaching more challenging to achieve. Also, CAL resources can offer the students the advantage of acquiring the core skills and knowledge prior to contact with patients, thereby allowing the students to best utilize patient contact time.

Research investigating the effectiveness of CAL on student learning within medical education is limited (6). The increasingly popular role of CAL in undergraduate medical education, as well as the cost and labor involved in its design, emphasize the importance of evaluating its usefulness in learning. Current evidence is equivocal and not methodologically strong (6–9). Our aim was to evaluate the impact of CAL on undergraduate clinical skills learning by using a model, the CD-ROM Virtual Rheumatology (10).

MATERIALS AND METHODS

We conducted 2 randomized controlled trials in 2 medical schools in the United Kingdom: University of Newcastle upon Tyne and Royal Free, and University College Medical School London. The curriculum structure at the 2 medical schools differed, so the trials were conducted separately, but addressed the same research objective. There were 2 outcome measures: an objective structured clinical examination (OSCE) and a confidence log (C-Log). OSCE is a valid and widely used measure for the assessment of clinical examination skills (11–13), and confidence is an important factor in educational achievement (14).

The CD-ROM Virtual Rheumatology (10).

The CD-ROM was produced with the support of an Arc funded educational project grant. It consists of 3 interactive modules: Introduction to Terms in Anatomy; Clinical Anatomy and Examination of the Shoulder Joint; and Clinical Anatomy and Examination of the Knee Joint. These modules include illustrations of anatomic terms and living anatomy, photographs of the skeleton, and digital video sequences to demonstrate examination techniques. A self-assessment component is included in each module.

Participants.

At Newcastle, third-year undergraduate medical students attending a musculoskeletal week during a 12-week clinical skills module were included in the study. Prior to the start of the placement, students had attended 1 week of clinical skills teaching.

In London, third-year undergraduates rotating through a 5-week locomotor module were included. Unlike Newcastle, there was no preplacement teaching of clinical skills, although musculoskeletal examination had been taught as part of the introductory course at the beginning of the third year.

Cluster randomization.

The cluster unit was the clinical placement group; each group was randomized to the intervention or control arms. Block randomization was used (block size = 2) to ensure that approximately equal numbers of students were distributed between the 2 arms of the study. An independent researcher not involved in outcome assessment was responsible for group allocation, using a computer-generated random number table.

Outcome measures.

For the initial Newcastle groups, the OSCE consisted of a station on knee examination and one on shoulder examination. For logistic reasons, students in later groups were examined on one station only. Each station was 6 minutes long. Assessors were blind to group allocation. There was a 14-item checklist for scoring the OSCE. For each item a score of 0, 1, or 2 was given for “not done,” “done,” and “done well,” respectively. In addition, we also added a global rating scale (GRS) as a supplementary measure (a 10-cm visual analog scale ranging from “poor” [score 0] to “excellent” [score 100]) to the OSCE score sheets. GRSs have been shown to be valid measures for the assessment of clinical skills (15). Stations were video taped to ascertain the consistency in scoring the OSCE.

In the London trial, as part of the end of year summative OSCE, one station assessed students' examination of the knee joint. The station was 5 minutes long and assessors were blind to group allocation. The OSCE checklist consisted of 17 items, and each item was scored 0 or 1 for “not done” and “done,” respectively.

A prevalidated, self-complete, 15-item confidence log was used, including 5 items assessing clinical examination of the shoulder and 4 for the knee, 2 addressing general musculoskeletal knowledge, and 4 on clinical anatomy of the knee and shoulder. Each item was rated on an 11-point numerical rating scale ranging from “not at all confident” (score 0) to “very confident” (score 10).

Procedure.

Figure 1 illustrates the trial procedures. Participants completed a baseline questionnaire on the first day of the musculoskeletal rotation. This included the perceived C-Log and questions on age, sex, access to computers, experience in using computer-based learning material, and use of other learning resources.

Figure 1.

Flowchart of trial procedures. OSCE = objective structured clinical examination.

On day 1 of the musculoskeletal module in the Newcastle trial, students allocated to the intervention arm were each given a CD followed by 1 hour access time to a computer laboratory during lunchtime (arranged in response to findings of the pilot, which suggested that CD use would be higher if access to computers was better). On day 1 of the module in the London trial, students allocated to the intervention were given a verbal introduction to the content of the CD-ROM, and each student was given a CD.

Students in the Newcastle trial took part in a formative OSCE on shoulder and/or knee examination on day 4. Students were also asked to complete a followup questionnaire, which included the perceived C-Log, on day 4 of the week's placement. Within the CD allocated group, the questionnaire also asked whether the individual had used the CD-ROM.

Students in the London trial completed a knee OSCE as part of the end of year assessment 1–10 months after completion of the module. The followup questionnaire was completed on day 24 of the placement.

Statistical analysis.

Cluster sizes were expected to be ∼20 on average, and between-cluster variances were expected to be small relative to within-cluster variances (expected intraclass correlation coefficient <0.05) (16). On this premise, a sample size of between 120–240 participants was required to detect a moderate to large effect size (measured as the ratio of the difference in mean scores between groups to the pooled SD of scores) of 0.5–0.8 (17) with 80% power and 5% significance level (2-tailed).

Item scores on the OSCE were summed: the scale range was 0–28 for both the shoulder and knee stations in Newcastle and 0–17 for the knee station in London. If both stations were completed in Newcastle, an overall score was calculated by taking the average of the shoulder and knee scores. The primary measure for the OSCE was the overall score based on scores for the shoulder and/or knee (OSCE total). Subanalyses were carried out for the separate scores of the shoulder and knee stations. Secondary analysis was carried out using the OSCE-GRS (scale: 0–100).

For the perceived C-Log, C-Logtotal was derived by averaging the 15 individual scores (scale: 0–10). Subanalyses were also carried out for average scores of the 5 clinical shoulder items and the 4 clinical knee items. Outcome data relating to the C-Log summary scores at followup are given both in terms of absolute scores and change scores compared with baseline.

The primary outcome was the OSCE. The principal analysis was performed using the intention-to-teach approach, i.e., comparing the outcomes of all students followed up in the CD allocated group with those of all students followed up in the non-CD group. A secondary (per protocol) analysis was also carried out on the main data. This approach compared CD users in the CD allocated group (excluding CD non-users in the CD allocated group) with all the participants in the non-CD group. Exploratory subgroup analyses of the main data were carried out in relation to other baseline factors.

Analysis was carried out using a hierarchical linear regression with 2 levels (cluster level and student level). “Cluster” was entered as a random variable to account for the variance between individual clusters. The primary analysis was unadjusted but a sensitivity analysis was also carried out adjusting for the baseline covariates age, sex, computer access, experience with CAL, and C-Log scores of participants. MlwiN (18) and SPSS (19) were used to perform the statistical analyses. Statistical significance is given at the 5% probability level (2-tailed).

Consent and ethics approval.

Approval was obtained from respective chairs of the University Ethics committees. Verbal consent was obtained from all students for the completion of questionnaires. At Newcastle, written consent was obtained from the students for video recording of the OSCE and use of the data for research. At London, the OSCE station was one station in a set of summative final examination stations. It was agreed that this score should be excluded from the aggregate OSCE score, if the difference between the 2 groups on the knee station was sufficiently large to significantly affect the students' aggregate score on the OSCE.

RESULTS

Recruitment and participant flow.

Figure 2 illustrates the flow of subjects through the trial. Baseline characteristics of those recruited are shown in Table 1.

Figure 2.

Trial profile, flow of participants through trial. * There were no specific exclusion criteria. All students approached at baseline were both eligible and consented to participate in the study. OSCE = objective structured clinical examination; ΔC-Log = change in the perceived confidence log from baseline to followup.

Table 1. Baseline characteristics of the CD group and non-CD group*
 CD groupNon-CD group
  • *

    Values are number (percentages) unless stated otherwise. CAL = computer-assisted learning; C-Log = confidence log; C-Logtotal = average score across all 15 items of the confidence log; C-Logshoulder = average score for the 5 items assessing clinical examination of the shoulder; C-Logknee = average score for the 4 items assessing clinical examination of the knee.

  • Numbers do not always add to the total due to some missing data. In Newcastle, 111 in the CD group and 127 in the non-CD group completed the C-Log. In London, 48 in the CD group and 64 in the non-CD group completed the C-Log.

  • Confidence log scales range from 0 (not at all confident) to 10 (very confident).

Newcastlen = 112n = 129
 Age, years  
  19–25103 (92)122 (96)
  >259 (8)5 (4)
 Sex  
  Female77 (69)87 (67)
  Male35 (31)42 (33)
 Most regular access to computers  
  Home36 (33)27 (22)
  University71 (64)92 (75)
  Other3 (3)3 (3)
 Experience using CAL material  
  No7 (6)16 (12)
  Yes105 (94)113 (88)
 C-Log scores, mean ± SD  
  C-Logtotal6.6 ± 1.16.4 ± 1.3
  C-Logshoulder6.4 ± 1.46.2 ± 1.5
  C-Logknee7.2 ± 1.26.8 ± 1.4
Londonn = 48n = 65
 Age, years  
  19–2545 (93.8)64 (98.5)
  >253 (6.2)1 (1.5)
 Sex  
  Female32 (66.7)36 (55.4)
  Male16 (33.3)29 (44.6)
 Most regular access to computers  
  Home18 (37.5)20 (31.3)
  University30 (62.5)44 (68.8)
  Other0 (0)0 (0)
 Experience using CAL material  
  No8 (16.7)8 (12.3)
  Yes40 (83.3)57 (87.7)
 C-Log scores, mean ± SD  
  C-Logtotal38 ± 1.53.7 ± 1.3
  C-Logshoulder2.9 ± 1.83.0 ± 1.5
  C-Logknee3.6 ± 1.63.4 ± 1.5

At Newcastle, 241 students were recruited. Thirteen clusters were randomized between May 2002 and July 2003: 6 to the CD allocated group and 7 to the non-CD group. The median cluster size was 18 (range 18–20). A total of 112 subjects were assigned to the CD allocated group, and 129 to the non-CD group.

At London, 113 students were recruited. Seven clusters were randomized between October 2002 and July 2003: 3 to the CD allocated group and 4 to the non-CD group. The median cluster size was 16 (range 14–18). A total of 48 subjects were assigned to the CD allocated group, and 65 to the non-CD group.

OSCE scores.

In the Newcastle trial, 156 students underwent OSCE assessment: 34 completed both shoulder and knee stations, and 122 completed either one of the stations. Baseline characteristics of the students who completed the OSCE were similar to those who did not, e.g., baseline mean C-Log scores were 6.5 and 6.4, respectively. OSCE scores for the 2 study groups and their differences are summarized in Table 2. The primary outcome was significantly higher in the CD allocated group: mean difference 1.6 (95% confidence interval [95% CI] 0.5, 2.6; P = 0.002); effect size = 0.5. Observed mean differences were greater for the assessment of the shoulder than the knee.

Table 2. Summary of OSCE scores for the comparison of CD and non-CD groups (intention-to-teach approach)*
OSCECD groupNon-CD groupDifference (95% CI)
nMean ± SDnMean ± SDUnadjusted§Adjusted
  • *

    OSCE = objective structured clinical examination; 95% CI = 95% confidence interval; GRS = global rating scale.

  • Mean difference in scores for CD group–non-CD group using MLwiN fitting 2-level linear models with random intercept and slope (taking into account the hierarchical structure with random cluster effects).

  • Number of OSCE scores analyzed.

  • §

    Crude analysis not adjusted for baseline factors.

  • Analysis adjusted for baseline characteristics of participants (age, sex, computer access, experience of CD-ROM, C-Log score).

  • #

    OSCE scale range: 0–28.

  • **

    Primary outcome.

  • ††

    Precise confidence intervals (rounding to 3 decimal places): unadjusted analysis (−0.046, 1.682); adjusted analysis (0.040, 1.828).

Newcastle      
 Shoulder/knee#7520.5 ± 3.28118.8 ± 3.21.6 (0.5, 2.6)**1.9 (0.8, 3.0)
 Shoulder/knee–GRS††5975.0 ± 12.24867.9 ± 12.47.0 (2.4, 11.7)7.5 (2.4, 12.7)
 Shoulder#4320.2 ± 4.05017.6 ± 2.92.4 (0.9, 3.9)2.7 (1.1, 4.3)
 Shoulder–GRS (range 0–100)2977.0 ± 12.42570.1 ± 12.56.8 (0.3, 13.4)7.3 (0.0, 14.6)
 Knee#4621.2 ± 2.75120.5 ± 3.40.7 (−0.5, 1.9)1.0 (−0.3, 2.3)
 Knee–GRS (range 0–100)3073.1 ± 11.92365.5 ± 12.27.5 (1.1, 14.0)7.1 (0.8, 13.5)
London      
 Knee (range 0–17)4715.1 ± 1.85814.1 ± 2.40.8 (0.0, 1.7)††0.9 (0.0, 1.8)††

In the London trial, 105 students underwent the OSCE assessing knee examination. The mean score of the CD allocated group was significantly higher than that of the non-CD group (Table 2) after adjusting for baseline covariates (unadjusted P = 0.064; adjusted P = 0.040).

C-Log scores.

At Newcastle, ΔC-Logtotal scores were available for 178 students at the primary end point. The baseline profile of these participants was similar to those 70 students whose scores were not available, e.g., baseline mean C-Log scores were 6.6 and 6.3, respectively. C-Log scores for the 2 study groups and their differences are summarized in Table 3. The CD allocated group had a significantly higher mean ΔC-Logtotal score than the non-CD group: mean difference 0.3 (95% CI 0.1, 0.6; P = 0.005); effect size = 0.4. Mean scores were significantly different between the CD allocated group and non-CD group for the ΔC-Log shoulder assessment, but not for the ΔC-Log knee assessment.

Table 3. Summary of confidence log (C-Log) scores for the comparison of CD and non-CD groups (intention-to-teach approach)*
 CD groupNon-CD groupDifference (95% CI)
nMean ± SDnMean ± SDUnadjusted§Adjusted
  • *

    Confidence log absolute scales range from 0 (not at all confident) to 10 (very confident); change scales (ΔC-Log) represent scores at followup minus those at baseline and have a possible range of −20 to +20. C-Logtotal = average score across all 15 items of the confidence log; C-Logshoulder = average score for the 5 items assessing clinical examination of the shoulder; C-Logknee = average score for the 4 items assessing clinical examination of the knee.

  • Mean difference in scores for the CD group and non-CD group using MLwiN fitting 2-level linear models with random intercept and slope (taking into account the hierarchical structure with random cluster effects).

  • Number of C-Log scores analyzed.

  • §

    Crude analysis not adjusted for baseline factors.

  • Analysis adjusted for baseline characteristics of participants (age, sex, computer access, experience of CD-ROM, C-Log score).

Newcastle      
 C-Logtotal827.4 ± 1.0986.7 ± 1.30.7 (0.3, 1.0)0.4 (0.1, 0.6)
 ΔC-Logtotal810.7 ± 0.8970.3 ± 0.80.3 (0.1, 0.6) 
 C-Logshoulder827.1 ± 1.2996.3 ± 1.50.8 (0.4, 1.2)0.4 (0.1, 0.8)
 ΔC-Logshoulder810.6 ± 1.2980.2 ± 1.10.4 (0.0, 0.7) 
 C-Logknee837.8 ± 1.21017.3 ± 1.30.5 (0.2, 0.9)0.1 (−0.1, 0.4)
 ΔC-Logknee830.4 ± 1.01000.4 ± 1.00.1 (−0.2, 0.4) 
London      
 C-Logtotal357.2 ± 1.3467.4 ± 1.1−0.2 (−0.7, 0.3)−0.2 (−0.7, 0.3)
 ΔC-Logtotal353.5 ± 1.7453.7 ± 1.4−0.2 (−0.9, 0.5) 
 C-Logshoulder356.9 ± 1.4467.0 ± 1.5−0.2 (−0.8, 0.5)−0.1 (−0.8, 0.6)
 ΔC-Logshoulder354.0 ± 1.9464.0 ± 1.80.0 (−0.8, 0.8) 
 C-Logknee357.5 ± 1.4467.7 ± 1.0−0.2 (−0.8, 0.3)−0.3 (−0.8, 0.2)
 ΔC-Logknee354.0 ± 1.8454.2 ± 1.6−0.3 (−1.1, 0.5) 
       

At London, change scores were available for 80 students at the primary end point. There were no statistically significant differences between the 2 study groups (Table 3).

Further analyses.

On subgroup analysis, statistical interaction in relation to the effect size of the intervention (for both OSCE and ΔC-Logtotal) separately with respect to sex and baseline C-Logtotal was not statistically significant (P > 0.05). On-CD analysis, mean scores were significantly higher for CD users than for participants in the non-CD group (per protocol). The mean difference for the OSCE was 2.3 (95% CI 1.2, 3.4), and for the ΔC-Logtotal was 0.3 (95% CI 0.1, 0.6). In the CD allocated group, CD users were more likely than CD nonusers to access a computer at home and have a higher baseline confidence (Table 4).

Table 4. Baseline characteristics of CD users and CD non-users among participants in the CD group*
 CD usersCD non-users
  • *

    Values are numbers (percentages) unless stated otherwise. CAL = computer-assisted learning; C-Log = confidence log.

  • Sample sizes reflect the numbers of participants who responded to the question asking about use of the CD; 23 of the 112 participants in the CD group in Newcastle and 12 out of the 48 participants in the CD group in London did not complete this question.

  • Confidence log scales range from 0 (not at all confident) to 10 (very confident). C-Logtotal is the average score across all 15 items of the confidence log.

Newcastlen = 54n = 35
 Age, years  
  19–2551 (94)33 (94)
  >253 (6)2 (6)
 Sex  
  Female35 (65)24 (69)
  Male19 (35)11 (31)
 Most regular access to computers  
  Home20 (37)9 (27)
  University33 (61)23 (70)
  Teaching hospital1 (2)1 (3)
 Experience using CAL material  
  No2 (4)3 (9)
  Yes52 (96)32 (91)
 C-Logtotal score, mean ± SD7.0 ± 0.96.2 ± 1.3
Londonn = 20n = 16
 Age, years  
  19–2519 (95)15 (94)
  >251 (5)1 (6)
 Sex  
  Female13 (65)13 (81)
  Male7 (35)3 (19)
 Most regular access to computers  
  Home10 (50)5 (31)
  University10 (50)11 (69)
 Experience using CAL material  
  No1 (5)4 (25)
  Yes19 (95)12 (75)
 C-Logtotal score, mean ± SD3.9 ± 1.73.6 ± 1.3

DISCUSSION

Our findings provide evidence that the provision of a CAL package has measurable educational value for the learning of musculoskeletal examination skills. At both Newcastle and London, the CD allocated group performed significantly better in the musculoskeletal OSCE. Also, the increase in students' perceived confidence in performing joint examination was significantly larger in the CD group than the non-CD group in Newcastle, although this was not the case in London.

Newcastle students received 1 week of clinical skills training immediately prior to the musculoskeletal rotation, whereas London students received their musculoskeletal teaching at the beginning of the academic year (i.e., 1–8 months prior to undertaking the current rotation). This contrast in curriculum structure was reflected in the different baseline scores of the 2 schools: Newcastle students had higher baseline perceived confidence than London students. However, it is encouraging that scores were comparably high in both medical schools at followup. As an additional learning resource, the CD appears to have made a significant impact on self confidence in Newcastle. However, for students who start off with lower self confidence as in London, any difference made by the CD on perceived confidence may be small relative to the overall increase in confidence gained from a full teaching program. Although perceived confidence is not an accurate indicator of the level of learning of the student (20, 21), it is an important predictor of student success in learning (14). The higher baseline confidence of CD users in comparison with CD nonusers in London and Newcastle suggests that students' confidence is a factor that determines the likelihood of using novel resources such as CAL tools.

In 2001, Greenhalgh (6) identified 12 randomized controlled trials (RCTs) of CAL programs in undergraduate medical education, 8 of which involved <100 participants. Rogers et al (22) found that the use of a CAL program as the sole teaching method resulted in an inferior performance in tying a surgical knot compared with a seminar approach. However, the seminar method included personalized feedback with 1 tutor per 6 students. A number of studies compared classroom teaching or printed text methods with a CAL program (23–25). However, it seems likely that CAL would be available in addition to printed materials and often alongside face to face teaching; hence, the importance of our study, which was aimed at identifying any additional benefit from a CAL program. Summers et al (23) found that students taught via the CAL program performed better in terms of tying surgical knots than those receiving instruction from a teacher or by videotape. However, the study was somewhat artificial in that the tutor was not permitted to provide performance feedback to students in any of the groups, including the didactic group. An extension of the same search strategy by Greenhalgh (6) to present day revealed 24 studies, none of which were RCTs and none used skills learning as an outcome measure.

Designing a valid, educational RCT poses a number of methodologic challenges. Statistical power issues were addressed by determining sample size a priori. Currently the gold standard for assessing clinical competence is the OSCE (26). The sensitivity of the OSCE may be restricted by the small range of scoring points on the checklist. Moreover, not all students underwent the OSCE in Newcastle, but there was little reason to suspect, when reviewing the baseline data, that these students were a select sample of the study population. Furthermore, the true effect size of CD use might have been underestimated by the intention-to-teach analysis, because only 50% of the participants in the intervention group actually used the CD. Embedding the resource in the curriculum would have improved uptake (9, 27, 28). The fact that OSCE results were statistically significant in favor of the CD-allocated group in both centers, despite the different time intervals between intervention and assessment, lends weight to the argument that the CD has an impact on improving student performance.

A number of questions, however, remain unanswered. What would be the cost implications of providing CAL resources to undergraduates? The cost of designing the Virtual Rheumatology CD was £11,740 ($22,045). A web-based resource can be more cost effective because it will reduce the cost of producing CD-ROMs and the cost of updating the resource. However, the disadvantage is that students need Internet access of sufficient speed for interactive visual and animated material. Further work also needs to be carried out to determine if CAL resources are of equal educational value in all specialties or whether they are of more interest to only certain specialties. Most importantly, the educational impact of a CAL resource will probably depend on its quality.

In conclusion, we have shown in 2 well-established musculoskeletal curricula that a CAL resource (Virtual Rheumatology CD) has a measurable educational impact on the learning of musculoskeletal examination skills. The features of CAL offer potentially more than the traditional learning resources by, for example, enabling the students to better use the clinical experience and limited contact time with clinicians and patients. Further studies should explore the limitations to the effective embedding of CAL resources into the medical curriculum and to investigating whether the same advantages could be achieved across other clinical areas. Finally, this study demonstrates that RCTs may have a role in assessing educational interventions, and can be used in conjunction with other nonquantitative methodologies for informing medical educators regarding issues of best practice.

Acknowledgements

The authors would like to thank professor Peter Croft for his support throughout this research project. Dr. Louisa Badcock was involved in the preliminary study design.

APPENDIX A

MEMBERS OF THE ARC VIRTUAL RHEUMATOLOGY COMPUTER-ASSISTED LEARNING (CAL) RESEARCH GROUP

Pirashanthie Vivekananda-Schmidt, D.Phil, Martyn Lewis, PhD, Andrew B. Hassell, MD: Keele University; David Coady, MRCP, David Walker, MD, Lesley Kay, FRCP: University of Newcastle upon Tyne; Monica J. McLean, PhD: University of Oxford; Inam Haq, MRCP, Anisur Rahman, MRCP, Jane Dacre, MD: Royal Free and University College Medical School.

Ancillary