Trained lay observers can reliably assess medical students’ communication skills

Abstract

Context  Our project investigated whether trained lay observers can reliably assess the communication skills of medical students by observing their patient encounters in an out-patient clinic.

Methods  During a paediatrics clerkship, trained lay observers (standardised observers [SOs]) assessed the communication skills of Year 3 medical students while the students interviewed patients. These observers accompanied students into examination rooms in an out-patient clinic and completed a 15-item communication skills checklist during the encounter. The reliability of the communication skills scores was calculated using generalisability analysis. Students rated the experience and the validity of the assessment. The communication skills scores recorded by the SOs in the clinic were correlated with communication skills scores on a paediatrics objective structured clinical examination (OSCE).

Results  Standardised observers accompanied a total of 51 medical students and watched 199 of their encounters with paediatric patients. The reliability of the communication skills scores from nine observed patient encounters was calculated to be 0.80. There was substantial correlation between the communication skills scores awarded by the clinic observers and students’ communication skills scores on their OSCE cases (r = 0.53, P < 0.001). Following 83.8% of the encounters, students strongly agreed that the observer had not interfered with their interaction with the patient. After 95.8% of the encounters, students agreed or strongly agreed that the observers’ scoring of their communication skills was valid.

Conclusions  Standardised observers can reliably assess the communication skills of medical students during clinical encounters with patients and are well accepted by students.