Evaluating communication skills in the objective structured clinical examination format: reliability and generalizability
Version of Record online: 29 JAN 2009
1996 Blackwell Publishing
Volume 30, Issue 1, pages 38–43, January 1996
How to Cite
Hodges, B., Turnbull, J., Cohen, R., Bienenstock, A. and Norman, G. (1996), Evaluating communication skills in the objective structured clinical examination format: reliability and generalizability. Medical Education, 30: 38–43. doi: 10.1111/j.1365-2923.1996.tb00715.x
- Issue online: 29 JAN 2009
- Version of Record online: 29 JAN 2009
- Received 5 September 1994 editorial comments to authors 13 March 1995; accepted for publication 27 September 1995
- *clinical competence;
- educational measurement;
- patient simulation;
- physician–patient relations
In most objective structured clinical examinations (OSCEs), communication skills are assessed as an ‘add-on’ to history-taking stations, rather than in stations designed to assess communication skills in the broadest sense. This study investigated the feasibility of developing such stations. In part one, 60 clinical clerks and 36 residents were rated in four 10-min emotionally charged situations portrayed by standardized patients. Inter-rater reliability was demonstrated (r = 0.59–0.63) and a highly significant effect of educational level was found. Generalizability between communication stations was low (0.17–0.20).
Several explanations for poor generalizability, including poor discrimination as a result of low score variance and the confounding effect of content knowledge, were addressed in part 2. Ninety-five final-year medical students participated in an OSCE in which six 10-min encounters examined the students' ability to manage difficult emotional situations such as fear, anxiety, mania, sadness, confusion and anger. Half the students encountered a patient with moderate emotional symptoms and half an extreme emotional state. For difficult stations, students' scores were lower and standard deviation higher, suggesting that manipulating difficulty increases score variance and potentially discrimination. However, a strong interaction was found between difficulty and station content, and communication scores were highly correlated with content. Scenarios which created major communication difficulties (such as mania) resulted in much larger differences in scores between the easy and difficult versions.
Communication OSCE stations can be created with acceptable reliability including difficult cases which address communication skills beyond simple history taking. Nevertheless, a generalizable set of communication skills remains elusive.