Making sense of work-based assessment: ask the right questions, in the right way, about the right things, of the right people
Version of Record online: 13 DEC 2011
© Blackwell Publishing Ltd 2012
Volume 46, Issue 1, pages 28–37, January 2012
How to Cite
Crossley, J. and Jolly, B. (2012), Making sense of work-based assessment: ask the right questions, in the right way, about the right things, of the right people. Medical Education, 46: 28–37. doi: 10.1111/j.1365-2923.2011.04166.x
- Issue online: 13 DEC 2011
- Version of Record online: 13 DEC 2011
- Received 25 January 2011; editorial comments to authors 24 February 2011, 30 August 2011; accepted for publication 7 September 2011
Medical Education 2012: 46: 28–37
Context Historically, assessments have often measured the measurable rather than the important. Over the last 30 years, however, we have witnessed a gradual shift of focus in medical education. We now attempt to teach and assess what matters most. In addition, the component parts of a competence must be marshalled together and integrated to deal with real workplace problems. Workplace-based assessment (WBA) is complex, and has relied on a number of recently developed methods and instruments, of which some involve checklists and others use judgements made on rating scales. Given that judgements are subjective, how can we optimise their validity and reliability?
Methods This paper gleans psychometric data from a range of evaluations in order to highlight features of judgement-based assessments that are associated with better validity and reliability. It offers some issues for discussion and research around WBA. It refers to literature in a selective way. It does not purport to represent a systematic review, but it does attempt to offer some serious analyses of why some observations occur in studies of WBA and what we need to do about them.
Results and Discussion Four general principles emerge: the response scale should be aligned to the reality map of the judges; judgements rather than objective observations should be sought; the assessment should focus on competencies that are central to the activity observed, and the assessors who are best-placed to judge performance should be asked to participate.