Pitfalls in the pursuit of objectivity: issues of reliability
Article first published online: 29 JAN 2009
1991 Blackwell Publishing
Volume 25, Issue 2, pages 110–118, March 1991
How to Cite
VAN DER VLEUTEN, C. P. M., NORMAN, G. R. and DE GRAAFF, E. (1991), Pitfalls in the pursuit of objectivity: issues of reliability. Medical Education, 25: 110–118. doi: 10.1111/j.1365-2923.1991.tb00036.x
- Issue published online: 29 JAN 2009
- Article first published online: 29 JAN 2009
- Received 20 November 1989; editorial comments to authors 11 January 1990; accepted for publication 11 September 1990
- educational measurement/*methods;
- reproducibility of results;
- clinical competence;
- rev tutor
Summary. Objectivity has been one of the hallmarks in the assessment of clinical competence in recent decades. A consistent shift can be noticed in which subjective measures are being replaced by objective measurement methods. In the transition from subjective to objective methods trade-offs are involved, both in the effort expended and in the range of behaviours assessed. The issue of the presumed superiority of objective measures is addressed in two successive papers.
In this paper a distinction is made between objectivity as a goal of measurement, marked by freedom of subjective influences in general, and objectivity as a set of strategies designed to reduce measurement error. The latter has been termed objectification. The central claim of this paper is that these two approaches to assessment do not necessarily coincide. By reviewing a number of studies comparing subjective and objectified measurement methods, the claim of the supremacy of the latter with respect to reliability is discussed.
The results of these studies indicate that objectified methods do not inherently provide more reliable scores. Objectified methods may even provide unwanted outcomes, such as negative effects on study behaviour and triviality of the content being measured. The latter issues, related to validity, efficiency and acceptability, are discussed in a second paper.