Programmatic assessment and Kane’s validity perspective
Article first published online: 13 DEC 2011
© Blackwell Publishing Ltd 2012
Volume 46, Issue 1, pages 38–48, January 2012
How to Cite
Schuwirth, L. W. T. and van der Vleuten, C. P. M. (2012), Programmatic assessment and Kane’s validity perspective. Medical Education, 46: 38–48. doi: 10.1111/j.1365-2923.2011.04098.x
- Issue published online: 13 DEC 2011
- Article first published online: 13 DEC 2011
- Received 14 March 2011; editorial comments to authors 14 April 2011; accepted for publication 19 July 2011
Medical Education 2012: 46: 38–48
Context Programmatic assessment is a notion that implies that the strength of the assessment process results from a careful combination of various assessment instruments. Accordingly, no single instrument is superior to another, but each has its own strengths, weaknesses and purpose in a programme. Yet, in terms of psychometric methods, a one-size-fits-all approach is often used. Kane’s views on validity as represented by a series of arguments provide a useful framework from which to highlight the value of different widely used approaches to improve the quality and validity of assessment procedures.
Methods In this paper we discuss four inferences which form part of Kane’s validity theory: from observations to scores; from scores to universe scores; from universe scores to target domain, and from target domain to construct. For each of these inferences, we provide examples and descriptions of approaches and arguments that may help to support the validity inference.
Conclusions As well as standard psychometric methods, a programme of assessment makes use of various other arguments, such as: item review and quality control, structuring and examiner training; probabilistic methods, saturation approaches and judgement processes, and epidemiological methods, collation, triangulation and member-checking procedures. In an assessment programme each of these can be used.