Evaluation of three short-listing methodologies for selection into postgraduate training in general practice
Article first published online: 17 DEC 2008
© Blackwell Publishing Ltd 2009
Volume 43, Issue 1, pages 50–57, January 2009
How to Cite
Patterson, F., Baron, H., Carr, V., Plint, S. and Lane, P. (2009), Evaluation of three short-listing methodologies for selection into postgraduate training in general practice. Medical Education, 43: 50–57. doi: 10.1111/j.1365-2923.2008.03238.x
- Issue published online: 17 DEC 2008
- Article first published online: 17 DEC 2008
- Received 27 November 2007; editorial comments to authors 6 February 2008, 29 April 2008; accepted for publication 28 May 2008
- *education, medical, graduate;
- family practice/*education;
- Great Britain;
- *school admission criteria;
- evaluation studies [publication type];
- problem-based learning;
Objective This study aimed to evaluate the effectiveness and efficiency of three short-listing methodologies for use in selecting trainees into postgraduate training in general practice in the UK.
Methods This was an exploratory study designed to compare three short-listing methodologies. Two methodologies – a clinical problem-solving test (CPST) and structured application form questions (AFQs) – were already in use for selection purposes. The third, a new situational judgement test (SJT), was evaluated alongside the live selection process. An evaluation was conducted on a sample of 463 applicants for training posts in UK general practice. Applicants completed all three assessments and attended a selection centre that used work-related simulations at final stage selection. Applicant scores on each short-listing methodology were compared with scores at the selection centre.
Results Results indicate the structured AFQs, CPST and SJT were all valid short-listing methodologies. The SJT was the most effective independent predictor. Both the structured AFQs and the SJT add incremental validity over the use of the CPST alone. Results show that optimum validity and efficiency is achieved using a combination of the CPST and SJT.
Conclusions A combination of the CPST and SJT represents the most effective and efficient battery of instruments as, unlike AFQs, these tests are machine-marked. Importantly, this is the first study to evaluate a machine-marked SJT to assess non-clinical domains for postgraduate selection. Future research should explore links with work-based assessment once trainees are in post to address long-term predictive validity.