Study 2 was funded, in part, by a Faculty Innovation Grant from Angelo State University. The authors wish to thank Josh Collins, Lauren Felton, Jessica Stoner, and Jennifer Thorndike for their assistance with the data collection for Study 2.
Evidence for the Effectiveness of an Alternative Multisource Performance Rating Methodology
Article first published online: 2 AUG 2012
© 2012 Wiley Periodicals, Inc.
Volume 65, Issue 3, pages 531–563, Autumn 2012
How to Cite
Hoffman, B. J., Gorman, C. A., Blair, C. A., Meriac, J. P., Overstreet, B. and Atchley, E. K. (2012), Evidence for the Effectiveness of an Alternative Multisource Performance Rating Methodology. Personnel Psychology, 65: 531–563. doi: 10.1111/j.1744-6570.2012.01252.x
- Issue published online: 2 AUG 2012
- Article first published online: 2 AUG 2012
Despite persistent concerns as to the quality of performance information obtained from multisource performance ratings (MSPRs), little research has sought ways to improve the psychometric properties of MSPRs. Borrowing from past methodologies designed to improve performance ratings, we present a new method of presenting items in MSPRs, frame-of-reference scales (FORS), and test the efficacy of this method in a field and lab study. The field study used confirmatory factor analysis to compare the FORS to traditional rating scales and revealed that FORS are associated with increased variance due to dimensions, decreased overlap among dimensions, and decreased error. The laboratory study compared rating accuracy associated with FORS relative to frame-of-reference training (FORT) and a control group and demonstrated that FORS are associated with higher levels of accuracy than the control group and similar levels of accuracy as FORT. Implications for the design and implementation of FORS are discussed.