Evidence for the Effectiveness of an Alternative Multisource Performance Rating Methodology

  1. Brian J. Hoffman1,
  2. C. Allen Gorman2,
  3. Carrie A. Blair3,
  4. John P. Meriac4,
  5. Benjamin Overstreet5,
  6. E. Kate Atchley6

Article first published online: 2 AUG 2012

DOI: 10.1111/j.1744-6570.2012.01252.x

Issue

Personnel Psychology

Personnel Psychology

Volume 65, Issue 3, pages 531–563, Autumn 2012

Additional Information

How to Cite

Hoffman, B. J., Gorman, C. A., Blair, C. A., Meriac, J. P., Overstreet, B. and Atchley, E. K. (2012), Evidence for the Effectiveness of an Alternative Multisource Performance Rating Methodology. Personnel Psychology, 65: 531–563. doi: 10.1111/j.1744-6570.2012.01252.x

Author Information

  1. 1

    University of Georgia

  2. 2

    Radford University

  3. 3

    College of Charleston

  4. 4

    University of Missouri, St. Louis

  5. 5

    University of Georgia

  6. 6

    University of Tennessee, Knoxville

* and requests for reprints should be addressed to: Brian J. Hoffman, 228 Psychology Building, The University of Georgia, Athens, GA 30602; hoffmanb@uga.edu

  1. Study 2 was funded, in part, by a Faculty Innovation Grant from Angelo State University. The authors wish to thank Josh Collins, Lauren Felton, Jessica Stoner, and Jennifer Thorndike for their assistance with the data collection for Study 2.

Publication History

  1. Issue published online: 2 AUG 2012
  2. Article first published online: 2 AUG 2012

SEARCH

SEARCH BY CITATION

ARTICLE TOOLS

View Full Article (HTML) Get PDF (565K)

Despite persistent concerns as to the quality of performance information obtained from multisource performance ratings (MSPRs), little research has sought ways to improve the psychometric properties of MSPRs. Borrowing from past methodologies designed to improve performance ratings, we present a new method of presenting items in MSPRs, frame-of-reference scales (FORS), and test the efficacy of this method in a field and lab study. The field study used confirmatory factor analysis to compare the FORS to traditional rating scales and revealed that FORS are associated with increased variance due to dimensions, decreased overlap among dimensions, and decreased error. The laboratory study compared rating accuracy associated with FORS relative to frame-of-reference training (FORT) and a control group and demonstrated that FORS are associated with higher levels of accuracy than the control group and similar levels of accuracy as FORT. Implications for the design and implementation of FORS are discussed.

View Full Article (HTML) Get PDF (565K)