Assessing inter-rater reliability when the raters are fixed: Two concepts and two estimates


  • Valentin Rousson

    Corresponding author
    1. Statistical Unit, Institute for Social and Preventive Medicine, University Hospital Center Lausanne, Route de la Corniche 2, CH-1066 Epalinges, Switzerland
    • Phone: +41-21-314-73-28, Fax: +41-21-314-73-73
    Search for more papers by this author


Intraclass correlation (ICC) is an established tool to assess inter-rater reliability. In a seminal paper published in 1979, Shrout and Fleiss considered three statistical models for inter-rater reliability data with a balanced design. In their first two models, an infinite population of raters was considered, whereas in their third model, the raters in the sample were considered to be the whole population of raters. In the present paper, we show that the two distinct estimates of ICC developed for the first two models can both be applied to the third model and we discuss their different interpretations in this context.