Modelling covariate effects in observer agreement studies: The case of nominal scale agreement
Article first published online: 28 FEB 2007
Copyright © 1995 John Wiley & Sons, Ltd.
Statistics in Medicine
Volume 14, Issue 3, pages 299–310, 15 February 1995
How to Cite
Graham, P. (1995), Modelling covariate effects in observer agreement studies: The case of nominal scale agreement. Statist. Med., 14: 299–310. doi: 10.1002/sim.4780140308
- Issue published online: 28 FEB 2007
- Article first published online: 28 FEB 2007
- Manuscript Revised: APR 1994
- Manuscript Received: NOV 1993
Correction for chance-expected agreement has become an accepted technique in the analysis of observer agreement data and may be particularly useful when the level of agreement achieved in different populations is compared. However, formal methods for making comparisons of chance-corrected agreement or, more generally, for studying the effects of covariates on chance-corrected agreement have not received much attention. For nominal scale agreement data we show how Tanner and Young's model for observer agreement can be applied to this problem. The models discussed can be fitted using existing software and certain model parameters have interpretations in terms of positive and negative agreement odds ratios. The proposed methodology facilitates investigation of issues such as confounding of covariate effects and interaction between covariates in their effect on chance-corrected agreement. The methods outlined therefore allow observer agreement data to be analyzed in a manner strongly analogous to the logistic modelling of the association between disease and suspected risk factors. The methods are illustrated using data on the comparability of primary and proxy respondent reports of the primary respondents participation in physically vigorous leisure time activity.