A new permutation-based method for assessing agreement between two observers making replicated quantitative readings

The coefficient of individual equivalence is a permutation-based measure of agreement between two observers making replicated readings on each subject. It compares the observed disagreement between the observers to the expected disagreement under individual equivalence. Individual equivalence of observers requires that for every study subject, the conditional distributions of the readings of the observers given the subject's characteristics are identical. Therefore, under individual equivalence it does not matter which observer is making a particular reading on a given subject. We introduce both nonparametric and parametric methods to estimate the coefficient as well as its standard error. We compare the new coefficient with the coefficient of individual agreement and with the concordance correlation coefficient. We also evaluate the performance of the estimates of the new coefficient via simulations and illustrate this new approach using data from a study comparing two noninvasive techniques for measuring carotid stenosis to an invasive gold standard. Copyright © 2012 John Wiley & Sons, Ltd.