Circularity and consistency in paired comparisons


Department of Psychology, Catholic University, Montessorilaan 3, 6500 He Nijmegen, The Netherlands.


Let R be a linear order on a set Z = {x, y,…} of n objects. Let Dm and D(t) be systems of paired comparisons on Z such that in every one of the inline image pairs with xRy the system Dm contains an observed proportion p = 0, 1/m,…, 1 of choices for x over y while D(t) contains a probability of choosing x over y as given by a choice theory (t) which also specifies R. For systems D1 a circularity index is proposed as an alternative to Kendall's and Slater's consistency indices. Some properties of these three indices are investigated in the sets of all possible D1 for some small n. If m > 1, Dm may be more or less compatible with D(t) and also give rise to a D1, estimating R. It is argued that the compatibility of Dm with D(t) may be regarded as an index of the external consistency of D1, with respect to R which is to be distinguished from its circularity or internal consistency. This distinction is corroborated by computer simulations of paired comparisons under Thurstone's case V model. The correlations between the three circularity indices and four indices of external consistency are essentially zero. Finally, the bearing of assessing circularity to the assessing and testing of the (in)transitivity of data systems is indicated.