Ten Cate et al commented on our article regarding the reliability of the assessment of synovitis by US in RA. As outlined in the review, several problems made interpretation difficult. First, testing of reliability was not part of the primary objective in a majority of studies and, therefore, incomplete descriptions of reliability testing methodology and statistics were inevitable. Second, there was clearly heterogeneity between studies, which explains why we did not pool the studies.
Ten Cate et al commented specifically on the use of ICCs to assess reliability. Indeed, we agree on the statistical properties of the ICC as outlined by Ten cate et al, which may lead to aberrant results when assessing reliability. In our study, we mainly reported the kappa statistic because it was the most frequently reported measure in the studies reviewed. However, ICCs were also reported in some cases. To our knowledge, there is no consensus as to which reliability statistic is superior (Lassere M, McQueen F, Ostergaard M, Conaghan P, Shnier R, Peterfy C, et al. OMERACT rheumatoid arthritis magnetic imaging studies. Exercise 3: an international multicentre reliability study using the RA-MRI score. J Rheumatol 2003;30:1366–75). The method suggested by ten Cate et al in assessing heterogeneity is certainly instructive for future reliability studies in US. In any case, more reliability studies are required, especially with respect to image-acquisition reliability.